Images on the retina can change drastically in only a few milliseconds. A robust description of visual temporal processing is therefore necessary to understand visual analysis in the real world. To this end, we investigated sub-second visual changes and asked how prefrontal neurons in monkeys respond to stimuli presented in quick succession. We recorded the visual responses of single neurons in the frontal eye field (FEF), a prefrontal area polysynaptically removed from the retina that is involved with higher level cognition. For comparison, we also recorded from small groups of neurons in the superficial superior colliculus (supSC), an area that receives direct retinal input. Two sequential flashes of light at varying interstimulus intervals were presented in a neuron's receptive field. We found pervasive neuronal adaptation in FEF and supSC. Visual responses to the second stimulus were diminished for up to half a second after the first stimulus presentation. Adaptation required a similar amount of time to return to full responsiveness in both structures, but there was significantly more neuronal adaptation overall in FEF. Adaptation was not affected by saccades, although visual responses to single stimuli were transiently suppressed postsaccadically. Our FEF and supSC results systematically document sub-second visual adaptation in prefrontal cortex and demonstrate that this adaptation is comparable to, but stronger than, adaptation found earlier in the visual system.