Voluntary behavior critically depends on attentional selection and short-term maintenance of perceptual information. Recent research suggests a tight coupling of both cognitive functions with visual processing being selectively enhanced by working memory representations. Here, we combined a memory-guided saccade paradigm (6 s delay) with a visual discrimination task, performed either 1500, 2500 or 3500 ms after presentation of the memory cue. Contrary to what can be expected from previous studies, our results show that memory of spatial cues can transiently delay speeded discrimination of stimuli presented at remembered locations. This effect was not observed in a control experiment without memory requirements. Furthermore, delayed discrimination was dependent on the strength of actual memory representations as reflected by accuracy of memory-guided saccades. We propose an active inhibitory mechanism that counteracts facilitating effects of spatial working memory, promoting flexible orienting to novel information during maintenance of spatial memoranda for intended actions. Inhibitory delay-period activity in prefrontal cortex is a likely source for this mechanism which may be mediated by prefronto-tectal projections.