Psychophysical findings have revealed a functional segregation of processing for first-order motion (movement of luminance modulation) and second-order motion (e.g. movement of contrast modulation). However neural correlates of this psychophysical distinction remain controversial. To test for a corresponding anatomical segregation, we conducted a new fMRI study to localize direction-selective cortical mechanisms for first- and second-order motion stimuli, by measuring direction-contingent response changes induced by motion adaptation, with deliberate control of attention. The second-order motion stimulus generated direction-selective adaptation in a wide range of visual cortical areas, including areas V1, V2, V3, VP, V3A, V4v and MT+. Moreover, the pattern of activity was similar to that obtained with first-order motion stimuli. Contrary to expectations from psychophysics, these results suggest that in the human visual cortex, the direction of second-order motion is represented as early as V1. In addition, we found no obvious anatomical segregation in the neural substrates for first- and second-order motion processing which can be resolved using standard fMRI.