Humans use differences in the timing of sounds at the two ears to determine the location of a sound source. Various models have been posited for the neural representation of these interaural time differences (ITDs). These models make opposing predictions about the lateralization of ITD processing in the human brain. The weighted-image model predicts that sounds leading in time at one ear activate maximally the opposite brain hemisphere for all values of ITD. In contrast, the -limit model assumes that ITDs beyond half the period of the stimulus centre-frequency are not explicitly encoded in the brain, and that such 'long' ITDs activate maximally the side of the brain to which the sound is heard. A previous neuro-imaging study revealed activity in the human inferior colliculus consistent with the -limit. Here we show that, cortical responses to sounds with ITDs within the -limit are in line with the predictions of both models. However contrary to the immediate predictions of both models, neural activation is bilateral for 'long' ITDs, despite these being perceived as clearly lateralized. Furthermore, processing of long ITDs leads to higher activation in cortex than processing of short ITDs. These data show that coding of ITD in cortex is fundamentally different from coding of ITD in brainstem. We discuss these results in the context of the two models.