Inferotemporal (IT) cortex plays a critical role in the primate ability to perceive and discriminate between images, but the relationship between responses of single neurons and behavioral capacities is poorly understood. We studied this relationship by recording from IT neurons while monkeys performed a delayed-match-to-sample task with two images. On each day, two sample images were chosen to maximize the selectivity of the neuron, and task difficulty was manipulated by varying sample duration and by masking the sample. On each trial, monkeys reported which of the two sample images was presented. Neural performance was described using an ideal observer analysis. Across the population, neural and behavioral sensitivity to changes in sample duration were indistinguishable. Neural sensitivity was dependent on epoch used to analyze neural response; maximal neural sensitivity was achieved in the 128 ms epoch that began 85 ms after sample onset. At most sample durations, the epoch that yielded optimal neural performance was longer than the sample duration, suggesting that neural selectivity persisted after the presentation of the mask during performance of the task. A control experiment showed that neural and behavioral performance improved in the absence of the mask. These observations suggest that the responses of individual IT neurons contain sufficient information to allow behavioral discrimination of images in a demanding task.