Attractive faces can be considered to be a form of visual reward. Previous imaging studies have reported activity in reward structures including orbitofrontal cortex and nucleus accumbens during presentation of attractive faces. Given that these stimuli appear to act as rewards, we set out to explore whether it was possible to establish conditioning in human subjects by pairing presentation of arbitrary affectively neutral stimuli with subsequent presentation of attractive and unattractive faces. Furthermore, we scanned human subjects with fMRI while they underwent this conditioning procedure in order to determine whether a reward prediction error signal is engaged during learning with attractive faces, as is known to be the case for learning with other types of reward such as juice and money. Subjects showed conditioning-related changes in behavioral ratings to the CS stimuli, notably for those CSs paired with attractive female faces. We used a Rescorla-Wagner learning rule to generate a reward prediction error signal, entered as a regressor in our fMRI analysis. We found significant prediction error-related activity in the ventral striatum during conditioning with attractive compared to unattractive faces. These findings suggest that an arbitrary stimulus can acquire conditioned value by being paired with pleasant visual stimuli just as with other types of reward such as money or juice. The findings we describe here may provide insights into the neural mechanisms tapped into by advertisers seeking to influence behavioral preferences by repeatedly exposing consumers to simple associations between products and rewarding visual stimuli such as pretty faces.