To adapt to novel unstable environments, the motor system modulates limb stiffness to produce selective increases in arm stability (Franklin et al. 2007a, 2007b, Burdet & Osu et al. 2001). The motor system receives information about the environment via somatosensory and proprioceptive signals related to the perturbing forces, and visual signals indicating deviations from an expected hand trajectory. Here we investigated whether subjects modulate limb stiffness during adaptation to a purely visual perturbation. In a first experiment, measurements of limb stiffness were taken during adaptation to an elastic force field (EF). Observed changes in stiffness were consistent with previous reports (Franklin et al. 2007a): subjects increased limb stiffness and did so only in the direction of the environmental instability. In a second experiment, stiffness changes were measured during adaptation to a visual perturbing environment that magnified hand path deviations in the lateral direction. In contrast to the first experiment, subjects trained in this visual task showed no accompanying change in stiffness, despite reliable improvements in movement accuracy. These findings suggest that this sort of visual information alone may not be sufficient to engage neural systems for stiffness control, which may depend on sensory signals more directly related to perturbing forces, such as those arising from proprioception and somatosensation.