Smooth pursuit eye movements allow the approximate stabilization of a moving visual target on the retina. To study the dynamics of smooth pursuit we measured eye velocity during the visual tracking of a Gabor target moving at a constant velocity plus a noisy perturbation term. The optimal linear filter linking fluctuations in target velocity to evoked fluctuations in eye velocity was computed. These filters predicted eye velocity to novel stimuli in the 0-15 Hz band with good accuracy, demonstrating that pursuit maintenance is approximately linear. The shape of the filters were indicative of fast dynamics, with pure delays of merely ~67 ms, times-to-peak of ~115 ms, and effective integration times of ~45 ms. The gain of the system, reflected in the amplitude of the filters, was inversely proportional to the size of the velocity fluctuations and independent of the target mean speed. A modest slow-down of the dynamics was observed as the contrast of the target decreased. Finally, the temporal filters recovered during fixation and pursuit were nearly identical in shape, supporting the notion that they might share a common underlying circuitry. These findings show that the visual tracking of moving objects by the human eye includes a reflexive-like pathway with high contrast sensitivity and fast dynamics.