In order to track an object of interest that moves across a dynamic visual context, motion signals from target and context have to be processed. Physiological and behavioral studies provide evidence for at least two different strategies for motion processing that are reflected in the neuronal or behavioral response: averaging absolute target and context motion signals (motion assimilation), or calculating relative motion between target and context (motion contrast). Here, we show that perceived velocity and pursuit velocity can follow different motion cues. Eye movements were recorded from seven naïve human observers to a medium contrast Gaussian dot that moved horizontally at 11.3 deg/s. A peripheral context, consisting of two vertically oriented sinusoidal gratings, one above and one below the stimulus trajectory, drifted into the same direction as the target at the same speed. During the steady-state phase of the pursuit eye movement, target and context moved continuously at 11.3 deg/s, or were independently perturbed for 250 ms to move slower (5.6, 8.5 deg/s) or faster (14.1, 16.9 deg/s). Observers were asked to smoothly track the target, and to indicate whether the target velocity had increased or decreased. In all observers, eye velocity transiently increased or decreased to the velocity vector average for an increase or decrease in context velocity, time-locked to perturbation onset. In contrast, psychophysical responses followed relative motion between target and context. When the context was perturbed to move slower, target velocity was overestimated, and when the context moved faster during perturbation, target velocity was underestimated. We conclude that motion signals are processed in different ways for perception and pursuit. In the presence of a moving context, perceived velocity is driven by relative motion, and pursuit velocity is driven by a motion average.