If “Remove Brainwashing” is active and a monster equipped with “Vengeful Servant” switches sides, that monster immediately switches back to the original controller. The effect of “Vengeful Servant” then starts a chain (and only activates once, in this case) and the original controller of the monster takes damage.[1]

OCG Rulings

When "Remove Brainwashing" is on the field while a monster is equipped with "Vengeful Servant", if the equipped monster changes control, it will immediately switch back before the effect of "Vengeful Servant" can activate. Afterwards, "Vengeful Servant" will activate once, and inflict damage to the equipped monster's current controller: its original owner.[4]