Now an explanation for this is published here (it’s based, as you might expect, on differences in contrast), but I can’t be arsed to read it. Those who have more patience than I are welcome to explain it below.

They said its the contrast – one box is much lighter and more reflective than the other – seems to me the black perpendicular bands accentuate this so when there the eye jumps back and forth between them perhaps so they appear to jiggle forward with the pale one jiggling first

I’m too lazy to read the explanation too, but my guess would be that it has to do with constructivism theory, about how we construct images through series of rapid eye movements. So you have to look at the yellow bar and then the blue one in rapid succession to track their movements. Somehow the bars must interfere with the brain piecing those two observations together.

No, it’s contrast making some areas of the bars stand out more. If you look at the GIF before it starts moving, with yellow & blue bars just nudging a black stripe on their right, the right-hand end of the blue bar (on a white stripe) stands out more than its left-hand end (on a black stripe), so its ‘visual centre of gravity’ is displaced to the right. Opposite for the yellow bar. So the blue bar appears in this static state to be to the right of the yellow bar.

When the ends of the bars are on the white background, the yellow is barely visible so it is hard to see it move. On the black background the blue bar is barely visible, so now it is hard to see that move. Dunno why the given explanation is so long and complicated!

… and a very quick scan of the linked paper confirms (if I read it aright) that Alan and my explanations are correct. Also that it’s contrast that causes it rather than ‘polarity’ (which I interpret as relative darkness).

The paper is complicated because it goes into a lot of substantiating detail, I guess.