There's a sense in which I'm not entirely sure we can legally order this. The thing is, || ( a b ) doesn't exactly mean "a or b". If a and b are both installed, || ( a b ) means "a and b", or possibly "a or b and I'm not telling you which"... This is very fiddly.

Well, I agree when a and b are both installed it makes sense to treat the dependency like that.
The situation described is a bit different, it is "a installed, b is not". And b depends on || (a b) must be giving the b the green light to build. Why is paludis enforcing self-dependency in this case?

It's enforcing it because || ( ) dependencies are squished down to a ( ) block early on (leaving alternatives in place for ordering leads to all sorts of crazy problems). The actual issue here is that the values being selected for that ( ) block aren't the ones you want...