This isn't much of an answer, so I'll put it in a comment. I don't think it really matters as long as it's obviously which parts are selected, both of which do very well. I would opt for the first option as it's more visually pleasing/easier on the eyes, but I consider them one in the same. Just as long as the highlighted colour contrasts enough with the text/background.
–
AnonymousJan 26 '12 at 16:11

Well, the second (XOR) is impossible to read. So that should answer your question. And to give you a lttle background: it exists because it is technically easy to program: it's just a matter of inverting every pixel. The first thing is harder as it requires to keep track of each pixel's original color.
–
User 1Jan 27 '12 at 13:03

1

As someone with mild colour vision deficiency, please don't use XOR for highlighting - so often the result is difficult to see (and remember there's more than just Red-Green colour deficiency too).
–
BevanJan 27 '12 at 23:42

3 Answers
3

If you can, you should leave it up to the user's operating system settings. OS X does a highlight effect by default, while the linux machine I'm on now (as well as Windows, if I remember right) does a bluish XOR.

As a user, it would definitely look strange to me if I was accustomed to my blue XOR effect and some application displayed a highlight effect (or vice versa). It would throw me off a second. Sure, I could figure out what was going on, but it's a visual anomaly with the rest of the system that could easily be avoided by either not overriding the effect, or inheriting from the system (if you must specify in a lower language).

If the background and text are OS colors, then the selection (and all other colors) MUST be too. If they are not, then all colors must be non-OS colors, other wise you may end up with background = foreground where one is OS default and the other is a custom color.
–
Danny VarodJan 26 '12 at 23:43

Always just shade the background and allow the text to be rendered over it.

Rendering the text first and then applying a bitwise XOR highlighted area over the top might appear to work ok superficially, but you cannot guarantee that the results are as effective as leaving the anti-aliasing to the font rendering engine. Consider for example what happens if the anti aliasing does not follow a linear drop off at the edges of the text, but a curved drop-off - by XORing it you invert the curve and end up with something slightly different to how the text would be rendered straight on to the background.

White on black is already pretty ugly in a black on white context...it just doesn't add unintentional, hideous colors into the mix.
–
Ben Brocka♦Jan 26 '12 at 17:49

If you shade the background, make sure to pay attention to foreground colors. A blue background tint is great until it completely obscures, e.g., hyperlink text. Some editors allow you to define both bg and fg colors for highlighted text (e.g., vi) when defining syntaxes.
–
Alex FeinmanJun 20 '13 at 14:06

Inverting colors can look bad, depending on colors and also the inversion of contrast of B/W <--> W/B can be uneasy to look at.

Changing foreground only makes it hard to tell if color change is due to selection or due to formatting.

Changing background only makes text loose its emphasis relative to its background.

That is why I prefer a different approach...

If the possible foreground and background colors are known in advance change them both slightly, e.g. white on black to yellowish-white on dark grey, black on white to medium blue on light grey.

If they are not known in advance then use a mathematical function e.g. slightly reduce contrast of all three channels of background and slightly reduce contrast of blue channel of foreground. This approach however won't work if the theme is low contrast.

By reduction of contrast of a single color I mean moving its value closer to the middle of the possible range.