What am I doing to inadvertently invoke the Color Samplers in Photoshop CS6's Info palette?

I never use the four Color Samplers in Photoshop CS6's Info Palette. Yet they keep popping up during my normal workflow (I rely heavily on key commands to switch tools, show and hide interface elements, etc.). Is there a key command I'm accidentally hitting to invoke these Color Samplers? And if so, is there a way to disable it? I keep having to toggle them off in the Info palette.

If you set color samplers in a document, they are remembered as part of the document.

Otherwise I'd guess that you switched to the color sampler tool by mistake when you were trying to get the eyedropper, or held down SHIFT while using the eyedropper tool (which makes it put down a color sampler).