Today's Little program shows how to read mouse input from
a console program.
You might use this if you are writing a console-mode text
editor with mouse support,
or maybe you want to want to add mouse support to your
roguelike game.

But I'm not going to implement the game itself.
Instead, I'm just going to print mouse coordinates to the
screen.

Remember, this is just a Little Program,
so there is little to no error checking.

Okay, now you can run the program,
and as you move the mouse around the window, you get...
Well, it depends.
Some of you may get output,
and others may get nothing.

Those of you who got nothing aren't getting anything
because you set Quick Edit mode on your console.
Quick Edit mode commandeers the mouse and uses it
for copy/paste operations rather than passing it through
to the application.
It's handy if you spend most of your time using programs
that don't use the mouse,
since it saves you from having to go to the
Edit menu all the time.

It's not so handy if you're running a program that actually
wants to use the mouse.

I'm not really knowledgeable about the subjects, but the answer is obviously "for compatibility".

Maybe some programs used to hard-code the bit-mask they wanted and set that directly, instead of going the "get current – modify – set" routine displayed here. That means whenever a previously unknown flag was added, those programs would accidentally disable them.

Do these flags get reset to default when the process quits or do they stick to the console window.

Probably some previous Windows version just used the first N bits of the DWORD for actual flags, leaving the rest unused. Those "free" bits were left garbage or even used as free storage by some crazy application, and to avoid breaking them you have the ENABLE_EXTENDED_FLAGS to keep the "official" usage of those bits as opt-in, without altering the crazy games those legacy apps played with them.

Still, I'm not really sure about this explaination, since SetConsoleMode is a relatively obscure API (as all console APIs, I'd say), so it seems too far fetched that applications "cleverly" exploited those "free bits".

As for other solutions, the WM folks would have solved the problem with a SetConsoleModeEx function (as in window styles/extended windows styles); the GDI team, instead, would have opted for ExtSetConsoleMode. :)

The answer to "Why does Windows do this thing in such a weird way?" is usually "Because doing it the obvious way would have broken some program."

Nothing is too obscure to be abused. If there are some free bits lying around, some programmer will use them. I can't find the blog post now, but I remember that Raymond once said that the Explorer team couldn't use the item data field in a header control because some shell extension was already using it to keep its own data.

I can speculate on the reason, but I believe that Matteo Italia already beat me to the punch. Based on his speculation regarding old apps that used those bits for their own strange purposes, he suggests creating a new function with extended capabilities to achieve this.

An alternative approach would be to create an application compatibility shim that emulates the old behavior and enable it for the applications in question. Any new development can happily ignore that this was ever an issue.

Given that my understanding of Windows compatibility is not quite up to your level, Raymond, how would you do it?

The ENABLE_INSERT_MODE and ENABLE_QUICK_EDIT_MODE flags were introduced in a later version of Windows, where all other flags were already specified and applications were already setting the console mode with an OR'ed mask (*_INPUT | *_INPUT | …).

Existing applications would have annoying behavior if they enabled or disabled these modes when that was not the original intention. Note that even the suffix on the names are different.

What I can't explain is why the quick-edit mode takes precedence on enabling mouse input. We're talking about Win32 native console applications, not an old DOS application that uses the mouse. But I reckon this is an issue where you just can't please everyone, and where "Yet Another Checkbox"™ that asks the user what prevails if both flags are set would seem really confusing and/or stupid to almost everyone.

Gotch'ya Ray!! You did what you chanted for years => "Do not overwrite user settings with your superawesomemegaextra program". Now, why you overwrite user settings regarding Quick Mode? Maybe he wants to use the mouse in Quick Edit mode on all his console programs and is your burden to give him mouse support without messing with it. How do I do it? Well, I'm a big fan of hooks, but there are other ways as well. Anyway, I expect that star!!!

In this case, it would be fine because it makes sense. It is the same sort of tale as games choosing to change the screen resolution, which was especially important back in the day when 256 color and specific resolutions were pretty much hammered into place due to game code generally not having the leeway to be flexible, assuming that a customers computer was even able to use 16-bit or 24-bit color. For games, it makes no sense to ask the user 'hey, please change your settings to mode X with Y colours at Z hertz refresh rate'.

Anyhow, back to the point. The user runs an application, and that application would appear to be designed to be controlled with a mouse. Ignoring the fact that it is odd to have a strongly mouse-driven application in a console window, it is completely weird for an application to be signaling 'click me' and the user is unable to do that. They'll think the application is broken, maybe not even remembering that they have the Quick Edit setting turned on. Additionally, in those mouse-driven console applications where Quick Edit is indeed turned on, there is often a fair bit User Interface chrome (I'm thinking Q(uick)Basic-style interfaces atm). This is generally not of a very copy-pastable nature and work counter-intuitive to what a user expects.

My only real curiousity is about the way the console window behaves with regards to an application shutting down. Does it return the settings to what they were? (I am pretty sure modern Windows versions have started to babysit old games / buggy applications that (used to) wreck the screen resolution to try and help prevent it from happening.) The console window and setting is a shared resource, so it might make sense for it to stick around, but it also makes no sense to expect applications to be baby-sitting their console window flags every time they launch another application. Whatever the implementation is, I suspect it is similar to what happens to have been the first public implementation… backwards compatibility and all. :)

Maybe, in some past version of Windows, it went something like: bits 0-9 are flags, bit 10 is reserved, bits 11-31 are unused. So developers used bits 12-31 for their own purposes and the reserved bit turned into ENABLE_EXTENDED_FLAGS. Of course this relies on developers not also stomping on the reserved bit.

To "do the same thing another way" – add a bitmask field, that tells the API which bits are actually to be considered valid in the bitset field. Old software wouldn't use the new API, new ones would explicitly say which bits were being twiddled.

[That's what I had in mind. That way we won't have to add a ENABLE_MORE_EXTENDED_FLAGS in the future. -Raymond]