Don’t be helpless: I don’t know anything about MFC modal loops, but unlike some people, I’m not afraid to find out

If I have a modal CDialog that is visible and usable to the user.
Let's say I receive an event somewhere else in the program
and I call DestroyWindow on the modal CDialog from within the event.
I notice that the OnDestroy is called on the CDialog,
but DoModal never exits until a WM_QUIT is posted to the modal's message pump.
What are the pitfalls to this?
Unfortunately, there is really no way to avoid this situation.

I'm not sure what the question is, actually.
The question as stated is "What are the pitfalls to this?"
but he answered that in his own question:
The pitfall is that "DoModal never exits until a WM_QUIT is
posted to the modal dialog's message pump."

I'm going to assume that the question really is,
"Why doesn't destroying the window work?"
with the follow-up question,
"What is the correct way to dismiss a modal dialog?"

The first problem with this question is that it assumes that I know
what a CDialog is.
From its name, I'm going to assume that this
is an MFC class for managing a dialog box.
But you don't even have to know that to answer the first
reformulated question operating only from Win32 principles:
DestroyWindow is not how you exit a modal dialog.
You exit a modal dialog with EndDialog.
The DestroyWindow technique is for modeless
dialogs.

But let's look at the question another way,
which is my point for today:
You have the MFC source code.
Don't be afraid to read it.
Especially since I don't use MFC personally;
I don't even know the basic principles of application design with MFC.
I work in straight Win32.
As a result,
I don't know the answer off the top of my
head, but fifteen minutes reading the MFC source code quickly reveals
the reason why destroying the window doesn't work.

Watch me as I go and find out the answer.
It's nothing you can't already do yourself.

The
CDialog::DoModal method
calls CWnd::RunModalLoop to run the dialog loop.
If you look at CWnd::RunModalLoop,
you can see the conditions under which it will exit the modal loop.
Here's the code with irrelevant details deleted.
(They're irrelevant because they have nothing to do with how the
modal loop exits.)

There are only two ways out of this loop.
The first is the receipt of a WM_QUIT message.
The second is if CWnd::ContinueModal decides that
the modal loop is finished.
The commenter already mentioned the quit message aspect to the
modal loop, so that just leaves CWnd::ContinueModal.

The CWnd::ContinueModal method is very simple:

BOOL CWnd::ContinueModal()
{
return m_nFlags & WF_CONTINUEMODAL;
}

Therefore, the only other way the loop can exit is if somebody
clears the WF_CONTINUEMODAL flag.
A little grepping shows that there are only three places where this
flag is cleared.
One is in CPropertyPage, which is a derived class
of CDialog and therefore isn't relevant here.
(I'll ignore CPropertyPage in future searches.)
The second is in the line above right after the label
ExitModal.
And the third is this method:

void CWnd::EndModalLoop(int nResult)
{
// this result will be returned from CWnd::RunModalLoop
m_nModalResult = nResult;
// make sure a message goes through to exit the modal loop
if (m_nFlags & WF_CONTINUEMODAL)
{
m_nFlags &= ~WF_CONTINUEMODAL;
PostMessage(WM_NULL);
}
}

Following the money one last step,
the CDialog::EndDialog method is called
from four places in CDialog.
It's called from CDialog::HandleInitDialog and
CDialog::InitDialog if some catastrophic error
occurs during dialog initialization.
And it's called from CDialog::OnOK
and CDialog::OnCancel in response to the
user clicking the OK or Cancel buttons.

Notice that the CDialog::EndDialog method is not
called when somebody forcibly destroys the dialog from
the outside.

That's why destroying the dialog window doesn't break the modal loop.
If you want to break out of the modal loop, your only choices are
to post a quit message or call CWnd::EndModalLoop,
either directly or indirectly (via CDialog::EndDialog,
for example).

Notice that the MFC modal loop obeys the convention on quit messages
by re-posting the quit message when it breaks out of the modal loop.
(Though it really should have posted the wParam from
the quit message rather than just posting zero.)

The workaround therefore is not to destroy the dialog with
DestroyWindow (something you should have known
not to do a priori since that's not how you exit
modal dialog boxes) but rather by calling
CDialog::EndDialog, passing a result code that
lets the caller of CDialog::DoModal know that
the dialog box exited under unusual circumstances.

This took me fifteen minutes to research and a little over an hour to
write up.
All this work to answer a question that you should have been able
to answer yourself with a little elbow grease.
You're a smart person.
Have confidence in yourself.
You can do it.
I know you can.

One might add that the MFC Reference (help) for the CDialog class states, "A modal dialog box closes automatically when the user presses the OK or Cancel buttons or when your code calls the EndDialog member function."

Oops, point being that reading the help answers the question without the need to dive into the source code. It not only tells you to call EndDialog but also directs you toward the OnOk and OnCancel member functions with a whack from a pretty strong cluestick.

It illustrates the point well – so many people just act helpless and immediately want help without even doing the basics to research the problem. People seem to think I have a psychic ability to solve problems and find out the cause of error messages, when in fact a lot of the time I simply have a non-psychic ability to copy and paste the error message into google then read the top few most likely looking results.

I, for one, just acted helpless back in the day. I also often "cargo-culted" (something didn’t work for some bug I didn’t understood, so I don’t use that something anymore because I believe it’s something which simply doesn’t work or things like that).

What changed me was a sum of experience and gained self-esteem.

Help from colleagues and boss was a key factor in this, between other factors.

johnf: Reflector gives you Windows Forms (the Microsoft-supplied .NET Framework DLLs are _not_ obfuscated), while for the BCL and the runtime, there’s the Shared Source CLI (Rotor). You can use Reflector on the BCL as well, of course, but SSCLI has the original comments in.

You cannot of course copy that code yourself, but you can understand what it’s doing.

I’m pretty sure the subject of this article is dealt with in every MFC text *ever*. If you’re not comfortable just grepping (or other tool of your choice) the MFC source, there’s a book called "MFC Internals" by Scot Wingo and George Shepherd which will guide you through it. It’s ‘out of date’ in that it targets MFC 4.0, but the architecture hasn’t actually changed all that much in 10 years.

MFC’s dialogs are actually a little odd in that an MFC ‘modal’ dialog is actually a modeless Win32 dialog. There’s some very good reason for this which I’ve forgotten (and I can’t be bothered to dig out the book I mentioned above to look it up).

I haven’t done enough testing, but have the impression that the non-modality of MFC pseudo-modal dialogs is the reason why WM_CLOSE can be received by an MFC application where a corresponding plain Win32 application never gets the WM_CLOSE (turning the Win32 app into a "not responding" app).

Even with the source code, this person

couldn’t be bothered.

Maybe that, or maybe he didn’t know that there’s an option to install it during Visual Studio installation. Meanwhile there are other people who well could be bothered, who have found some parts of the source readable, who agree with johnf’s opinion.

Source code to Windows Forms would be nice, if only so Go To Definition went somewhere interesting. The documentation tends to be good enough for .NET while the source for MFC is basically required to solve many issues. I prefer plain old Win32 API calls to MFC.

> The documentation tends to be good enough for .NET while the source for MFC is basically required to solve many issues. I prefer plain old Win32 API calls to MFC.

I agree with you, but I’d have thought that an API where you are called, is always going to be way more complex to program, than one where you do all the calling. It’s all part of the idea of encapsulation, and black vs white boxes.

Clearly there are practical constraints in what can be achieved if one goes bonkers and says, "I’ll only implement a call based API" or vice versa.

At the end of the day, you’re a very lucky person, if you never write code that is called in somebody elses context.

I’ve never used .NET, what is it? :)

I never liked MFC, because it led me up the path to nowhere so, so, many times. Win32 never, ever, has.

Part of the information is there, but with all the comments and local variable names gone, it’s a lot harder to understand than it has to be.

Take the Dialog example above: In MFC, you could just put a breakpoint on DoModal, step into it, and see what it does, in your program, look at the values of all local variables, you can see which branch it takes, see values of local variables and everything. All you have to do is hit F11 a few times. With .NET, this is _a lot_ harder, especially if you have to look into ROTOR.

BTW: I’d really like to understand why MS *didn’t* release the .NET source code. They’ve released MFC, ATL in the past, so what’s the big deal about .NET? Are they afraid someone might steal their intellectual property? Then why didn’t they obfuscate?

Jumping into source is a last resort for figuring out why something doesn’t work, but I’ll agree it’s a useful approach. The only way I figured out why my window creation code was crashing and burning when I called CreateWindowEx with an extended style of WS_EX_MDICHILD was by reading the Wine source code, for instance. I note that the strange behaviour is documented now. It definitely wasn’t when I was working on this.