Saturday, 18 March 2006

Subtle interaction between WM_DESTROY handler and OnFinalMessage in ATL for CE

There’s a subtle issue around handling WM_DESTROY in an ATL window and also overriding OnFinalMessage, if using the ATL windowing classes on Windows CE.

Typically, you would override OnFinalMessage if you wanted to do some final cleanup when the window has been destroyed and no further messages will be received. The canonical example – given in the documentation – is automatically deleting the object that manages the window using delete this.

On the desktop, OnFinalMessage is called by the CWindowImplBaseT::WindowProc function after the WM_NCDESTROY message is received. Since Windows CE top-level windows don’t have a non-client area, the non-client messages were removed from the platform. The Windows CE version of ATL therefore uses WM_DESTROY instead.

The code inside WindowProc only cleans up the window proc thunks (dynamic objects containing code used to patch between Windows’ expectation of a flat API, and a call to a C++ object member function) and calls OnFinalMessage if the message (WM_DESTROY or WM_NCDESTROY as appropriate) is not handled. This means it will work fine if you don’t override WM_DESTROY, but what if you need to do some cleanup here?

The last parameter of a message handling function in ATL is a BOOL reference typically named bHandled. This is set to TRUE by the message map macros (ATL message map macros generate a big if statement, rather than being table-driven like MFC) on calling the message handling function. If the value is still TRUE when the function returns, the message map considers it handled and stops processing; if it’s FALSE the macro code keeps looking for a handler.

If you handle WM_DESTROY on CE using ATL, you must set bHandled to FALSE, otherwise the window will never be cleaned up, and OnFinalMessage will not be called. The same is true if you handle WM_NCDESTROY on the desktop, but since this message is rarely handled, it’s less of an issue.

This issue was responsible for me leaking 96 bytes in a program every time an activity was completed, which doesn’t sound like a lot, but any long-running application needs to be free of leaks as far as possible. This was actually a Compact Framework program for the most part – this window implemented a signature capture control, which is actually pretty hard to do in CF 1.0, and we already had working C++ code (excepting leaks…). For some strange reason, though, a leak of 96 bytes here was causing an overall leak of over 300KB of virtual memory! Removing this leak has got it down to about 160KB but I’m stumped on where the rest of it’s going – the amount of VM allocated increases by 160KB whenever the form containing the control is created, but that isn’t returned when the form is disposed. A C++ program containing the control doesn’t exhibit the same behaviour.

Compact Framework seems to do wacky things to the process default heap, which usually means that you can’t actually view it with Remote Heap Walker – it aborts after several seconds of trying to walk the heap. To find this leak I created a non-default heap, overrode operator new and operator delete to allocate and free from this alternative heap, then viewed that.