Mike Dimmick's Bleurgh

Friday, 5 March 2010

As one of those people who only have a hammer :-( ... It looks like MFC just told us off with this?
WM_GETDLGCODE has meaningfull wParam and lParam, but CWnd::OnGetDlgCode has none of that.
Anyone used OnGetDlgCode? Help?

The specific context is handling the return key - Windows tells you what key caused WM_GETDLGCODE to be fired in the wParam parameter, but MFC doesn't pass that information on to the OnGetDlgCode handler.

Something that often catches people out with MFC is the thought that you have to use the specific handler that MFC declares for a certain message. In fact, it's simply a shortcut - ON_WM_GETDLGCODE in the message map causes the message map handling to go down a certain branch that happens to call the handler function with no parameters.

Instead, you can simply use ON_MESSAGE in the message map, using WM_GETDLGCODE and declaring the function as taking WPARAM, LPARAM and returning LRESULT. Do make sure you don't also include ON_WM_GETDLGCODE.

The message map macros themselves simply produce a table of message number ranges and handler function pointers. The MFC WindowProc scans the table to find a handler for a given message number, then - based on an enumeration value set by the message map macro which indicates the argument types - decodes wParam and lParam, then calls the handler with the appropriate arguments.

Many of the predefined helper message map macros, which usually don't take any arguments, define the function name that you're expected to implement. ON_MESSAGE allows you to specify the function name (as you'd expect, as a generic macro).

The Visual Studio Wizards have always simplified the full power of message maps, not supporting custom message numbers (which you have to use ON_MESSAGE for) and not supporting ranges of command IDs (ON_COMMAND_RANGE, ON_NOTIFY_RANGE, etc). Few people even seem to realise that you can use the same handler function for more than one message map macro, for example if you have discontinuous ranges of control IDs.

Tuesday, 13 October 2009

…that’s how long I was running Windows 7 RC1 without switching back to my hard disk with Windows Vista. The System event log shows I last booted it on 31 May 2009.

This is practically the last action I’m going to take before wiping this disk – the Vista disk – and installing Windows 7 RTM. Just a little check here and there to find anything that I might not have backed up when transferring over to 7 RC1, then it’s formatting time.

The only thing I’ve missed? The extra 200GB of disk space!

In retrospect Windows Vista has not been a bad operating system, but it had some pretty rough edges, particularly at the start with poor hardware support. It was a rush 18–month project, after the initial three-year disaster of Longhorn, and in some areas it shows. Windows 7 has had a further two-and-a-half years to bake, and a further two-and-a-half years of device driver and hardware development; it’s not fundamentally different but feels more complete.

Saturday, 11 July 2009

I thought I’d break this into another post so that the fix is easier to understand for non-technical people.

Comparing the old and new driver source shows that the definition of the libusb_request structure had changed. Visual inspection suggested that this was probably the source of the compatibility issue, when using the old DLL with the new driver.

When using the new DLL, the following error occurs: “The ordinal 76 could not be located in the dynamic link library libusb0.dll.” Inspecting the program and DLL shows that while the DLL’s entry points are exported by name, the program links to the DLL by ordinal. This is peculiar; while it’s a little faster than string-matching, and it saves a little space, care has to be taken to keep the ordinals the same between versions of a DLL. Reading the .def file shows that the ordinals weren’t specified, and that numerous functions were added and removed between the two versions.

My fix is to add specific ordinal declarations to the .def file for functions that humax.exe is using, so that the correct functions are used. I did this, and recompiled the DLL with the Windows Vista WDK (Windows Driver Kit). Success! Or, at least, as much success as I’ve had with Windows XP 32–bit; the box still has a tendency to hang during file transfer.

The Humax PVR9200T is a pretty good Freeview/digital terrestrial personal video recorder. I’ve had mine for a few years now. Over time, the disk fills up with some programmes you want to keep. It’s possible to copy recorded programmes from the box to your computer, over USB.

Unfortunately the supplied eLinker software is not highly-regarded. As a result some replacements have been written, such as Humax Media Controller. However, the driver supplied only works on 32–bit Windows.

The USB library and driver used are a Win32 port of libusb, a driver which makes it possible to write your USB transfers fully in user-mode. A later version (0.1.12.2) has both 32– and 64–bit versions of the driver. However, the updated driver doesn’t work with the libusb0.dll supplied with Media Controller 1.05, and the program won’t run if you overwrite it with the new DLL.

I’ve rebuilt the new DLL so it will work with the program. The attached ZIP file contains the new version and the new drivers. To use, extract the contents of the folder. Copy the libusb0.dll in the top-level folder to the Humax Media Controller folder, overwriting the current version. Then, use Update Driver in Device Manager to install the driver from the Driver folder.

Windows Vista introduced a driver signing requirement for x64, and this driver isn’t signed. To get it to load on Windows Vista and Windows 7, reboot, and press F8 until the boot menu appears. From this menu, select Disable Driver Signature Enforcement, then continue to boot normally.

Hopefully this will help other people with the x64 versions of Windows!

Wednesday, 30 July 2008

OK, I think my router has become too hackable. The reliability has hit the toilet, and I’ve had two messages from my ISP warning me that there has been odd traffic. The first time I got a warning message I think I was using BitTorrent at the time, but the second was yesterday at around lunchtime – when my computers were all off and I was at work.

So, list of demands requirements:

Wired router with a few Ethernet ports

802.11g WiFi with at least WPA encryption (can’t do WPA2 on the handheld)

Xbox Live compatible

DHCP implementation that works

DNS server address passthrough from DHCP server (i.e. I want to use my ISP’s DNS servers, not have the box do DNS for me)

ADSL2+ compatible, for whenever that gets rolled out by Demon

PPPoA since this is of course a UK requirement

A firewall that can be completely switched off or is actually configurable intelligibly

Not excessively burdened with blinkenlights

Will be updated regularly with patches for whatever its OS is

That last point is critical. There are way too many routers out there that run Linux and aren’t patched. D-Link haven’t produced an update since last November and even that’s not officially official, you have to find it on the FTP site. The last official update is September 2006.

I don’t need 802.11n as a) it’s still a draft, b) none of my other equipment needs it, and c) 54Mbps is already sufficiently fast to access the Internet. I don’t do much copying between devices and I wouldn’t do it when they were running on battery anyway.

Dump this snippet into Notepad, save as simple.cs file and compile it. We want to see what it’ll be like when the user runs it, so we’ll do a release build. From the VS2005 Command Prompt, run csc /debug:pdbonly /o+ simple.cs. (I find it’s quicker to do command-line compiles for simple test programs than firing up Visual Studio. It’s not as if we’re designing a Form.) If you’re doing this on a 64–bit computer, like me, add /platform:x86 so we get consistent output across platforms. (Marking as x86 means that the executable headers are set to indicate a 32–bit program. The IL is the same whatever you set here, but if you leave it as anycpu, the default, the .NET Framework will create a 64–bit process which obviously gives different output.)

Why call Console.ReadLine? Well, the JIT compiler is helpful (or at least it thinks it is). If you start a program under the debugger, it will generate less optimized code because it thinks you want to debug it, but that changes the code from what the user will see. So I want to add a stop point in the program so I can attach the debugger after the process has started.

The next thing you’ll need is the Debugging Tools for Windows kit. Grab the 32–bit kit – the 64–bit kit can debug 32–bit processes, but SOS (the debugger extension DLL which implements the commands we’re going to use) doesn’t appear to work. We’re going to use WinDBG, which is slightly friendlier than the other debuggers although not a lot!

Open WinDBG. Run simple.exe and, when it’s waiting for input, go to File/Attach to a Process in WinDBG. You’ll see a list of other processes (and if running Windows Vista with UAC enabled, or on XP as a standard user, a load of access denied errors). Select simple.exe from the list and hit OK. WinDBG automatically stops once you’ve attached to a process so you can start manipulating the program straight away. This is the output I got:

You enter your debugging commands in the edit box at the bottom, next to the prompt 0:003>. This means we’re debugging the 0th process we’re attached to, and our commands by default apply to thread number 3. Four threads? Well, thread 0 is our main thread, thread 1 was created by the CLR’s debugging support in case we attached a debugger, thread 2 is the finalizer thread, and thread 3 was just created by WinDBG so it could stop the process safely. (You can see this by running ~* k, although you’ll need to be set up to get debugging symbols from the symbol server to get good stack traces.) The Debugging Tools debuggers automatically stop the process when you attach, so that you can start manipulating the process straight away.

The first thing we need to do is ask WinDBG to load the SOS extension. This is installed with the CLR, so we ask it to load from the same folder that mscorwks.dll (the DLL which implements the virtual machine, effectively the guts of the CLR) lives in:

0:003> .loadby sos.dll mscorwks

Now we can see what state of the managed threads are in by running !threads. Any command beginning ! comes from a debugger extension DLL, and they can be disambiguated if necessary, but they’re searched in reverse order loaded (last loaded = first searched) so anything we want will be found in SOS anyway.

0:003> !help U
-------------------------------------------------------------------------------
!U [-gcinfo] [-ehinfo] <MethodDesc address> | <Code address>
Presents an annotated disassembly of a managed method when given a MethodDesc
pointer for the method, or a code address within the method body. Unlike the
debugger "U" function, the entire method from start to finish is printed,
with annotations that convert metadata tokens to names.
...
03ef015d b901000000 mov ecx,0x1
03ef0162 ff156477a25b call dword ptr [mscorlib_dll+0x3c7764 (5ba27764)] (System.Console.InitializeStdOutError(Boolean), mdToken: 06000713)
03ef0168 a17c20a701 mov eax,[01a7207c] (Object: SyncTextWriter)
03ef016d 89442414 mov [esp+0x14],eax
If you pass the -gcinfo flag, you'll get inline display of the GCInfo for
the method. You can also obtain this information with the !GCInfo command.
If you pass the -ehinfo flag, you'll get inline display of exception info
for the method. (Beginning and end of try/finally/catch handlers, etc.).
You can also obtain this information with the !EHInfo command.

Right, so we need the address of a MethodDesc structure, or the address of the code itself. How can we get one of these? There’s a command called DumpMD…

0:003> !help dumpmd
-------------------------------------------------------------------------------
!DumpMD
This command lists information about a MethodDesc. You can use !IP2MD to turn
a code address in a managed function into a MethodDesc:
0:000> !dumpmd 902f40
Method Name: Mainy.Main()
Class: 03ee1424
MethodTable: 009032d8
mdToken: 0600000d
Module: 001caa78
IsJitted: yes
m_CodeOrIL: 03ef00b8
If IsJitted is "yes," you can run !U on the m_CodeOrIL pointer to see a
disassembly of the JITTED code. You can also call !DumpClass, !DumpMT,
!DumpModule on the Class, MethodTable and Module fields above.

Finally we have something we can pass to !U. Note the JIT column. Here, PreJIT means that the code has come from a native image generated by ngen, JIT means that the JIT compiler in this process has generated the code, and NONE means that the method hasn’t been compiled yet. We didn’t override any of System.Object’s virtual methods, so they occupy the first four places in our MethodTable.

I placed the call to Console.ReadLine at the end of the program so everything has already been compiled – remember, the JIT compiles code on-demand, one method at a time, when that method is called (barring very simple methods which might be inlined into their callers). Let’s just get on and show the code for Main.

Interesting. The JIT clearly sees Console.WriteLine(string) and Console.ReadLine as trivial and has inlined the calls. In turn it’s inlined Console.get_Out. The reference to SyncTextWriter isn’t the JIT being clever – it’s SOS telling us that the address 0x0392108C is the base of SyncTextWriter’s vtable, as this call to TextWriter.WriteLine(string) is virtual.

I realise, and hope, that this isn’t everyday usage. More often than not you’ll have crashed somewhere and want to find out where (for which see !IP2MD). But it can be useful to see just how your C# code turns into machine instructions.

I was looking at what the .NET x64 JIT compiler generates for some code, and saw something very odd at the end of the routine: the last instruction of the function was rep ret. Looking a bit further, this is the same at the end of every JIT-compiled routine.

The thing is, the rep prefix to an instruction is supposed to tell it to be repeated. Repeat the return? How do I do that?

The Intel architecture software developer’s manual set says it’s only defined for the ‘string’ instructions like movs (which moves a byte/word/dword from the address pointed to by ESI to the address pointed to by EDI). The rep prefix repeats the string instruction ECX times. Yes, this means that you can implement memcpy in a single instruction. (You can do memset with a single rep stos instruction, once AL is loaded with the value to be stored.) It’s explicitly undefined for anything else.

So where the heck has this illegal usage come from? I followed a couple of clues and found this patch notification for glibc on x64. And indeed the current version of AMD’s optimization guide [PDF] for Athlon 64 processors says that you should do this. The reason? The branch predictor gets it wrong if the ret instruction is jumped to directly by a branch instruction, or if the ret directly follows a branch instruction.

I’m not sure doing it throughout, even when you’ve got epilog code in there which prevents the bug, is necessary though.

AMD have now published a new optimization guide for their Family 10h processors [PDF] and guess what, the advice has changed. Instead of using a two-byte illegal instruction, they now recommend the three-byte instruction ret 0. The difference between a plain ret and a retimm16 (where imm16 is an immediate 16–bit value) is that retimm16 pops the return address, then specified number of bytes from the stack before jumping to the return address. It’s common to see this in 32–bit Windows WINAPI (__stdcall) code as this calling convention requires the called function to clean up the parameters from the stack. (64–bit Windows has only one calling convention and it mainly passes parameters in registers, so stack cleanup is not required.

Still, it’s a shame to see the JIT generating this on my Core 2 Duo laptop, which doesn’t have the bug (as far as I know, but there’s no mention in Intel’s optimization guide). And it’s an even bigger shame on AMD that they a) didn’t fix the damn bug in the processor and b) recommended an illegal instruction as a way round it.

Sunday, 13 January 2008

Recently, at the end of September, I got a new neighbour in the flat downstairs. The property is an end terrace, split into two flats in 1999, but the landlady never did a great job of separating the services out (electricity and gas usage is included in the meter counts downstairs, there isn’t a separate supply). When I moved in, the landlady lived downstairs; she allowed a previous tenant of this flat to put in extension wiring to use her old BT landline, as she’d got cable service with NTL (now Virgin Cable, of course), so I was using the BT line. After leaving, other tenants of the downstairs flat had simply taken over the cable service.

When the new bloke moved in downstairs, he wanted a BT service. He claims that he asked them for a new line – they interpreted this to mean taking over the old one, and started the process of transferring my line to him. First I knew of this was a letter dated 16 October saying “we understand you no longer live at the above address and so we’ll disconnect the phone on 30 October.” Obviously I phoned up straight away and said no, I’m not leaving, please cancel this transfer. They prevaricated a bit, then said, “oh, I can’t cancel this on your say-so, the person ordering has to cancel.” Fine, I say – can I give him a direct number to contact you? “Nope.”

Anyway, I ask the new guy if that’s OK, he says he got fed up of BT trying to transfer the line and went with cable anyway, and he’d been trying to get through to cancel and would try again. A day or so later he tells me he’s cancelled and I don’t think any more about it.

In early November my parents tell me they couldn’t get through. (They didn’t actually tell me they were getting a message saying ‘number not in service.’) A few days into November, suddenly, no connection. A horror begins to dawn on me, and my neighbour – BT have gone ahead and done it anyway.

Now comes the reason for the tale. Most of the time I can’t get through to BT Customer Service, even if I wait for half an hour on the phone. The first time I get off hold, there’s silence for a few seconds and the line goes dead. Next time I’ve apparently called the wrong department and they try to transfer me, but again I end up on hold for half an hour (when I give up). After several goes of this I go for an end-around: I call sales and ask for the line to be transferred back to me.

No problem, says the salesman, but there’s a 12–month minimum contract and you can’t have the line until the 27th of November, unless the current customer cancels before that, oh, and you can’t have your old number back. I did try explaining but it seemed futile. Actually I got pretty angry which I did apologise for.

I thought about it for a couple of days and decided to do what should have been done a very long time ago – get a separate line terminating in this flat. You’d expect that to take a long time, right? Longer than transferring the billing from one customer to another?

Nope. I could have it put in on the 20th. So I paid my £125 (£125!) and got it done.

Unfortunately you can’t order DSL until the phone line is in your name and it appears, under the new number, in BT’s database. So the earliest I could order ADSL was the afternoon of the 20th, then it takes another 14 days or so, taking us to an official activation date of 7 December. I got straight through to Demon sales on practically the first ring. They said I couldn’t have my old hostname/website/email address back initially but once activated I could ask for it back.

The 27th came and went and I suddenly realised I hadn’t cancelled the transfer! Never fear, either the second BT salesman cancelled it for me, or (more likely) they cocked up again. My neighbour still hasn’t managed to cancel and I think he’s refusing to pay.

Finally I got DSL back on the 6th, called up Demon support (got through on first ring again) and got my old hostname back, but the contents of my website were lost. So there are a lot of broken images around here, and if you emailed me in November, I didn’t get it.

I’d always heard that NTL, Telewest, and the unbundled broadband suppliers had worse customer service than BT. If it’s worse than BT they must send round people with pitchforks to stab you with if you even dream of calling them.

CodeProject’s one of my favourite sites – indeed, one of the few real regular sites I visit – but it’s always had a bit of a problem with errors and with the load on the servers. I commented on this before, but sadly the image is gone – lost when my DSL account was closed (BT cock-up, nothing to do with Demon).

Anyway, they recently moved to an all-new ASP.NET implementation of their forums, but unfortunately the errors are still a common occurrence. And the error in the error handler is back:

(Yes, running Vista now.)

One of the things I love most about CP is the more… relaxed attitude to reporting problems.

Friday, 7 December 2007

When I bought my Dell Latitude D820 laptop, I went through and deleted a certain amount of the shovelware that was preinstalled, but some of it, to be honest, I didn’t know what it did. I tried to work out what EMBASSY Trust Suite did, and it seems to be something to do with the Trusted Platform Module in the system. Encrypting files using the TPM key, or something like that.

Charles Petzold wrote an article yesterday commenting on random number generators, and I added a comment earlier on mentioning in passing the reported flaw in CryptGenRandom. I decided to see if this seemed to be the case in XP SP2. My answer was inconclusive, as the assembler was hard to follow. By chance I was debugging Internet Explorer with WinDBG (easiest way to force the RSA Enhanced Cryptographic Provider, rsaenh.dll, to load so I could disassemble it) and noticed an odd number of access violation exceptions occurring when I accidentally did a search in the instance of IE I was debugging. That’s weird, I thought – where am I on the stack?

Looks like we’re trying to get the favourite icon for the address bar. But how is IEFRAME calling into wxvault? Microsoft can’t know that this library exists. Is there something on the stack that somehow isn’t being included (can happen if a function was compiled with the Frame Pointer omitted and no symbols are available to get FPO data [which tells the debugger how to fix up]). Let’s disassemble around PathFileExistsW:

Note how they’ve failed to rebase the DLL, using the default 0x10000000 base address, making it collide with everything ever which also uses that default address.

Needless to say this is going to get uninstalled as soon as I take a full backup of the laptop! In my book, this is a user-mode rootkit. I don’t use the feature, so it’s going.

How should they have implemented this? Well, I’d start by seeing if it’s possible to change the algorithm for the Encrypting File System. It should be, it’s implemented using the Cryptographic API and CSPs (involving callbacks into LSASS in usermode!), so I would have thought that simply providing your own CSP would be sufficient.

If that’s not possible, my next port of call would be a file system filter driver. That would have the downside (like this) that every file system call would go through it, rather than the tiny amount of calls which actually target a file encrypted in this way.

The access violation looks like it might ultimately be caused by a bug in IE – it looks like IE tried to pass the URL to the favicon to GetFileAttributesW, which I would hope would fail (or would it try to invoke WebDAV?)