information when you need it

Category Archives: Programming

This patch allows you to use more than 3/4GB of RAM on an x86 Windows system. Works on Windows Vista SP2, Windows 7 SP0, Windows 7 SP1, Windows 8 and Windows 8.1. Instructions and source code included.

This patch allows you to use more than 3/4GB of RAM on an x86 Windows system. Works on Vista, 7, 8, has been tested on Windows Vista SP2, Windows 7 SP0, Windows 7 SP1 and Windows 8 SP0. Instructions and source code included.

If you’re writing a backup program, you need to be able to read and access files that are in use by other programs. The proper way to do this is to use the Volume Shadow Copy Service. The MSDN pages cover a lot of material that is unnecessary for a simple backup program, so here are some simple steps to get you started.

Before you start: important notes

This article assumes that you are using Windows Vista or later. For Windows XP, you may need to make some modifications to this code.

You cannot use VSS from a 32-bit program running under a 64-bit version of Windows. On 64-bit Windows, you need to compile a 64-bit version of your program.

Usually, your program needs to run under an administrator account. If UAC is enabled, your program needs to be elevated.

Step 2. Connect to VSS

VSS_ID snapshotSetId;
result = backupComponents->InitializeForBackup();
if (!SUCCEEDED(result))
abort(); // If you don't have admin privileges or your program is running under WOW64, it will fail here
result = backupComponents->SetBackupState(FALSE, FALSE, VSS_BT_INCREMENTAL);
if (!SUCCEEDED(result))
abort(); // Handle error
result = backupComponents->SetContext(VSS_CTX_FILE_SHARE_BACKUP);
if (!SUCCEEDED(result))
abort(); // Handle error
return backupComponents->StartSnapshotSet(&snapshotSetId);

Step 3. Add the volumes

Now you need to add the volumes that you’re interested in by calling AddToSnapshotSet. This will give you a snapshot ID that you need to save. Make sure your volume name has a trailing backslash. In this article we’ll assume that you only have one volume that you’re interested in, but you can add as many as you want.

Step 5. Use the snapshot(s)

Your snapshot(s) are now ready. For each of the volumes, call GetSnapshotProperties to get a VSS_SNAPSHOT_PROP structure. Use the m_pwszSnapshotDeviceObject field to get the device name (e.g. “\Device\HarddiskVolumeShadowCopy1″) for your snapshot. If you created a snapshot for D:\ and you want to access D:\somefile.txt, open \Device\HarddiskVolumeShadowCopy1\somefile.txt.

OK, Visual Studio 2012’s new user interface isn’t that bad. But you need to get some serious work done, and you don’t want to be forced to hover your cursor over each monochrome icon to get a tooltip explaining what it’s for. NOR DO YOU WANT THE MAIN MENUS SHOUTING AT YOU ALL THE TIME. That’s why you need to get NiceVS, which restores the old colorful icons and sensible menu labels.

Choose TOOLS > Extensions and Updates…, click Online, and search for NiceVS. Once installed, you will see the following (hilarious) screen:

The reason this works is that FindWindowEx, unlike EnumWindows, does not ignore Metro windows. We just call FindWindowEx in a loop to keep retrieving the next window in the list until there are no more. Note the i < 1000 condition, which is there to prevent infinite loops (highly unlikely but possible, to my knowledge).

Here’s some output from WinDbg on Windows 8 while I was debugging a driver:

These two UNICODE_STRINGs are from the OBJECT_TYPE structures of the Section and TmTx (transaction) object types. Can you spot the difference between these two strings? The Section string’s MaximumLength includes two extra bytes for a null terminator, while the TmTx string’s MaximumLength doesn’t.

Now take a look at the code for ObQueryTypeInfo, which is called from NtQueryObject when you pass in the ObjectTypeInformation information class. Look out for the buffer overrun at the end. Also notice that the string “TmTx” happens to be 8 bytes long (when using WCHARs).

The solution is pretty simple. Until Microsoft fixes this bug, if you’re calling NtQueryObject with some ObjectInformationLength, make sure you have ObjectInformationLength + 2 bytes allocated in the buffer that you are passing in.

Recently I became frustrated with Cobian Backup. It was the only free software I could find that:

supported incremental backups,

supported Volume Shadow Copy,

and didn’t install a bunch of extra, useless startup entries and services.

However, two things sucked:

Incremental backups only seemed to work properly when it used the “archive” attribute. This meant that it couldn’t recover from interrupted backups properly.

It didn’t provide any way of restoring files, so you had to go through the different .7z files and find the file you want to restore.

So, I decided to create my own backup program: “WJ’s Backup”. The possessive in the title is not to indicate some obsessive sense of ownership, but to emphasize the fact that the program was not developed with any other user’s perspective in mind.

Unlike most other backup programs I could find, WJ’s Backup maintains a database with file/directory metadata for each revision. It keeps file versions in a SVN-like way, but without diffs for file content (each new version is stored in full). 7-Zip compression is mandatory.

This patch allows you to use more than 3/4GB of RAM on an x86 Windows system. Works on Vista and 7, has been tested on Windows Vista SP2, Windows 7 SP0 and Windows 7 SP1. Instructions and source code included.

Windows 7 introduced two new object types: UserApcReserve and IoCompletionReserve. What do these object types have in common? They’re both created using NtAllocateReserveObject. If we look inside this system call we can see that the third argument is an index into two arrays, PspMemoryReserveObjectSizes and PspMemoryReserveObjectTypes. Notice that PspInitPhase0 creates a set number (currently two) of object types, drawing the names from another array: PspMemoryReserveObjectNames. Here’s my reversed definition of NtAllocateReserveObject:

Two new system calls take advantage of the UserApcReserve object and IoCompletionReserve object: NtQueueApcThreadEx and NtSetIoCompletionEx, respectively. In NtQueueApcThreadEx we can see that if a user APC reserve handle/object is supplied, the function uses the space allocated for that object to store the APC, instead of allocating from the pool. Similarly NtSetIoCompletionEx uses the I/O completion reserve object’s already allocated space to store the I/O completion mini-packet, instead of allocating from the pool. It is now clear what the purpose of these reserve objects are: to allow processes to reserve memory before performing certain system calls in order to avoid out-of-memory problems occurring at bad spots (or critical code). Here’s my reversed definitions for the two system calls: