I need to amend the ticket a little. The system used to abort the guest, but now it hangs with 100% - 98% of CPU if I assign 2 processors and try to use 3D (aero). Sending the shutdown signal does work but reports a BSOD on error reporting when resumed.

I gave it a try and it does close to the same thing. Instead of aborting now it freezes. Selecting pause and resume brings up a black screen in 640x480. First time I saw a BSOD when I paused and resumed so I entered the

VBoxManage setextradata "VM name" "VBoxInternal/PDM/HaltOnReset" 1

to see if I could capture it but that does not bring up the BSOD when going paused resume with that since I assume the freeze is stopping it. I can send another log file is you want. It sure looks like driver issues to me.

Replying to Perryg: Thanks a lot. Unfortunately it's not yet clear what have caused this misbehave from the latest minidump.
Could you please upload a full memory dump to the anonymous FTP server ftp://ftp.oracle.com:/appsdev/incoming ?
To enable full dump generation, go to My Computer -> Properties -> Advanced System Settings -> "Settings" button under "Startup and Recovery" and select "Full memory dump" if available or "Kernel Memory dump" otherwise.
The "Dump file" edit box specifies the dump file location, which is %SystemRoot%\MEMORY.DMP by default.

Once you uploaded the dump, please post me it's name either here or at mikhail dot sennikovsky at oracle dot com so that I can pick it up.

Besides that, could you specify what steps do you do to make the system crash? Are you starting/running some apps, or just logging in, etc.?

HaHa well it will be later today. Windows finally got tired of it and no way will it boot now. It now bsod's on ntfs. I need to restore from backup and then make sure all updates and the like are installed.

I used my restore backup which is the default install with nothing installed, not even the GAs.
Installed the test GAs you had me use and did the system updates.

As for when it freezes, or reboots. It does both but all I need to do is log in and wait for Windows to settle down. First indication is a freeze. If I wait it will reboot itself. Then I have to set the processor count to 1 or disable the 3D accel. to be able to boot back in.

In two words, when handling a VBOXVDMACMD_TYPE_DMA_PRESENT_FLIP command in DxgkDdiSubmitCommand the driver scheduled a DPC to notify the dxgkrnl runtime about the dma command completion and do some command post-processing (see vboxVdmaGgCmdDmaNotifyCompleted), the DPC then referenced the memory buffer representing that command and did some op on it (see vboxVdmaDdiCmdHandleCompletedList).
Besides that the driver also posted additional work item to a worker thread, which did some op on the same memory buffer and then destroyed it (see vboxVdmaPipeCltCmdPut in vboxVdmaGgCmdSubmit).
Thus on SMP systems it could happen that the worker thread could process & destroy the buffer before the DPC.
Pretty silly bug, and I'm recalling now that I added that code as a temporary measure to quickly test & solve some other bigger problem we had here, but given there is no more permanent than a temporary this bug stayed for so long.. sorry about that and really glad it should be fixed now!
Once again thanks a lot for helping with hunting this bug.