Saturday, January 21, 2012

DX Debugging and Time To Make The Hookers

While earlier than I had hoped, I think it's time I make at least a basic hooking library. After some consideration I decided I want to make a generic texture hacker. Basically, I want to be able to hook the DirectX create object call and inspect the textures and then hook the SetTexture call to either modify or remove which ever textures I mark. I also would like to be able to export the texture and maybe display it on screen or at least create a hash of it. This would be good because it would enable you to specify simple hashes for textures you want to mark as red or green or whatever you want. So that's where I am at and I assume this will take me a few weeks but what I learn won't be specific to texture hacking and I'll be able to reuse a lot of it in other things. (I'm looking at tackling speed hacks next, then probably aimbots).

So all this was brought about by a few things. First, I think I am doing texture hacks incorrectly. I am concerning myself too much with the application code and not the DirectX library itself. I don't know why I didn't think of this to begin with, but I don't need to find *where* in the game that it loads textures, I just need to set breakpoints on the CreateObject and SetTexture methods from dx9s.dll or whichever DirectX version is being used.

When I had some free time the other day I did a quick search for DirectX debugging which lead me to information regarding Microsoft's PIX. For those who aren't aware, PIX is a free tool that comes with the DirectX SDK. It's hooks all of the DirectX calls and lets you inspect each object as it's being created. This is obviously of great interest to me as I can find all the textures being loaded. They call the profiling process 'Experiment' which is pbbfftt, but whatever. So you choose the application and select the 'A replayable Direct3D call stream, saved to file' and start the target application. Here's an 'experiment' of the Meshes DirectX sample application.

PIXRun with the rendered tiger

The captured texture

In the second picture you can actually right click on the texture and save to file. Pretty neat way of extracting images/textures directly from a game. Another great thing is I can right click on the texture address and go to 'Object Creation' It will show me the exact DirectX CreateObject call that initialized that texture. It was also worth just going through an entire frame setup to see how and how many calls are actually made just to render that one single stupid friggen tiger. Quite impressive considering you should be getting about 60 frames per second.

So now I want to find the SetTexture call in windbg. Here's a quick windbg session of me finding the SetTexture call in the Meshes DirectX tutorial application.

SetTextures call from Meshes

I initially set the breakpoint where I knew the SetTextures call was from Meshes and followed it in to d3d9.dll. So the method I want is: d3d9!CD3DHal::SetTexture_FP. I can now just set a breakpoint on this method for the Havok demo application. Here's that debugging session:

Finding and setting a breakpoint on d3d9!CD3DHal::SetTexture_FP

So this is all well and good, but there's a lot of set texture calls and I'd like to not have to inspect each one individually. This is why I want to build a quick hooking example that can export the textures and a hash so I can mark each one that I want set to red. If nothing else, this will allow me to brush up on my C++ which is far dustier than I hoped. So yeah, next few posts will demonstrate me flailing around with C++ and hooking.