Shader Debugging for BGFX Rendering Engine

This is a computer translation of the original content. It is provided for general information only and should not be relied upon as complete or accurate.

Sorry, we can't translate this content right now, please try again later.

In the development of real-time rendering graphics, shader code becomes increasingly complex, so simple development and debugging methods based on experience and trial-error can no longer meet actual needs. The debugging tool plays a critical role in shader debugging and is important in graphics development.

Bgfx is an excellent cross-platform and abstract rendering engine that encapsulates most mainstream graphics APIs. As an example, this article demonstrates the use of Microsoft Visual Studio* and RenderDoc to debug DX11 shader code in bgfx on the Windows* platform.

We can get more information about project generation through the --help command line parameter.

Open bgfx.sln with Visual Studio 2017 in the generated bgfx/.build/projects/vs2017 directory. We see in Figure 1 that the sample and tool projects are already in our Visual Studio solution.

Figure 1. Some of the many examples and tools included in the Visual Studio project file

The example-37-gpudrivenrendering project, the basis of this article, uses compute shader. Our debugging will focus on it. Another project, the folder labelled shaderc, features the bgfx shader compiler responsible for compiling the bgfx shader source into shader binary for a target API runtime.

Next, build the project in Visual Studio. Then, find all the compiled sample programs and shaderc.exe in the bgfx/.build/win64_vs2017/bin directory. Select debug mode in Visual Studio to see Debug appended to the name of each generated application (Figure 2).

Figure 2. The two applications (highlighted in blue) we’ll be working with in this article as they appear with debug mode selected.

Test the “example-37-gpudrivenrendering” Sample Program

We can’t directly double-click to start the target sample program, because we need to specify the program startup directory to the bgfx/examples/runtime directory. That directory contains resource files the application needs to load. It’s much easier to debug in Visual Studio, because the working directory for debugging has already been set in the project settings (Figure 3).

Figure 3. Properties>Debugging, \examples\runtime is already set up in the Working Directory.

If all goes well, we will see the screen in Figure 4 when we run the example.

Figure 4. Example-37-gpudrivenrenderingDebug running

Recompile DX11 Shader

By default, the bxfx DX11 shader file is under bgfx/examples/runtime/shaders/dx11. That said, these shader files with the suffix .bin do not contain debugging information and have already been optimized. There’s no need to debug them, so we need to recompile these DX11 shaders, give shader debugging information and disable optimizations so that we can see more useful information in the debugging tool. We’ll use the cs_gdr_downscale_hi_z shader as an example for debugging.

Set the current directory to bgfx/src

The reason the current working directory is set to the src directory is because the common shader source files included by the shader compilation are also in this directory. Without this step, the shaderc will not find these public shader files.

cd bgfx/src

Use the following command to compile the shader that needs to be debugged.

We can see different panels, and the most important ones include Event List, Call Stack, Pipeline Stages, Object Table, and so on. Select a compute shader related Dispatch interface event in the Event List, and select CS Shader in the expanded sublist. Since we compile in debug mode, the source code of the target shader will be displayed in the opened source panel.

Next, we can start debugging the shader by clicking the green debug arrow (above, circled in red) under the target shader. We can debug the shader as we normally do in the Visual Studio. We can Step Into (F11), Step Over (F10), Step Out (Shift + F11), and watch the values of the variables.

Use RenderDoc for Shader Debugging

RenderDoc is a powerful open source graphics development and debugging tool. Download the latest version of RenderDoc from https://renderdoc.org/, and install it for later use.

Launch gpudrivenrenderingDebug.exe in RenderDoc as shown in the Figure 8. Be sure to set Working Directory to the examples/runtime directory. Click Launch to run the gpudrivenrendering program.

Figure 8. Use RenderDoc to launch example-37-gpudrivenrenderingDebug.exe by entering its path and working directory in their respective fields.

With your cursor positioned in the running gpudrivenrendering window, press F11, and grab a graphics frame. Double-click the newly generated preview to open the frame (Figure 9).

In the opened frame, click steps 1, 2, 3 as shown in the Figure 10 to enter the compute shader debugging mode. We can check disassembly (Figure 10B) or HLSL (Figure 10c), as well as variables, registers, constants, and resources. At this point, we can move on with the debugging.

Please note, RenderDoc’s documentation states its debugging support for Compute Shader is experimental:

“This feature is highly experimental and is provided with no guarantees yet! It may work on simple shaders which is why it is available.”

Conclusion

Due to the diversity of hardware platforms and graphics APIs, we need to use different tools, and adopt different development and debugging methods for specific cases. Starting with an actual rendering engine, this article introduces in detail the use of Visual Studio and RenderDoc to debug shader on Windows platform. It has certain reference value for other platforms and graphics APIs.