GLSL Shader Live-Reloading

Live-reloading shaders can improve productivity in rendering projects by an order of magnitude or two, so definitely worth your time to set up. In this article, I’ll describe a design for a live-reloading system that is unintrusive to your code, and simplifies a lot of resource binding.

Separation of Concerns

Since the ShaderSet returns a plain GLuint* (which points to to the program), the code making use of the program is separated from the ShaderSet system, since it doesn’t have to know about any ShaderSet specific interface details. This is in contrast to creating a boilerplate “Shader” class and passing that around, which makes your everyday OpenGL code strongly tied to your custom shader system.

Shader Stage Detection

Since OpenGL needs to know the shader type (vertex, fragment, geometry, etc) to create a shader object, I rely on a convention for the shader extension. The convention is as follows:

Vertex shader: .vert

Fragment shader: .frag

Geometry shader: .geom

Tessellation Control shader: .tesc

Tessellation Evaluation shader: .tese

Compute shader: .comp

Update Method

In my current design, I poll timestamps at every frame. This might become a performance problem if you have hundreds or thousands of shaders, but my average research codebases don’t, so polling keeps things simple. In contrast, getting file change callbacks on Windows is very complicated, and hard to make robust. I poll the timestamp using Win32 GetFileTime, and stat can be used for Unixes.

Compile/Link Errors

If a shader fails to compile, I output the error to the console and just keep on trucking. The user can then fix the compile errors through live-reloading.

If the user keeps running code with a shader that failed to compile, it’s likely that they’ll get spammed with errors from every call that uses that bad shader. For this reason, the ShaderSet automatically sets bad shaders to 0 until the compilation is fixed. This makes it possible to detect and handle failed compilation from user code. For example:

if (*program) {
glUseProgram(*program);
render();
}

Resource Binding

In order to link resources like vertex attributes, uniforms, textures, and buffers to the shader, I rely exclusively on in-shader specification of locations using layout directives. This removes the need to query or bind all the attribute/uniform locations every time the programs are re-linked, which removes a lot of tedious binding code. Also, it makes it easier to swap between shaders that use the same inputs.

Synchronizing Bindings

One major drawback of in-shader specification of locations is the risk of C++ and GLSL going out of sync, and the risk of introducing “magic numbers” for locations. In order to counter this, I define a “preamble” file, which contains code that is prepended to all compiled shaders. This file contains all binding locations. A preamble file might be as follows:

If you’re careful, you can even put simple structs inside the preamble, which will also compile as C++ if you’ve defined types like vec4 and mat4 on the C++-side. If you’re even more careful, you can also put simple functions to share code between C++ and GLSL. This works especially well with libraries designed to work like GLSL, like glm.

I’d like to emphasize that the static nature of these bindings is not a weakness. By establishing conventions about the input locations, it’s easy to plug in new shaders by using only the inputs you want, and the C++ code you have to write to bind resources is both simplified (no need to query and re-bind constantly) and more efficient (less overhead of switching bindings.)

5 comments

Also ..what happens if you need to add a secondary uv channel let’s say ? i guess the c++ code needs to be recompiled . Or you define in the preamble all supported bindings ?

It would be interesting to see a higher level integration .. how do “materials” respond to shader changes. what happens if the resources are reloaded as well and now you have more or less vertex attributes and samplers?

All in all hot reloading is the future. i have worked on a small system that would reload everything even the lua scripts used to drive the whole prototype and MAN it was FAST and AWESOME . would uses again 10/10 !

I wanted this to be standalone, so didn’t want to use a third party file watching library. Though might be useful to improve performance if necessary.

Yeah, if you want to add a new attribute, you put all your bindings in the preamble and recompile your cpp code. Ideally, make your cpp code manipulate generic lists of attributes and textures so this isn’t necessary, then the bindings exist only script side. That’s one way to implement materials and differing vertex formats.