Bloom

This tutorial covers how to add support for a bloom effect to a camera. It assumes you're familiar with the material covered in the Rendering series.

This tutorial is made with Unity 2017.3.0p3.

A little bloom makes bright things glow.

Setting the Scene

The amount of light that a display can produce is limited. It can go from black to full brightness, which in shaders correspond to RGB values 0 and 1. This is known as the low dynamic range – LDR – for light. How bright a fully white pixel is varies per display and can be adjusted by the used, but it's never going to be blinding.

Real life isn't limited to LDR light. There isn't a maximum brightness. The more photons arrive at the same time, the brighter something appears, until it becomes painful to look at or even blinding. Directly looking at the sun will damage your eyes.

To represent very bright colors, we can go beyond LDR into the high dynamic range – HDR. This simply means that we don't enforce a maximum of 1. Shaders have no trouble working with HDR colors, as long as the input and output formats can store values greater than 1. However, displays cannot go beyond their maximum brightness, so the final color is still clamped to LDR.

To make HDR colors visible, they have to be mapped to LDR, which is known as tonemapping. This boils down to nonlinearly darkening the image, so it becomes possible to distinguish between originally HDR colors. This is somewhat analogous to how our eyes adapt to deal with bright scenes, although tonemapping is constant. There's also the auto-exposure technique, which adjust the image brightness dynamically. Both can be used together. But our eyes aren't always able to do adapt sufficiently. Some scenes are simply too bright, which makes it harder for us to see. How could we show this effect, while limited to LDR displays?

Bloom is an effect which messes up an image by making a pixels' color bleed into adjacent pixels. It's like blurring an image, but based on brightness. This way, we could communicate overbright colors via blurring. It's somewhat similar to how light can diffuse inside our eyes, which can become noticeable in case of high brightness, but it's mostly a nonrealistic effect.

Many people dislike bloom because it messes up otherwise crisp images and makes things appear to glow unrealistically. This isn't an inherent fault of bloom, it's simply how it happens to be used a lot. If you're aiming for realism, use bloom in moderation, when it makes sense. Bloom can also be used artistically for nonrealistic effects. Examples are dream sequences, to indicate wooziness, or for creative scene transitions.

Bloom Scene

We're going to create our own bloom effect via a camera post-effect component, similar to how we created the deferred fog effect in Rendering 14, Fog. While you can start with a new project or continue from that tutorial, I used the previous advanced rendering tutorial, Surface Displacement, as the basis for this project.

Create a new scene with default lighting. Put a bunch of bright objects inside it, on a dark background. I used a black plane with a bunch of solid white, yellow, green, and red cubes and spheres of varying sizes. Make sure that the camera is HDR enabled. Also set the project to use linear color space, so we can best see the effect.

Bright objects on a black surface.

Normally, you'd apply tonemapping to a scene with linear and HDR rendering. You could do auto-exposure first, then apply bloom, and then perform the final tonemapping. But in this tutorial we'll focus on bloom exclusively and won't apply any other effects. This means that all colors that end up beyond LDR will be clamped in the final image.

Bloom Effect

Create a new BloomEffect component. Just like DeferredFogEffect, have it execute in edit mode and give it an OnRenderImage method. Initially it does nothing extra, so just blit from the source to the destination render texture.

Blurring

The bloom effect is created by taking the original image, blurring it somehow, then combining the result with the original image. So to create bloom, we must first be able to blur an image.

Rendering to Another Texture

Applying an effect is done via rendering from one render texture to another. If we could perform all the work in a single pass, then we could simple blit from the source to the destination, using an appropriate shader. But blurring is a lot of work, so let's introduce an intermediate step. We first blit from the source to a temporary texture, then from that texture to the final destination.

Getting hold of a temporary render texture is best done via invoking RenderTexture.GetTemporary. This method takes care of managing temporary textures for us, creating, caching, and destroying them as Unity sees fit. At minimum, we have to specify the texture's dimensions. We'll start with the same size as the source texture.

Because we're using HDR, we have to use an appropriate texture format. As the camera should have HDR enabled, the source texture's format will be correct, so we can use that. It's most likely ARGBHalf, but maybe another format is used.

Although the result still looks the same, we're now moving it through a temporary texture.

Downsampling

Blurring an image is done by averaging pixels. For each pixel, we have to decide on a bunch of nearby pixels to combine. Which pixels are included defines the filter kernel used for the effect. A little blurring can be done by averaging only a few pixels, which means a small kernel. A lot of blurring would require a large kernel, combining many pixels.

The more pixels there are in the kernel, the more times we have to sample the input texture. As this is per pixel, a large kernel can require a huge amount of sampling work. So let's keep it as simple as possible.

The simplest and quickest way to average pixels is to take advantage of the bilinear filtering built into the GPU. If we halve the resolution of the temporary texture, then we end up with one pixel for each group of four source pixels. The lower-resolution pixel will be sampled exactly in between the original four, so we end up with their average. We don't even have to use a custom shader for that.

Using a half-size intermediate texture means that we're downsampling the source texture to half resolution. After that step, we go from the temporary to the destination texture, thus upsampling again to the original resolution.

Bilinear upsampling, showing interpolation for one pixel.

This is a two-step blurring process where each pixel gets mixed up with the 4×4 pixel block surrounding it, in four possible configurations.

Relaltive weights for indicated pixel, total 64.

The result is an image that's blockier and a little blurrier that the original.

Using a half-size intermediate texture.

We could increase the effect by decreasing the size of the intermediate step further.

Dividing dimensions by 4, 8, 16, and 32.

Progressive Downsampling

Unfortunately, directly downsampling to a low resolution leads to poor result. We mostly end up discarding pixels, keeping only the averages of isolated groups of four pixels.

Direcly going to quarter size eliminates 12 out of 16 pixels.

A better approach is to downsample multiple times, halving the resolution each step until the desired level is reached. That way all pixels end up contributing to the end result.

Downsampling to half resolution twice keeps information of all pixels.

To control how many times we do this, add a public iterations field. Make it a slider with a range of 1–16. That would allow us to downsample a 655362 texture all the way down to a single pixel, which should be enough.

[Range(1, 16)]public int iterations = 1;

Iterations slider.

To make this work, first refactor-rename r to currentDestination. After the first blit, add an explicit currentSource variable and assign currentDestination to it, then use that for the final blit and release it.

Now we can put a loop in between the declaration of the current source and the final blit. As it comes after the first downsample, its iterator should start at 1. Each step, begin by halving the texture size again. Then grab a new temporary texture and blit the current source to it. Then release the current source and make the current destination the new source.

This works unless we end up with too many iterations, reducing the size to zero. To prevent that, break out of the loop before that happens. The height of a typical display is usually smaller than its width, so you can base this on the height only. Because a single-pixel line doesn't really add much, I already abort when the texture height drops below 2.

width /= 2;
height /= 2;
if (height < 2) {break;}

What about mobiles in portrait mode and other exceptions?

It's good that you thought of that! If you want to support all aspect ratios, simply check both the width and the height.

Progressive downsampling 2 to 5 iterations.

Progressive Upsampling

While progressive downsampling is an improvement, the result still gets too blocky too fast. Let's see whether it helps if we progressively upsample as well.

Iterating in both directions means that we end up rendering to every size twice, except for the smallest. Instead of releasing and then claiming the same textures twice per render, let's keep track of them in an array. We can simply use an array field fixed at size 16 for that, which should be more than enough.

Then add a second loop after the initial one. This one starts one step from the lowest level. We can hoist the iterator out of the first loop, subtract 2 from it, and use that as the starting point of the other loop. The second loop goes backwards, decreasing the iterator all the way to 0. This is where we should release the old source texture, instead of in the first loop. Also, let's clean up the array here as well.

Custom Shading

To improve our blurring, we have to switch to a more advanced filter kernel than simple bilinear filtering. This requires us to use a custom shader, so create a new Bloom shader. Just like the DeferredFog shader, begin with a simple shader that has a _MainTex property, has no culling, and doesn't use the depth buffer. Give it a single pass with a vertex and fragment program.

The vertex program is even simpler than the one for the fog effect. It only has to transform the vertex position to clip space and pass through the texture coordinates of the full-screen quad. Because we'll end up with multiple passes, everything except the fragment program can be shared and defined in a CGINCLUDE block.

We'll define the FragmentProgram function in the pass itself. Initially, simply sample the source texture and use that as the result, making it red to verify that we're using our custom shader. Typically HDR colors are stored in half-precision format, so let's be explicit and use half instead of float, even though this makes no difference for non-mobile platforms.

Add a public field to our effect to hold a reference to this shader, and hook it up in the inspector.

public Shader bloomShader;

Bloom effect with shader.

Add a field to hold the material that will use this shader, which doesn't need to be serialized. Before rendering, check whether we have this material and if not create it. We don't need to see it in the hierarchy and neither do we need to save it, so set its hideFlags accordingly.

Box Sampling

We're going to adjust our shader so it uses a different sampling method that bilinear filtering. Because sampling depends on the pixel size, add the magic float4 _MainTex_TexelSize variable to the CGINCLUDE block. Keep in mind that this corresponds to the texel size of the source texture, not the destination.

sampler2D _MainTex;
float4 _MainTex_TexelSize;

As we're always sampling the main texture and only care about the RGB channels, let's create a convenient minimal Sample function.

half3 Sample (float2 uv) {return tex2D(_MainTex, uv).rgb;}

Instead of relying on a bilinear filter only, we'll use a simple box filter kernel instead. It takes four samples instead of one, diagonally positioned so we get the averages of four adjacent 2×2 pixels blocks. Sum these samples and divide by four, so we end up with the average of a 4×4 pixel block, doubling our kernel size.

Different Passes

The result is much smoother and has much higher quality, but it is also much blurrier. This is mostly due to upsampling with the new 4×4 box filter. As we're using the source's texel size to position the sample points, we end up covering a large area, with an unfocused regular weight distribution.

Upsampling with a 4×4 box.

We can tune our box filter by adjust the UV delta that we use to select the sample points. To make this possible, turn the delta into a parameter, instead of always using 1.

Duplicate our shader pass so we end up with two. The first one – pass 0 – will be for downsampling, so it should use the original delta of 1. The second pass will be for upsampling, for which we'll use a delta of 0.5.

With an UV delta of 0.5 we end up covering a 3×3 box with overlapping samples. So some pixels contribute to the result more than once, increasing their weight. The middle pixel is involved in all samples, the diagonal pixels are used only once, while the other pixels occur twice. The result is a more focused upsampling kernel.

Focused upsampling.

Next, we have to indicate which pass should be used when blitting. To make this easy, add constants to BloomEffect so we can use names rather than indices.

At this point we have a fairly simple yet decent blurring process. There are many different kernels that we could use instead of these simple filter kernels, each having its own advantages – like better temporal stability – and costs. However, for this tutorial we'll stick with these.

Creating Bloom

Blurring the original image is the first step of creating a bloom effect. The second step is to combine the blurred image with the original, brightening it. However, we won't just use the final blurred result, as that would produce a rather uniform smudging. Instead, lower amounts of blurring should contribute more to the result that higher amounts of blurring. We can do this by accumulating the intermediate results, adding to the old data as we upsample.

Additive blurring.

Additive Blending

Adding to what we already have at some intermediate level can be done by using additive blending, instead of replacing the texture's contents. All we have to do is set the blend mode of the upsampling pass to One One.

This simple approach works fine for all intermediate passes, but will go wrong when we render to the final destination, because we haven't rendered to it yet. We likely end up accumulating light each frame, blowing out the image, or something else, depending on how the textures are reused by Unity. To solve this we have to create a separate pass for the last upsample, where we combine the original source texture with the last intermediate texture. So we need a shader variable for the source.

sampler2D _MainTex, _SourceTex;

Add a third pass that is a duplicate of the second pass, except that it uses the default blend mode and adds the box sample to a sample of the source texture.

Bloom Threshold

Right now we're still blurring the entire image. It's just most obvious for bright pixels. But one of the uses of bloom is to apply it only to very bright pixels. To make this possible, we have to introduce a brightness threshold. Add a public field for this, with a slider range from 0 to some very bright value, like 10. Let's use a default threshold of 1, excluding LDR pixels.

[Range(0, 10)]public float threshold = 1;

Threshold slider.

The threshold determines which pixels contribute to the bloom effect. If they're not bright enough, they shouldn't be included during the downsampling and upsampling process. Simply converting them to black will do this, which has to be done by the shader. So set the material's _Threshold variable before we blit.

if (bloom == null) {
…
}
bloom.SetFloat("_Threshold", threshold);

Add this variable to the CGINCLUDE block of the shader as well, again using the half type.

half _Threshold;

We'll use the threshold to filter out pixels that we do not wish to include. As we do this at the start of the blurring process, it's known as a prefilter step. Create a function for this, which takes a color and outputs the filtered one.

half3 Prefilter (half3 c) {return c;}

We'll use the color's maximum component to determine its brightness, so `b = C_r vv C_g vv C_b`, where the `vv` symbol is an operator that I use to denote the max function.

We can determine the contribution factor of the color by subtracting the threshold from its brightness, then dividing that by its brightness, `c = (b - t) / b`, where `t` is the threshold. The result is always 1 when `t = 0`, which leaves the color unchanged. As `t` increases, the brightness curve will bend downward so it drops to zero where `b = t`. Because of the curve's shape, it's known as a knee. As we don't want negative factors, we have to make sure that `b - t` doesn't drop below zero, leading to `c = (0 vv (b - t)) / b`.

Knee curves with thresholds 0.25, 0.5, 0.75, and 1.

To avoid divisions by zero in the shader, make sure that the divisor has a small value at minimum, like 0.00001. Then use the result to modulate the color.

At this point, with the threshold set to 1, you'll likely see no or almost no bloom, assuming the light and materials used have no HDR values. To make bloom appear, you could increase the light contribution of some of the materials. For example, I made the yellow material emissive, which together with the reflected light pushes the yellow pixels into HDR.

Emissive yellow.

Isolating Bloom

To better see what parts of the image contribute to bloom, it would be handy if we could see the blur effect in isolation. So let's add a debug option to our effect, controlled via a public boolean field.

public bool debug;

Debug toggle.

We'll create a separate pass for debug purposes, so add a constant for it at the bottom.

While in debug mode, we can clearly see that the yellow pixels end up generating bloom. Besides that, also some of the white pixels are included, but only when they end up reflecting a significant amount of the directional light.

Why do some white surfaces end up in HDR?

The default Unity scene is very bright. Besides the directional light, there's also environmental lighting and reflections which contribute to the final pixel color. All these together can result in brightness values greater than 1.

Soft Threshold

The knee curve that we use to modulate colors cuts through zero at an angle, leading to an abrupt cutoff point. This is why it's also known as a hard knee. This means that we can end up with sharp transitions between areas that generate bloom and those that do not. This can be seen in the large white sphere in the screenshot above. There's a clearly-defined portion of that sphere which gets included. This is somewhat obfuscated by the blurring, but it's still a harsh transition.

It's possible to make this transition smoother, blending from zero to full contribution. We'll control this with a slider. At 0, we get the harsh transition that we currently have. At 1, we get a soft threshold that smoothly fades the bloom in all the way from brightness 0 until it matches the hard knee. We'll use 0.5 as the default.

[Range(0, 1)]public float softThreshold = 0.5f;

Soft threshold slider.

This fading is also done by the shader, so pass the soft-threshold factor on to the material.

We have to cut off this curve where it touches 0 and where it meets the hard knee, which is done by using `s = (0 vv (b - t + k) ^^ 2k)^2/(4k)`, where `^^` represents the minimum function. Adjust the prefilter function to perform this calculation, again preventing divisions by zero.

Note that some parts of the soft knee function can be isolated so they only depend on the configuration values, which are constant per pass. We can precompute these parts and pass them to the shader in a vector, reducing the amount of work it has to do. We can combine these with the threshold in a single filter vector.

Bloom Intensity

To wrap up, let's make it possible to modulate the intensity of the bloom effect. This makes it possible to fade it in, and also to create ridiculously-strong effects. Add a slider for this, with a range like 0–10. The default should be 1.

[Range(0, 10)]public float intensity = 1;

Intensity slider.

Pass this intensity value on to the shader, as a material property. As it is common to set a bloom's intensity with a factor in gamma space, convert it from gamma to linear space.

You now have a basic bloom effect. It is quite similar to the bloom effect of Unity's post processing stack, version 2. It's possible to extend it further, by adding a tint, making the sampling delta configurable, using different filters, and so on.