Wednesday, 5 June 2013

The surge in interest in Physically-Based Rendering has led to a surge in interest in area light sources. There is, however, something of a dearth of information available for the aspiring implementer.

Introduction

The objective of polygonal area light sources is to define an arbitrary polygon in space that may emit light. I'm going to make two simplifications here: firstly, that the polygon emits a constant colour across its surface and secondly, that we are not considering shadowing.

The earliest source I was able to find on this subject is James Arvo's 1994 paper on polyhedral light sources. John Snyder also presents another excellent reference. The approach taken here is to compute a vector field such that at any point in space r, we may obtain the direction of the light. This vector field is a function of the position r, but also the polygon's vertices. This direction can then be substituted into your brdf equation for a particular surface.

Arvo's algorithm is simple: for each edge of the polygon he computes a normal named gamma, and a weighting value named theta. The normal is of the triangle formed by the polygon edge and the point r. The weight theta is the angle formed by the pair of vectors from the point r to each edge vertex. He forms a weighted sum of these normals, divides it by 2*PI (to normalise the weights returned by acos) and scales it by the light's intensity. This is all you need to perform basic area lighting.

There are a couple of twists. Firstly, you typically need to negate the computed vector to get it into a form ready to dot against your normal. Secondly, you must clip the polygon against the plane of the point being shaded. It is possible to have a valid light source that straddles the plane of the point being shaded.

Basic HLSL code

Here is the HLSL code to perform the basic calculation to obtain the light vector for a polygon at a given point. Here, I process quads. This is no algorithm restriction; this is purely to minimise buffer traffic.

// Derive quad vertices in world-space

float3 quadCentre = g_quadLights[lightIndex].pos;

float3 quadAxisX = g_quadLights[lightIndex].axisX * 0.5f;

float3 quadAxisY = g_quadLights[lightIndex].axisY * 0.5f;

float4 quadVertices[4] =

{

float4(quadCentre - quadAxisX - quadAxisY, 1.0f),

float4(quadCentre + quadAxisX - quadAxisY, 1.0f),

float4(quadCentre + quadAxisX + quadAxisY, 1.0f),

float4(quadCentre - quadAxisX + quadAxisY, 1.0f),

};

float3 lightField = 0.0f;

uint polyVertex1 = 3;

for(uint polyVertex0 = 0; polyVertex0 < 4; ++polyVertex0)

{

float3 delta0 = quadVertices[polyVertex0].xyz - worldSpacePos;

float3 delta1 = clippedPolygon[polyVertex1].xyz - worldSpacePos;

float3 edge0 = normalize(delta0);

float3 edge1 = normalize(delta1);

float theta = acos(dot(edge0, edge1));

float3 gamma = normalize(cross(delta1, delta0)); // Note you can't just cross edge0 and edge1 to save the normalize - that gets you the wrong result!

lightField += theta * gamma;

polyVertex1 = polyVertex0;

}

// Note that we don't divide by 2*PI as in the paper. That is just there to account for the sum of the theta values

// We don't need it since we're doing a normalise - it's just a uniform scale

Clipping

We must now deal with the problem of clipping. This is a particularly awkward operation, especially when it may need to be done per-pixel. Firstly, we must establish whether we actually need to clip. Since the quads are formed using a quad centre and two axes, this may be handled using something very similar to an OBB vs plane test:

Friday, 17 May 2013

Preamble

Recently, I have been researching some next-gen rendering techniques. There's some great presentations out there by the usual suspects - Epic, Crytek, DICE, Guerilla and so on. Many of the vendors also have good sample code to explore. There's a lot of buzz about certain new techniques... but there's sometimes a little bit of a gap between the academic literature, the presentations and sample code. Sometimes its a little hard for the time-pressed practioner to google an example and jump straight in.

So, I thought I'd show a little example code to help people get started.

Introduction

Tone mapping using a histogram is becoming a trend in next-generation graphics. People are starting to experiment with moving away from the DX9/360/PS3 style "compute average luminance, apply exposure, apply film response curve" algorithms towards trying something different.

Histogram based methods aren't new. An early, commonly cited example is Ward et al '97. More recent examples are things like Duan et al 2010. The basic idea is that you take an input high dynamic range image, compute a histogram measuring the frequency of different luminances in that scene, and use that to build a response curve. The response curve maps from the scene's luminance range to your output display lumiance range.

So, in one fell swoop, you can take your HDR image and convert it to an LDR image. No repeated downsampling and cache thrashing, no incorrect half-pixel or half-texel offsets, no frame-by-frame adaptation. A simple algorithm that "just works".

This algorithm fits very naturally with compute shaders in DX11. However, not everyone out there is a DX11 expert; there are many practictioners busy shipping titles on DX9 class hardware who have not had the opportunity to experiment with DX11 yet. I shall detail the implementation of a very basic histogram based tone mapping operator that may serve as the basis for future expansion.

Running a Compute Shader Over an Image

Compute Shader Introduction

Compute shaders off a great deal of general, flexible power on modern DX11 GPUs. They are very flexible and offer immense general computation power. They are not, however, a direct replacement for C++ code; they have a specific programming model that you have to follow. Whilst you can ignore some details of this model, you will not achieve maximum performance. This programming model therefore largely dictates how you must structure your algorithm.

An individual compute shader contains the inner most operation of some loop. However, the compute shader is not executed as some kind of iterative loop. Instead, many separate invocations of the compute shader code are scheduled, dispatched and retired. Each invocation of the compute shader is known as a thread, and each thread receives unique addressing information. And these threads are executed in parallel. Threads are grouped into thread groups. Each thread is therefore separately executing some part of the desired operation. It is very akin to a parallel map operation in a functional language.

A compute shader is therefore unlike a general CPU process or job. It is unlike the jobs or task programming model of the Cell processor. It is very similar to shading a pixel. In contrast to pixel shading, compute shader threads can communicate with other concurrent threads. A thread may communicate with other threads in the same group using some on-chip fast memory known as group shared memory.

The conclusion of this is that when processing an image with a compute shader, you do not sit in a big x * y loop, fetching pixels and processing them. You do not DMA in parts of an image, process them, and DMA them out again. You write code that processes each pixel independently and dispatch that work to be processed as a whole in parallel. This requires a certain structure to your code, especially if you want to obtain maximum parallelism.

Tiling

A common paradigm when executing a compute shader over an image is to break the image up into tiles. Each thread typically corresponds to an input pixel.A thread group maps to a tile. Each thread is executed and aggregates data to the local, on-chip memory known as group shared memory. When all the threads in a group have completed, the results are exported from the transient group shared memory to some persistent off-chip memory.

But what is the reasoning behind this model?

If we liked, we could dispatch a compute shader with a thread group size of 1 x 1 x 1 thread. Inside this one thread of one shader, we would iterate through every single pixel in an image and perform our computation. But, this model offers very little parallelism. We would only enjoy the SIMD instruction level parallelism, and would have no thread-level parallelism. GPUs are built to be very, very wide. We would only be occupying a fraction of the GPU.

Ideally. we want to launch as many threads as possible. This undermines any kind of approach where one thread iterates across many pixels. We want a thread per pixel.

And this is what gives rise to the image tiling model. Here, we break the image into tiles of 16x16, 32x32 or whatever size yields the maximum threads and therefore parallelism possible. We exploit group shared memory rather than work directly in off-chip memory for performance reasons. A modern GPU will not simply sit and churn through every every thread of your compute shader until completion. It will execute wavefronts (groups of threads) from multiple different shaders simultaneously across the GPU. If we can read from off-chip memory but write to on-chip memory, we minimise our bandwidth consumption and play better with other shaders. Ultimately we will need to export the results, but we can do that once in a burst rather than constantly causing memory hazards.

We may need to execute subsequent passes that combine together this intermediate metadata to yield a final result. This arrangement is akin to the common map-reduce parallel programming model.

I will now dive straight into the HLSL compute shader code. For clarity and brevity, I won't present the detailed DX11 CPU-side code; I shall just detail what it needs to do.

Computing the Per-Tile Histogram.

Our first task is to compute a histogram for each tile of the image. These will later be combined to form a histogram for the whole image. Each histogram will map the scene's luminance range across N bins. Each bin holds a uint which represents the number of pixels in that luminance band.

To perform this, we need a few things on the DX11 side:

A shader resource view that allows us to read the source texture.

A shader resource view for our constant buffer.

A large buffer to store all of the per-tile histograms. Its size will be numTiles * numHistogramBins.

An unordered access view for the per-tile histogram buffer.

We can then dispatch our compute shader. Note, that we have to round up the image size to be a multiple of our compute size; we must then be careful to only sample the real pixels. Here's the code:

// It is possible that we emit more threads than we have pixels. This is caused due to rounding up an image to a multiple of the tile size if(globalIdx.x < outputWidth && globalIdx.y < outputHeight) { // We just use sample 0 for efficiency reasons. It is unlikely that omitting the extra samples will make a significant difference to the histogram float3 pixelValue = sourceTextureSRV.Load(int2(globalIdx.xy), 0).rgb; uint bin = rgbToHistogramBin(pixelValue); InterlockedAdd(histogram[bin], 1u); }

We now need to take this buffer and combine it to form a single histogram for the whole image.

Merging the Per-Tile Histograms

This is a relatively simple compute shader. In this case, each thread maps to one of the tiles just computed, and each threadgroup therefore represents a group of tiles. This shader atomically adds each tile's histogram to the global histogram. In addition to the previous resources, we also now need:

Note that here we perform atomic operations on the merged histogram. You may do this on both group shared memory and unordered access views; clearly there is extra overhead in the latter. We now have a histogram for the whole image. Our job is now to convert that to a response curve. This is a function that maps from the input HDR image to a new LDR image. Wikipedia offers an easy explanation.

Computing the Response Curve

Again, more resources are needed:

A buffer holding a float for each histogram bin.

An unordered access view to write to it with.

A shader resource view for the whole-image histogram.

There is a small wrinkle in this code. We also include an extra step - the adjustment phase discussed in Page 14 of the Ward '97 paper. This is included only as an example to get you started. I use a 1x1x1 compute shader here as this is such a minimal operation that parallelism would gain little.

Applying the Response Curve

Again, we need one more new resource: A shader resource view to access the response curve buffer. The code is again quite simple. Here, we're running a compute shader over tiles of the image. I've also included an MSAA resolve and gamma correction step, as these are features you will probably need to implement too in a next-generation linearly correct renderer. Note, that you ideally tone-map before the MSAA resolve. I felt it was simpler to present the code this way around. Secondly, ideally your frame buffer would have separated chrominance from luminance and you would just scale the luminance. Again, I've presented it this way for a simple example.

Thursday, 25 April 2013

Recently, I've been having a little bit of a think about how to handle realtime distant shadowing in games, in order to avoid shadow baking.

Clearly, it's entirely possible to use some kind of cascaded shadow map extending all the way out. But, this soon ends up getting expensive in terms of textures and rendering. Alternatively, you might consider using something like stencil shadow volumes, and using the extended compute capabilities of modern GPUs to clip down the shadow volume so it doesn't consume enormous amounts of rasterisation or clipping time.

Alternatively, I've been thinking a lot about reprojection. Let's say you interrogate your cascaded shadow map and use that to build a screen-space mask of the shadow response. You want soft(ish) shadows, so you commit a lot of PCF samples.

How about reprojecting last frame's shadow mask into this frame's shadow mask, then filling in the holes with some simple filtering operations? Distant shadows are typically pretty low-frequency things, so you could well get away with some blurring and imperfection in the shadow mask. Then, you need only render the geometry that became newly visible this frame into the shadow map. You could just subtract one frame's light frustum from the other, yielding a convex polytope, and only render the intersecting objects.

Thursday, 28 March 2013

Recently, I've been doing a lot of research and thinking about realtime GI and its related ideas. A common theme in realtime GI is the use of VPLs, a way to approximate the radiance bouncing off a surface after interacting with its material. A VPL is only formed where there is a definite light-to-surface interaction. The VPL can therefore be used to simulate light bouncing off geometry.

One technique in use today for generating VPLs is the Reflective Shadow Map algorithm. Here, we render the scene from the light's point of view, and every pixel in the output image represents a surface receiving primary illumination from the light source. This is obviously a fairly efficient way of finding all the surfaces that receive radiance from a given light source without too much wastage. You can then construct VPLs using that image, reduce the working set and inject them into some lighting algorithm to bounce light around your scene.

The difficulty is, this may not scale well as each new light source would require a new RSM render.

One alternative I considered was the possibility of using a cube map to sample the primary illumination from multiple lights simultaneously. You would put your cube map at your camera position, or some other meaningful point-of-interest. You then render the scene into each cube map face. As you render it, you light it from all of your light sources simultaneously, with shadow maps too if desired. Obviously this would be an expensive render.. but it would only be done once, you can vary the resolution of it and you can always use techniques like tiled lighting to accelerate it if you have many lights.

When complete, this cube map holds the radiance at the first light bounce for many surfaces in the scene near to the point of interest - ie the camera. It contains samples for surfaces all around - both in front of and behind the camera, which is important for GI. You may then use this cube map to generate VPLs for further bounce lighting.

Now, this is as-yet a completely untested idea. I will get round to it, though. I thought I'd throw it out there as an idea for using a commonly-accelerated rendering structure to try to help the scalability of realtime GI solutions.

Tuesday, 19 February 2013

I've recently been developing a lot of code in C++ 11. I find it a huge and useful improvement over previous version of C++; here's some observations on some of the features in it that I've found rather valuable.

emplace

This simple, easily-overlooked addition to the STL has to be one of the most practically useful features of C++ 11. In itself, it does very little: it constructs a new object in-place inside a container. This in itself is very useful for optimisation purposes. However, the real advantage lies in semantics, rather than optimisation.

emplace() means that you are no longer forced to include copy semantics into classes that really, really do not want them in order to place (copy) them into a container type. If the container implements some emplace() functionality, you are free to then delete all copy assignment and copy construction operations, only providing move assignment and construction instead.

This is a huge boon for classes such as resource classes, where assignment or copy can sometimes be a very unnatural and dangerous operation. With emplace() and move semantics, you are then free to create resource classes that make extensive use of const and RAII, ensuring that only one class ever binds to a particular resource.

Avoid Ternary Expressions Using Lambdas

A simple one. Ternary expressions can be a little difficult to debug, especially when they are nested:

const int a = d < e ? f : g < h ?: i : j < k ? m : n;

And so on. Whilst I don't advocate it, it does occur. Along the lines of the above point, using a lambda creates code you can place breakpoints on and debug easily, at no performance loss.

Reduction of Code Duplication Using Lambdas

Lambdas are very useful to reduce internal code duplication within a function. This perhaps seems a little excessive... but it can help because it factors out the details of a particular operation into one place, leaving the rest of the code simpler. Again, this avoids the problem of artifical external functions that reduce encapsulation... though I do not propose that everything be written as giant functions containing lambdas!

What's more, I've found that the compiler is generally smart enough to inline the expansion of these lambas as appropriate. So there is no performance penalty!

Move Semantics For Resource Assignment

A common issue I face with code is in creation code, where one function creates a resource, passes it to another object which is assumed to take responsibility for its lifetime management and destruction. However, often the constructor just takes a pointer to an object - this declaration is less than clear on what it will do with that resource. This doesn't tell you whether it'll just hold that reference, or manage it.

I find move semantics can help here, a little. A constructor which has an rvalue reference to some resource declares that it will acquire the resource, hold a reference to it, but also divest the calling code of that object. This perhaps makes it a little clearer that ownership of the resource is being transferred, not simply passed.

Adding the secret sauce of std::unique_ptr<> makes lifetime management even more painless.

Auto

Auto seems one of the potentially more divisive additions to C++ 11. At first, I was unsure. I can see how it can avoid template craziness... but would the lack of explicit type names make things harder to debug?

So far, I generally use auto by default as much as I possibly can. I find that if your code is making good use of functions and constructors, they tend to make it fairly clear what types are being constructed. Furthermore, it makes the code much more amenable to change and refactoring, as it reduces the number of times you have to name a particular type. This seems to be a good thing.

The main downsides of it are that you have to be extremely careful to remember to include & to make some types a reference. If you do not, then you end up implicitly copying objects around, which can lead to performance, correctness or compilation issues.

Range-based for

This one's not such a big deal. It can be useful, as it can save on code verbosity... but so often, you need to iterate over multiple containers simultaneously, and that's where range-based for ceases to be useful.

constexpr

This is a nice simple win. This at least means you don't have to start making trivial functions into templates to get some compile-time optimisations. It's unfortunate that it insists on the single-line return statements, as this can mean things such as data tables spill out of the scope of the function.

Class Member Initialisation

I find that being able to assign values to class instance members inside the class declaration to be enormously useful. This allows you to vastly simplify your constructors, especially when you have multiple constructors. This removes a lot of duplication, and allows you to instead implement only the specialisations. This seems to be the right way round to me.

Promises, and Futures

By and large, these are quite nice facilities. The big thing they lack though is the ability to poll a future. This seems like quite a major omission, and can make them difficult to use for some algorithms. As such, I tend to find myself rolling my own cut-down variants for many purposes.

In summary, it's very agreeable. No real change is mandated, so you simply opt in or opt out of as many or as few new features you'd like to employ. I'm a convert.