I was recently charged with the task of rendering outlines around models in the
game I am working on to show when they are selected. There are a number of
techniques available for accomplishing this, however they typically produce
outlines of inconsistent line weight or fail to accurately represent the
silhouette of the object.

In this short article I'll present my solution to the problem of producing an
accurate, constant weight outline around a polygonal mesh. It recently came
to my attention that a similar method was originally published by Rossignac and
van Emmerik [1] in 1992.

The technique is very simple:

When rendering the object, write a constant value into the stencil buffer. Then to
draw the outline, render the object in wireframe using thick lines wherever the
stencil buffer is unset. This will leave a outline around the border of the mesh
which is one half the wireframe line thickness.

This can be implemented in OpenGL as shown in the following code snippet.

glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT);

// Render the mesh into the stencil buffer.
glEnable(GL_STENCIL_TEST);

One additional detail demonstrated in the sample code is that the stencil buffer
can be reset to zero as the wireframe mesh is being rendered. This is done so that
each pixel in the outline is rendered only once. When drawing the outline
using alpha blending this can be essential to ensure a uniform opacity along
the entire outline.

Additional Notes

If rendering the wireframe version of the mesh is a performance bottleneck,
then a simple geometric silhouette detection method can be used to reduce the
number of lines which need to be drawn. This algorithm works by identifying
edges in the mesh which are bordered by a 'front facing' face and a 'back facing'
face. In addition to including all of the lines which make up the true
silhouette of the mesh, this set of edges may also include lines which will
fall in the interior of the mesh when they are rendered, however these lines
will be discarded by the stencil operation described in the first section.

Direct3D does not support thick line rendering, however the same result can
be achieved by drawing the lines using a camera-facing quad. Pierre Terdiman
presents source code [2] to accomplish this.

Finally, it may be prohibitive for an application to require a stencil buffer
just to achieve the outline effect. In the abscense of a stencil buffer, the
depth buffer can be used in a similar manner as descibed by Rossignac and van
Emmerik [1].

References

[1] J. Rossignac and M. van Emmerik. Hidden contours on a framebuffer. In
Proceedings of the 7th Workshop on Computer Graphics Hardware, Eurographics,
September 1992.