I am trying to draw an outline on a collection of quads in Direct3D. I have three intersecting quads on the same plane in the following configuration:

(The different colors here are just to visually break out the quads). Using two pixel shaders, one for the longer segments and another for their intersection (the purple quad), I am rendering this as a single color:

I would like to outline this shape, something like the following (please excuse my Paint skills):

My first approach was to write a pixel shader that would calculate the distance along the y-axis (before the transform) from the center of the quad and colorize based on that. However, when I render this, there is a problem where the quads overlap:

What is the best way to achieve my desired result? Is this possible with a texture and a clever use of blending, perhaps? What about somehow stenciling the non-outline areas?

I should mention that my outlines need to be thicker than one pixel, and that I'm working in DirectX 9 with shader model 4 level 9.1, and C++.

The really naive way of doing this would be the same way you draw stroked text - just draw the primitive a few times (x + 1, y + 1, x - 1, y - 1 etc.) You can use a loop from 1 .. thickness if you needed to render out various thickness shapes.
–
Vaughan HiltsFeb 25 '13 at 22:42

1 Answer
1

An ideal way to do this would probably involve changing your geometry such that, instead of three overlapping primitives, you have one continuous primitive -- your two primary line segments, which are quads (of two triangles) and then use "miter" triangles to join them and provide the smooth, thick joints. Using geometry for the edges in this fashion can simplify their rendering when they are thick, which you mention is a requirement, and also allows you to pack in additional data with them which you can use to help determine if the edge should be culled or not.

This technique illustrated in a paper from 2004 on hardware feature edge detection (which covers, in general, the topic you're asking about and is fairly straightforward to implement; it's worth a read even if you cannot alter your geometry).

The fundamental approach taken by the paper is to pack information about edges (not actual vertices) into some vertex data, such that each vertex in the vertex shader now has access to information about the 3D positions of the endpoints, which type of edge this is (a contour edge, a crease edge, or a hand-marked edge), and so on. This information is then used to transform irrelevant edges beyond the clip plane, which will cull them, and allow appropriate edges to be rendered.

It's very possible this technique could be updated to use a geometry shader and thus be simplified, although I haven't tried doing so myself.

Also of note, while I was looking up the URL to that paper I came across this reference to a nVidia extension for NV_path_rendering (with a more detailed guide here), which may be relevant to your interests.