One use for a Geometry Shader is, to treat an input-triangle as if to consist of multiple smaller triangles, each of which would then be called ‘a micropolygon’, so that the points between these micropolygons can be displaced along the normal-vector of the base-geometry, from which the original triangle came. One reason for which the emergence of DirectX 10, which also corresponds to OpenGL 3.x , was followed so quickly by DirectX 11, which also corresponds to OpenGL 4.y , is the fact that the tessellation of the original triangle can be performed most efficiently, when yet-another type of shader only performs the tessellation. But in principle, a Geometry Shader is also capable of performing the actual tessellation, because in response to one input-triangle, a GS can output numerous points, that either form triangles again, or that form triangle strips. And in fact, if the overall pattern to the tessellation is rectangular, triangle strips make an output-topology for the GS, that makes more sense than individual triangles. But I’m not going to get into ‘Geometry Shaders coded to work as Tessellators’, in this posting.

Instead, I’m going to focus on a different aspect of the idea of micropolygons, that I think is more in need of explanation.

Our GS doesn’t just need to displace the micropolygons – hence, the term ‘displacement shader’ – but in addition, must compute the normal vector of each Point output. If this normal vector was just the same as the one for the triangle input, then the Fragment Shader which follows the Geometry Shader, would not really be able to shade the resulting surface, as having been displaced. And this would be because, especially when viewing a 3D model from within a 2D perspective, the observer does not really see depth. He or she sees changes in surface-brightness, which his or her brain decides, must have been due to subtle differences in depth. And so a valid question which arises is, ‘How can a Geometry Shader compute the normal-vectors of all its output Points, especially since shaders typically don’t have access to data that arises outside one invocation?’

One concept which has been used often in the design of Fragment Shaders and/or Materials, is “DOT3 Bump-Mapping”. The way in which this scheme works is rather straightforward. A Bump-Map, which is being provided as one (source) texture image out of several, does not define coloration, but rather relief, as a kind of Height-Map. And it must first be converted into a Normal-Map, which is a specially-formatted type of image, in which the Red, Green and Blue component channels for each texel are able to represent floating-point values from (-1.0 … +1.0) , even though each color channel is still only an assumed 8-bit pixel-value belonging to the image. There are several ways to do this, out of which one has been accepted as standard, but then the Red, Green and Blue channels represent a Normal-Vector and its X, Y, and Z components.

The problem arises in the design of simple shaders, that this technique offers two Normal-Vectors, because an original Normal-Vector was already provided, and interpolated from the Vertex-Normals. There are basically two ways to blend these Normal-Vectors into one: An easy way and a difficult way.

Using DOT3, the assumption is made that the Normal-Map is valid when its surface is facing the camera directly, but that the actual computation of its Normal-Vectors was never extremely accurate. What DOT3 does is to add the vectors, with one main caveat. We want the combined Normal-Vector to be accurate at the edges of a model, as seen from the camera-position, even though something has been added to the Vertex-Normal.

The way DOT3 solves this problem, is by setting the (Z) component of the Normal-Map to zero, before performing the addition, and to normalize the resulting sum, after the addition, so that we are left with a unit vector anyway.

On that assumption, the (X) and (Y) components of the Normal-Map can just as easily be computed as a differentiation of the Bump-Map, in two directions. If we want our Normal-Map to be more accurate than that, then we should also apply a more-accurate method of blending it with the Vertex-Normal, than DOT3.

And so there exists Tangent-Space Mapping. According to Tangent-Mapping, the Vertex-Normal is also associated with at least one tangent-vector, as defined in model space, and a bitangent-vector must either be computed by the Vertex Shader, or provided as part of the model definition, as part of the Vertex Array.

What the Fragment Shader must next do, after assuming that the Vertex- Normal, Tangent and Bitangent vectors correspond also to the Z, X and Y components of the Normal-Map, and after normalizing them, since anything interpolated from unit vectors cannot be assumed to have remained a unit vector, is to treat them as though they formed the columns of another matrix, IF Mapped Normal-Vectors multiplied by this texturematrix, are simply to be rotated in 3D, into View Space.

(Above Corrected 07/05/2018 . )

I suppose I should add, that these 3 vectors were part of the model definition, and needed to find their way into View Space, before building this matrix. If the rendering engine supplies one, this is where the Normal Matrix would come in – once per Vertex Shader invocation.