Tutorial 13 : Normal Mapping

Welcome for our 13th tutorial ! Today we will talk about normal mapping.

Since Tutorial 8 : Basic shading , you know how to get decent shading using triangle normals. One caveat is that until now, we only had one normal per vertex : inside each triangle, they vary smoothly, on the opposite to the colour, which samples a texture. The basic idea of normal mapping is to give normals similar variations.

Normal textures

A “normal texture” looks like this :

In each RGB texel is encoded a XYZ vector : each colour component is between 0 and 1, and each vector component is between -1 and 1, so this simple mapping goes from the texel to the normal :

normal=(2*color)-1// on each component

The texture has a general blue tone because overall, the normal is towards the “outside of the surface”. As usual, X is right in the plane of the texture, Y is up (again in the plane of the texture), thus given the right hand rule Z point to the “outside” of the plane of the texture.

This texture is mapped just like the diffuse one; The big problem is how to convert our normal, which is expressed in the space each individual triangle ( tangent space, also called image space), in model space (since this is what is used in our shading equation).

Tangent and Bitangent

You are now so familiar with matrices that you know that in order to define a space (in our case, the tangent space), we need 3 vectors. We already have our UP vector : it’s the normal, given by Blender or computed from the triangle by a simple cross product. It’s represented in blue, just like the overall color of the normal map :

Next we need a tangent, T : a vector parallel to the surface. But there are many such vectors :

Which one should we choose ? In theory, any, but we have to be consistent with the neighbors to avoid introducing ugly edges. The standard method is to orient the tangent in the same direction that our texture coordinates :

Since we need 3 vectors to define a basis, we must also compute the bitangent B (which is any other tangent vector, but if everything is perpendicular, math is simpler) :

Here is the algorithm : if we note deltaPos1 and deltaPos2 two edges of our triangle, and deltaUV1 and deltaUV2 the corresponding differences in UVs, we can express our problem with the following equation :

Just solve this system for T and B, and you have your vectors ! (See code below)

Once we have our T, B, N vectors, we also have this nice matrix which enables us to go from Tangent Space to Model Space :

With this TBN matrix, we can transform normals (extracted from the texture) into model space. However, it’s usually done the other way around : transform everything from Model Space to Tangent Space, and keep the extracted normal as-is. All computations are done in Tangent Space, which doesn’t changes anything.

Do have this inverse transformation, we simply have to take the matrix inverse, which in this case (an orthogonal matrix, i.e each vector is perpendicular to the others. See “going further” below) is also its transpose, much cheaper to compute :

invTBN=transpose(TBN)

, i.e. :

Preparing our VBO

Computing the tangents and bitangents

Since we need our tangents and bitangents on top of our normals, we have to compute them for the whole mesh. We’ll do this in a separate function :

Finally, we fill the *tangents *and *bitangents *buffers. Remember, these buffers are not indexed yet, so each vertex has its own copy.

// Set the same tangent for all three vertices of the triangle.
// They will be merged later, in vboindexer.cpp
tangents.push_back(tangent);tangents.push_back(tangent);tangents.push_back(tangent);// Same thing for binormals
bitangents.push_back(bitangent);bitangents.push_back(bitangent);bitangents.push_back(bitangent);}

Indexing

Indexing our VBO is very similar to what we used to do, but there is a subtle difference.

If we find a similar vertex (same position, same normal, same texture coordinates), we don’t want to use its tangent and binormal too ; on the contrary, we want to average them. So let’s modify our old code a bit :

// Try to find a similar vertex in out_XXXX
unsignedintindex;boolfound=getSimilarVertexIndex(in_vertices[i],in_uvs[i],in_normals[i],out_vertices,out_uvs,out_normals,index);if(found){// A similar vertex is already in the VBO, use it instead !
out_indices.push_back(index);// Average the tangents and the bitangents
out_tangents[index]+=in_tangents[i];out_bitangents[index]+=in_bitangents[i];}else{// If not, it needs to be added in the output data.
// Do as usual
[...]}

Note that we don’t normalize anything here. This is actually handy, because this way, small triangles, which have smaller tangent and bitangent vectors, will have a weaker effect on the final vectors than big triangles (which contribute more to the final shape).

The shader

Additional buffers & uniforms

We need two new buffers : one for the tangents, and one for the bitangents :

And one for the 3x3 ModelView matrix. This is strictly speaking not necessary, but it’s easier ; more about this later. We just need the 3x3 upper-left part because we will multiply directions, so we can drop the translation part.

These three vector define a the TBN matrix, which is constructed this way :

mat3 TBN = transpose(mat3(
vertexTangent_cameraspace,
vertexBitangent_cameraspace,
vertexNormal_cameraspace
)); // You can use dot products instead of building this matrix and transposing it. See References for details.

This matrix goes from camera space to tangent space (The same matrix, but with XXX_modelspace instead, would go from model space to tangent space). We can use it to compute the light direction and the eye direction, in tangent space :

So we’ve got everything we need now. Diffuse lighting uses clamp( dot( n,l ), 0,1 ), with n and l expressed in tangent space (it doesn’t matter in which space you make your dot and cross products; the important thing is that n and l are both expressed in the same space). Specular lighting uses clamp( dot( E,R ), 0,1 ), again with E and R expressed in tangent space. Yay !

Results

Here is our result so far. You can notice that :

The bricks look bumpy because we have lots of variations in the normals

Cement looks flat because the normal texture is uniformly blue

Going further

Orthogonalization

In our vertex shader we took the transpose instead of the inverse because it’s faster. But it only works if the space that the matrix represents is orthogonal, which is not yet the case. Luckily, this is very easy to fix : we just have to make the tangent perpendicular to the normal at he end of computeTangentBasis() :

t=glm::normalize(t-n*glm::dot(n,t));

This formula may be hard to grasp, so a little schema might help :

n and t are almost perpendicular, so we “push” t in the direction of -n by a factor of dot(n,t)

Remember : don’t use immediate mode in real world ! Only for debugging ! And don’t forget to re-enable the core profile afterwards, it will make sure that you don’t do such things.

Debugging with colors

When debugging, it can be useful to visualize the value of a vector. The easiest way to do this is to write it on the framebuffer instead of the actual colour. For instance, let’s visualize LightDirection_tangentspace :

color.xyz=LightDirection_tangentspace;

This means :

On the right part of the cylinder, the light (represented by the small white line) is UP (in tangent space). In other words, the light is in the direction of the normal of the triangles.

On the middle part of the cylinder, the light is in the direction of the tangent (towards +X)

A few tips :

Depending on what you’re trying to visualize, you may want to normalize it.

If you can’t make sense of what you’re seeing, visualize all components separately by forcing for instance green and blue to 0.

Avoid messing with alpha, it’s too complicated :)

If you want to visualize negative value, you can use the same trick that our normal textures use : visualize (v+1.0)/2.0 instead, so that black means -1 and full color means +1. It’s hard to understand what you see, though.

Debugging with variable names

As already stated before, it’s crucial to exactly know in which space your vectors are. Don’t take the dot product of a vector in camera space and a vector in model space.

Appending the space of each vector in their names (“…_modelspace”) helps fixing math bugs tremendously.

How to create a normal map

Created by James O’Hare. Click to enlarge.

Exercises

Normalize the vectors in indexVBO_TBN before the addition and see what it does.

Visualize other vectors (for instance, EyeDirection_tangentspace) in color mode, and try to make sense of what you see