Non-photorealistic Rendering (Cel shading)

Non-photorealistic rendering can be used in stylized computer games, animated films, interactive comics, for technical illustrations, for simulation of artistic drawing and hatching, etc. In many cases such rendering reduces amount of information that users have to perceive.

This article describes how to create cel shading effect (Toon shading) with help of OpenGL and GLSL. This effect adds a silhouette to an image and reduces number of colors that are used.

There're different methods to create cel shading effect. In simplest case, mesh is visualized twice: to create silhouette and to create stylized colors. Two different shaders are required to accomplish this.

Creation of silhouette

First of all to create silhouette we should render enlarged mesh. This can be done with help of simple shader that assigns constant color (color of silhouette - black) for each rendered fragment and shifts each vertex of the mesh along normal of the vertex by constant value (size of silhouette). You should allow rendering only of back-faces of the mesh. First image depicts slightly enlarged mesh that is rendered with black color.

Visualize same mesh once more with same shader. But now set shift along normal to 0 and constant color to white, disable writes to depth buffer and enable visualization of front-faces of the mesh. Result is depicted on second image. As you can see, result of second draw call is overlayed over result of first draw call (if you need not only silhouette, but also effect of decreased number of colors, then this second draw call isn't required).

By shifting each vertex along normal and additionally in constant direction you can achive effect of "fat" mesh (as on third image).

Decreased number of colors

Effect of decreased number of colors is calculated in fragment shader. Calculate each component of standard Blinn-Phong lighting (ambient, diffuse and specular) and add them together. Clamp total value to range [0, 1]. With help of this value you can access to 1D texture that contains a set of colors for cel shading. High total lighting value will sample colors from right part of the texture, and low lighting intensities will sample colors from left part of the texture. Number of different colors on cel shading results will be same as number of colors in this texture.

Cel shading effect can be achieved without usage of intensity to color texture. You can define base color for cell chading and quantize value of total intensity. For example by following statement:
shadeIntensity = ceil(intensity * numShades)/numShades;

Ceil() function rounds input argument upward and returns the smallest integral value that is not less than input argument. NumShades - number of different colors that are required for cel shading.

ShadeIntensity value can be used for modulation of cell shading base color, as on first image. If mesh has diffuse texture, then you can modulate sampled color by value of ShadeIntendity, as on second image. Or you can mix cel shading base color with color sampled from diffuse texture, and modulate result with shadeIntensity value, as on third image.
finalColor = baseColor * shadeIntensity; // modulation base color
finalColor = modelTexture * shadeIntensity; // modulation of color from texture
finalColor = baseColor * modelTexture * intensity; // mixed modulation

Effect of metallic cartoon

There are many other ways to produce non-realistic images. And many of them are quite easy to implement. For example metallic cartoon effect. You can add this effect to your cel shading. Only slight modification of shader is required. If angle between normal and direction to camera is higher than defined value, then fragments' intensity is decreased. For example:
float metallic = dot(v_normal, v_directionToCamera); // cos of angle between N and V// if cos > 0.6 - full intensity// if cos < 0.4 - low intensity// if cos is within [0.4, 0.6] - interpolate values
metallic = smoothstep(0.4,0.6,metallic); // smooth interpolation between values// shift matallic value from range [0, 1] to range[0.5, 1]
metallic = metallic/2 + 0.5; // shift metallic intensity by 0.5
o_FragColor.xyz = metallic * u_baseColor; // modulate final color