Plaid Creature

Non-Photorealistic Rendering 101

Greetings, all. I’ve been asked by Dr. Conspicuous if I wanted to take his spot today since he wouldn’t be able to post. So I’ve accepted, and today we’re going to shift gears a bit. One of my passions is real-time computer graphics and the coding involved behind it. My Master’s thesis work, currently under development, is concerned with non-photorealistic rendering techniques and how they can be applied to games.  Non-photorealistic rendering (hereby to be called NPR for short) is the practice of rendering objects without following strict physical rules for rendering an object. In many cases NPR entails adding ‘artifacts’ to a rendered image either to make it more understandable or more aesthetically pleasing. A lot of NPR is concerned with attempting to make rendered scenes mimic hand-made media.

The neat thing about NPR in games is how easily it conveys a tone through a sense of style. Take, for example, the Legend of Zelda: Windwaker. It’s very light-hearted, almost child-like in nature. All the colors are very bright and vivid. The character’s proportions are very exaggerated. And the shading on the characters is like that of a cartoon (cel-shading or toon-shading), with large bands of color. These traits combine to create a very unique tone to the game, namely of being playful and adventurous. I’m very excited by the prospect of working on techniques and styles that have such a large impact on how the user interprets a game. I find rendering stylization to be sorely underused. Why, for example, does a game have comic book cut-scenes but is still rendered photorealistically? The style of the game should reflect the comic book feel, rather than having a huge disconnect between the game’s aesthetic and the aesthetics of the cut-scenes. Yes, I’m looking at you, inFamous.

Since I mentioned cartoon shading with Windwaker, I’ll go ahead and take the opportunity to explain how it is implemented since it’s very common and will help us cover some initial ground to get to some cooler stuff later on. Cel-shading, also called toon-shading, is a very common technique in games and animation and is named after clear sheets of acetate, called cels, used in traditional 2-D hand-drawn animation. The idea behind cel-shading is to create a cartoon-like quality to the image. The main trait of this style of rendering is the removal of smooth areas of shading on an object, instead conveying shading as large bands of color. The example below shows how the smoothness in the shading is removed in favor of distinct color bands.

This is implemented fairly simply in a game. First, note that the light an object receives is proportional to the orientation of the receiving surface to the light source. A flat surface aligned to a light (perpendicular to the direction of the light) will be fully lit while a surface parallel receives no light. A light somewhere between will be somewhat lit. This amount of scaling is implemented per pixel using the vector-dot product between the surface’s normal and the direction to the light and normally the value is from 1 to -1, and occurs at a per-pixel basis in the shader. A shader is essentially a set of instructions that can be run per vertex or pixel. Cel-shading works by applying a threshold to that value, creating a step-wise clamping that results in distinct bands of color whose number, size, and darkness depend on how the piece-wise threshold is specified. This piece-wise thresholding function is typically specified as a grayscale image so that it can be easily modified by artists on a per-object basis. I whipped up a few examples with the famous Utah teapot below to show you how the size, number, and blending of bands impacts the final shading on the object. On a related note, Valve uses this same technique for shading their characters in Team Fortress 2. Their threshold texture pulls the lights and darks together for their illustrative stylization, but still maintains a slight gradient between them, similar to the right-most example below.

I’ll end today with a taste of a watercolor technique I implemented in which cel-shading provides an intermediate abstraction step. Without giving away too many details, several paper and noise textures are used to provide the painterly effect. These textures are aligned to the screen, not to the surface of the object. This is one crucial aspect to a lot of NPR techniques, and there’s some pretty neat literature about the nature and interplay between 2D and 3D with artistic representation. I’ll try to provide some videos of these techniques in action, since they can’t be fully appreciated until they are seen in real-time. And it’ll also provide the perfect impetus for the idea of temporal coherence, and mapping those 2D screen-aligned patterns onto the 3D objects. Basically, imagine how the paper texture is supposed to move when the teapot rotates. It’s a pretty big issue I’m attempting to address in my thesis and it’s pretty tricky stuff.