Previously, I proposed that a major distinction between NPR and ‘realistic’ rendering is intention in dealing with the tools and materials involved. But what is material within the immaterial working space of computer graphics? And can the artistic sense of authenticity play a role in computer graphics? Can any one produced visual be considered more authentic than another?

You could argue that G.R. Exper’s fractal work (lower right) is more authentic than Weta’s Smeagol, as the fractal is visualizing a self-continuing algorithm. But I suspect they are equally justified. They both give the computer something and get something back. Right?

Ascii art (here by kaboz ) allows a glimpse from another direction. The artist finds and manipulates an existing something – software. He’s found software for writing text and used it to create imagery, allowing its existing form to influence the final work’s appearance.

For the consumer-artist who works with, around and against existing software, at least, there seems to be a guiding element in cg – and this emptiness becomes a material. In a similarly extreme example, what would result if an artist were to restrict himself to Microsoft Excel?

3D software has traditionally been full of materiality, as the brute force calculations required for the loyal replication of light and vision were simply too expensive. But as processing power grows more powerful and lots cheaper, these alternative methods and forced work-arounds are replaced with tweakable settings of singular models… the renderers are becoming so real, that they sacrifice their potential for alternative imagery.

consumer artist; software as material

Here’s a scenario from the world of 3D. Way back when, soft shadows and area lights were prohibitively expensive, if they existed at all. Shadow mapped spotlights were the lighting tool of the day, with sharp edges and aliasing issues corresponding to the resolution used.

To make things worse, smoothly smeared motion blur was also too expensive, so an iterative technique was used calculating x images

between frames. The more iterative steps, the smoother the blur and the longer the render. What a restriction!

Right? Yet clever artists such as erkki halkka and dave jerrard developed a method called the spinning light trick, which employed both these shortcomings to great effect.

The methodology here is based on the artistic concept of materials: the artist finds existing conditions, analyzes their artifacts and creates a previously inexistent usage via experimentation. The next step was pursued by a number of artists in the community, including myself: this method was expanded to further look development, namely the age-old issue of the too-perfect contours generated by 3D rendering processes. The mesh is displaced at a rate greater than the rendered frames-per-second… and synchronized with iterative motion blur. This gives us a number of discoherent images. By offsetting the displacement, an element of boil can be introduced.

As 3D prosumer software grows increasingly sophisticated and processing capacities cheaper, these approaches are disappearing. Like wild tomatoes getting plowed under for industrially produced veggies, they’ve given way to physically correct approximations of ‘reality’.

Why is it important to pursue ‘non-real’ techniques?

An artistic approach is concerned with an authentic recognition of materiality, of tools… and the viewer’s eye expects to see this approach both in the form and subject of the artwork and also in the artifacts left on its surface. Without these, the work feels empty. Numerous applications are available that are more than capable of fulfilling these criteria in still imagery. But moving images reveal that there is much work to be done. Each material alluded to in an animation – pigment, paper, binding substance, etc. brings with it a clear expectation of how it will and won’t move.

Confronted with this drawing/collage, we expect the crumpled paper to move as if photographed via pixelation, with deformation and lighting effects of the paper surface overlapping sequential photographs of the woman or perhaps a cut-out animation technique. Fibers from the paper’s edge might trail behind and the pigment and ink might smear. If these things are missing – if the temporal qualities are too perfect and smooth, we are likely to dismiss the result as unconvincing. It is in danger of becoming artificial as a result of lacking materiality.

It is interesting to note that – to a degree – the expected behavior does not exist and has never been seen. No watercolor animation has been made that incorporates the act of the pigment dispersing and drying. But computer simulations will make this movement possible. The npr artist must invent the temporal look to coincide with the expectations aroused by the represented materials.

Question: Is an animator’s goal to achieve a perfect simulation of “real life”?

Pixar always strives for believability instead of realism. When you make humans a little more stylized, like we tried for in The Incredibles, the audience can accept them as human being–type creatures, stop comparing them to the real thing, and instead just enjoy the story. However, there are definitely some things where we strive for more realism, like smoke, fire, explosions, and waterfalls. All of these things tend to look very fake if they don’t have some of the proper physics behind them. If one thing goes out of whack, the whole thing can look phony and pull the audience out of the story.

This is a re-post of parts of a talk I held at fmx/07 , together with Oliver Deussen from the University of Konstanz.

There is much frustration at the term “non-photorealism”. Something which defines itself as the negative of another thing is side-stepping an identification, of course – but more difficulty arises from the presumed assumption that we know what photorealism is. A director requesting “photorealism” as a visual style from a cameraman would rightly be scoffed at, yet cg artists are often confronted with exactly this definition of a desired style.

Photorealism isn’t a style.

The photo is a projected image captured when light falls on a light-sensitive surface. Initially, this process took place chemically, such as in the heliograph above – taken in 1826 by Joseph Nicephore Niepce. It could be considered the first photograph, certainly its the first permanent photograph. Earlier images had been created using photographic processes but they vanished after a short time.

The photoreal takes its reference from the physical properties of light as it finds its way through a focusing lens or pinhole and onto the flat plane of a photographic surface. As the image above shows, photoreal doesn’t necessarily refer to a crisp, focused representation of an existent scene… it includes all effects and artifacts arising from the physical properties of light and the chemical processes involved, as guided and manipulated by the photographer. Computer graphics has pursued rendering the photoreal within the limits of computational power and understanding of the processes.

In contrast, the non-photoreal takes as its reference the interaction between artist and material. At first glimpse, that says pretty much nothing. But its a solid point of departure to better define what it is that NPR research strives to achieve.

The intention of the photographer or the cameraman is to ‘capture’ existing objects – of course, both these objects and the photographic toolkit are both manipulated to achieve the desired visual and emotional impact. He/she is concerned with subject matter, composition, lighting and the camera’s position and viewing angle, focus, etc.

The artist is also interested in these things, but – to a much greater degree than the photographer – the artist is concerned with technique – the interplay of tool and material at the hand of the artist. Tools and materials are expected to influence or even guide the creation of an artwork.

materiality; the photographer looks, the artist listens

The grain of a tree stem suggests form to the sculptor, the grain of a woodcarving leaves its signature in ink as it is pressed against paper. It is a sign of authenticity that the artist listens to materials. As the sculptures below illustrate, one material can suggest endless possible forms and techniques, and the work lives from the tension between this material and its development.

two eyes are more than one

I see a further distinction in the multiplicity of perspective: looking through the camera means reducing one’s vision to one point in space. Artists can also pursue this singular perspective, but more often than not, they are in some form juggling multiple perspective. The extreme can be seen in cubism.

temporal concerns: it’s about time

A last distinction: only seldom does photography span longer frames of time… such as cameras set to capture sprinters as they cross the goal line. Most often, the photographer – even the cameraman – is intent on a fraction of time measured in f-stops. Even time-lapse photography consists of consistent segments of time. The artist is constantly confronted with temporal incongruities, for the simple fact that the process of formulating an image spans a longer period of time.

More about these distinctions in future posts…
next: NPR _ materials in the void

Luckily, I was able to meet Nina Paley and watch her inspirational film Sita Sings the Blues at last May’s FMX in Stuttgart, Germany. Since then, I’ve been following her struggle with distribution, copyright and (if I recall correctly) lice. I continue to be in awe of her achievement in making this film, and the qualities of the film itself. against that background I stumbled across this review by Roger Ebert.

In this wonderful review of a wonderful film, Ebert says something of central relevance to npr and stylistic development:One remarkable thing about “Sita Sings the Blues” is how versatile the animation is. Paley works entirely in 2-D with strict rules, so that characters remain within their own plane, which overlaps with others. This sounds like a limitation. Actually, it is the source of much amusement. Comedy often depends on the device of establishing unbreakable rules and then finding ways to cheat on them and surprise you. The laughs Paley gets here with 2-D would be the envy of an animator in 3-D. She discovers dimensions where none exist.

The appeal of npr stems in part from limitations surmounting themselves. This is true not only of what happens in the animation, as Ebert observes, but to some degree in the animation itself. The cut-out concentric circles are received by the viewer as two things at once… simultaneously, they are vector cg graphics and Sita’s eye. They are a transformation. While this true of illustration as well, it is more evident when viewing animation. This transformation is suspended, held in levitation – and our brains are subtly tickled in a way that is related to the functioning of comedic surprise. The artist’s pen is a suspended ‘reveal’ – and the eye lingers in hopes of catching the device’s resolution. At least, I suspect so. This would help explain why animated content is repeatedly viewed. .

I very much look forward to Spike Jonez’ upcoming Where the Wild Things Are. I expect it to be a visionary project with Jonez’ unique signature and I see it cooking up some debate about look development in film adaptions – as it has already been doing.

Forest Whitaker says:
It’s one thing to read [scary stuff] in a book, but when you see an itty-bitty kid running alongside a 10-foot-giant on the side of a cliff, it gets intense.

Spike Jonez says:
I wanted it to feel “real,” or not-real because it’s not “real,” I wanted it to feel like… like when I was a kid, and I would play with my Star Wars action figures, or read Maurice’s books and imagine me being Mickey in IN THE NIGHT KITCHEN, or whatever it was… it felt like it was everything, you know? It’s like your imagination is so convincing to yourself that… you’re there, you’re in it. And I wanted this movie to take it as seriously as kids take their imagination and not, like, fantasy it up.

Moriarty says (about the Sendak book “Where do I come from?”):
…the cartooning style got the harder things past you, so you’re able to grapple with some of those bigger ideas. What I loved about Maurice’s book and still love about it is that it’s about emotional states, and what a crazy thing it is to write about for children… how sometimes you have these emotions that are just so big you can’t control them.

Witnessing this wonderful struggle for authenticity in story and visual means of transporting a story’s emotion, I wonder how npr might be used to tackle the same issues. Could it transport the feeling of spontaneity? The “rough and organic” feel with nothing of an artificial environment that Moriarty praises? How might reactions to the film’s look have differed had the original Sendak artwork been used more directly as a point of departure? Could a wonderful director such as Jonez work with npr stylistics as masterly as he does with the real Australian landscape?

And beyond the level of look development, there’s the element of communicating this target look within the community team of producing a film. I catch myself grinning that I-feel-your-pain kind of grin when I read this by Jonez: I told the studio, “I don’t think this is gonna be a movie for four-year-olds.” And I think they said “Oh, okay,” but I think that when they saw it, that’s another… you know, that’s something else.”Determining expectations is difficult in any project, but the clash of cultures evident in Jonez’ art-speak is too beautiful. Just visualize what the ‘family-film’ suits at Warner Brothers had in mind when they hired Jonez and when they saw those grooved, raked talons hovering over a little boy’s shoulder in screening. Let’s hope they have the wisdom to allow it through as the director envisions it, and not sliding down that slippery slope of second-guessing.

Perspective 101: draw a telephone pole midway between the two poles above.
Then guess its position…

The knowing artist (or the 3D artist) can quickly calculate the correct position…

And places a pole that looks…. wrong:
The eye has a difficult time accepting the correct position as the true middle. So the experienced artist goes one step further and cheats a more acceptable result:

Much better, even if its wrong. This age-old example reveals a truth about stylized look development that comes naturally to traditional artists and less so to 3D artists… right is sometimes wrong. I suspect this has to do with the exclusion of fine-level detailing that would anchor the positions in a realistic scale and space. Without these supporting details, the eye questions otherwise correct positons, balancing them with empty space and compositional aspects of the image.

Right can be Wrong: once you accept this principal, however, countless opportunities to ‘right’ 3D wrongs pop up. I’ll be posting some cheats here…

non-photorealistic rendering

It’s a good thing that people get passionate about stylized looks visually – by viewing animations or imagining how an illustration would look should it come to life. Its a good thing, because it would be difficult to arouse interest using the word used to describe this undertaking: non-photorealistic rendering or “NPR”.

researcher npr

A graphics researcher is likely to interpret “NPR” by the mechanisms with which traditional techniques of image creation such as oil painting, watercolors, charcoal, etc. can be replicated. This is already a wide and varied field, concerned with simulation of pigment dispersion, artistic input or analogy-based replication of specific painting styles. The intention is both scientific (to propose and confirm underlying principles) and practical (to the degree in which these principals can be tested in a proof-of-concept).

zelda / William Kentridge / Pixar "Cars"

An artist is likely to understand “NPR” as a sketchy line rendered from a 3D object, or – more often than not – cel-shading with clean color gradients as in the image of Zelda above. The more technical artist will argue that the slick look that Pixar achieves in its films is not “NPR” – but rather a stylized realism, and argue that the surfaces and contours rely on physically correct models of light in determining their properties. The less technical (or ‘traditional’) animator may even be thinking of stop-motion or claymation when we mention 3D, and may well have National Public Radio in mind when we speak of NPR.

A producer – who has every reason to be attentive to a film’s look – is likely to draw a complete blank at the term – and otherwise have a dangerously limited vocabulary to discuss what a ‘look’ is or isn’t. Worst case, he or she will rely on empty verbiage like “the style we want is photorealism” or “we want it to look like Pixar”. So… what is non-photorealistic rendering?

Cel Shading – a case history

Cel-shading is a good example of the issues involved in defining and developing NPR looks and tools. A cel is a transparent sheet of acetate. Once upon a time, artists would ink in or xerograph frames of animation on cels, then paint in color fields on the reverse side. These would be then laid over background plates using register marks and photographed. Subsequent cels would be replaced for the subsequent frames. While hand-drawn animation, tweening and clean-up is still done on paper, the coloring process once done on cels has long since been replaced by digital processes.

And its easy to see why: thousands of individual images are required for an animated film – even a ten-minute film on twos (12 fps) consists of 7,200 individual frames. The cel technique lived and died beneath the forces of efficiency. The experienced animator is difficult to replace, and so has some weight in deciding whether paper or digital tools are used. Also, the increase in efficiency by digital drawing is mostly due to digitalization processes, so if you already have a scanning pipeline set-up that’s not such a big argument. But the coloring assistant is quickly trained and the production speed is increased dramatically with digital tools. So the cel technique has disppeared.

Cel animation was a technique – not a style.

Typical of NPR, we now refer to a technique as a style and do our best to replicate something we haven’t fully analyzed. The cel technique supported a wide variety of looks, from Bambi and Bugs Bunny to Akira. As Yuri Norstein showed, it could be combined with cut-out animation and stop-motion techniques to create lush, personal visions. The results of the techique were as varied as the artistic visions employing it, yet marked by the materials, tools and intentions that defined it.

We might characterize cel animation as:

– animated figures made up of clear shapes, each made up of one key color
– an outline of (more or less) clean ink lines
– still background plates (perhaps panned or zoomed)
– additional effects via layers of cels

The backgrounds generally have a higher level of detail and are more painterly in quality, as the artist can take more time with them – they appear throughout a larger number of frames. The shapes are sometimes given more form with additional tonal values alluding to shadow and highlight, which reflect the position of a light source and the represented surface information.

In the image above (from Ghost in the Shell), for example, the shapes read as certian materials with very little textural information: stretched, jagged forms read as hair (long, bunched fibers and anisotropic highlights) and a pink shape with a tight, round specular reads as lips. The shapes of shadow and highlight have little to do with the form of an actual contoured head… they deliver just enough clues to be intuitively read by the viewer.

Most NPR looks have this in common: material attributes are not portrayed directly, but rather communicated with the use of the stylistic means available. Hair is not portrayed fiber for fiber, but in the careful exageration of specific qualities and the reduction / exclusion of others.

Then there are the temporal qualities:

– animated figures have flexible frame rates according to the required motion (rapid movements can be on ones, normal motion on twos, and long holds can be held for a longer number of frames)
– looped elements can be used to keep still character from “freezing” (ie. lines boil, or hair sways in wind)
– motion blur is sometimes faked with brushed lines, but more generally incorporated into the character’s shapes via stretching and deformation… smearing effects such as in charcoal animation aren’t supported by the dried pigments.
– layers might be animated to create parallax effects (moving clouds, etc.)

(Piglet / Boris Dezhkin)

And production concerns:

– the artistic process is highly segmented, with both highly skilled animators and layout artists and less skilled assistants assuming more tedious processes
– more detailed characters are generally associated with reduced temporal animation
– shadows from the lights are cast during photography, particularly if more than one cel are used
– in hybrid animation methods, the cels themselves might be seen…

A closer look at the inspiration reveals further characteristics that we may or may not wish to emulate. So the first step is to determine what goals the look needs to fulfill: production targets, compatibility between visual elements, etc. Gradually, new possibilities stretch this “style” beyond its inspiration as the remnants of the original technique give way to expanded artistic goals.

On to “cel shading” in 3D…

Here are two impressive examples of ‘cel shaded’ 3D art. (IronMan fan art / adam baroody ) These images make evident how much stylistic potential NPR looks can have. They also reveal how many issues can be taken for granted. Here, the contour defining shadows is much busier, as it correctly (and thus incorrectly) follows the volume of the represented form. High-interest areas such as the eyes are treated with much the same rules as broader, lower-interest areas such as the body chest. The viewer is presented with an image without hierarchy, as the artistic processes of exclusion and reduction in service to composition have been neglected. Yet the result is promising, and the production efficiency undeniable.

Adam Baroody’s test above is interesting in that he has taken imagery generated from 3D to use in 2D filter processes (Photoshop). In so doing, he was able to create more convincing line qualities which were unattainable directly from 3D geometry. I highly recommend visiting his tutorial. What I find fascinating in NPR look development is how consistently “correct” can look wrong.