Open-Source Compositing in Blender

Working with image sequences offers three distinct advantages over working
with video. First, with image sequences it's easier to retouch problems
on individual frames without reprocessing an entire clip. Second,
the variety of alpha-enabled lossless formats gives the operator a far
greater flexibility than do video formats, very few of which are lossless
or allow embedded alpha channels. Third, video codecs are, frankly,
a big pain in the neck. They vary wildly in support from one program
to another; however, image formats are universal and comparatively open.
Converting your video to image sequences before piping it through your
compositor or motion tracker means that you're going to encounter far
fewer problems with moving between various programs in your work flow.

With Blender, there is one further advantage to using image sequences, and
it has to do with a shortcoming of the program. Blender does not support
NTSC drop or nondrop frame encoding, both of which run at 29.97 frames
per second (fps). Because it's a European program and has historically
been an animation program, the closest approximation it can hit is 30fps.
Though .3fps seems like a trivial difference, it's more than enough to slip
your sound sync beyond the limits of intelligibility, so it's far better
to work with image sequences and then re-multiplex your audio with your
video in your video editing software, such as KDENLIVE or OpenMovieEditor.

First, I build the nodes chain for the background image (Figure 6). The first node, moving
left to right, is our source node—the photo itself. The second is
a scale node, which I use to shrink my rather large digital photograph
down to something approximating SD video resolution. Next, I've used an
RGB curves node to blow the highlights out of the photo to help sell
the illusion that our character is looking from a shaded courtyard out
into a wild garden in direct sunlight.

Figure 6. The Background Nodes Tree

Next, I take the pillars element, which is a rendered
layer from within Blender proper, add a procedural glow layer to it,
and marry the glow and the pillars to the background. To do this,
I take an output from the source and run it through a curves pass
to cut out all but the brightest highlights (Figure 7).

Figure 7. Pillars and Glow Pass

I pipe the output from the curves node into a blur node, where I do
a 40-pixel x/y tent blur and then direct that into a Screen node,
where the glow is composited back over the source picture of the pillars.
This married image is then piped into an AlphaOver node, which pastes
the pillars and glow over the top of the photo.

Now we come to the color keying. There are a number of ways to do
color keying in a node-based compositor, not least among them
is building your own keyer out of basic mathematical functions. Although this
method is excellent and yields the best results (even on DV footage,
which has some very particular problems with color keying), it requires
far more ink than I have space for here. So, for the sake of brevity,
I selected one of Blender's three native keying nodes, the channel key,
to do the job (Figure 8).

Figure 8. The Color Keying Nodes Tree

The footage, first off, needs some prep, and I've prepared the same
footage two different ways—one for pulling the matte and the other for
color matching. I first ran the footage through a scaling node to correct
for the 16:9 aspect ratio I shot in—as the rest of my elements are in
4:3, I'm pre-correcting the footage rather than doing it at render time.
I then ran it through a translate node, which allowed me to reposition
the footage to the left, so that we actually can look over the actress'
shoulder rather than just staring at the back of her head. From there, I
send the output into two parallel subtrees—keying and color correction.

The keying subtree begins with a curves node, which pushes the green in
the greenscreen into a narrow band to make it easier for the keyer to
latch on to. Then, the output links to the input of a channel keyer,
set to pull the cleanest possible matte (which I accomplished by hooking
a viewer node to the image output, so I could see what I was doing when
I played with the settings). The resulting matte is then run through a
blur node. Normally, when keying DV footage, I would apply a 4x2 blur to
soften the matte and compensate for the edge artifacting introduced by
the DV compression. However, in this case, my edges were dark, because
of how I lit the original scene, and I needed some extra feathering so
the brightness from the background would bleed over. The output of
this blur node is then piped into the Fac input of an AlphaOver node,
which marries the greenscreen footage to the rest of the image. But,
I'm getting ahead of myself.

Let's back up to the other half of the keying tree. This takes an
additional output from the translate node into a curves node, which is set
to tamp down the green channel to get rid of the green spill and help sell
the different lighting conditions of the foreground vs. the background.
The output of this curves node is then run into the bottom input on
AlphaOver. Now, to complete the marriage of foreground with background,
we run an additional noodle from the AlphaOver node at the end of the
background subtree into the top image input on the keyer AlphaOver node.

I could leave things here, but the shot could use a little extra touch
to tie all the layers together. To accomplish this, I created a nice
lens flare and brought it in to Blender. I ran it through a translate
node to put it into the right spot, and from there into another screen
node, which lays it over the top of the previous composite. To do this,
the lens flare went into the top image input, and the previous AlphaOver
node went into the bottom image input, and I messed with the Fac, so I
got the right effect—just a hint of extra brightness and anamorphic
smear, which helps sell the integration of the different layers (Figure 9).

Figure 9. Layers of the Image Including Lens Flare

Now, all that remains is to hook up the Composite node, which is what
Blender draws from for its output. This can be found next to the viewer
node under output in the add nodes menu, which you get to by pressing
the spacebar. Once the composite node is hooked up to the output,
go to the renderbuttons window at the bottom of the screen, depress
the Do Composite button, and click Render, or, if it's an animation, click
Anim (Figure 10). The result of your hard work appears in the
render window, from where you can save it using F3 (if it's a still).
Or, you can find it on your hard drive in the temp directory or another
directory that you have set for output in the renderbuttons window.

Figure 10. Click Render or Anim in the Blender Controls

Figure 11. The Completed Project

Simple though this project is, it gives a good grounding in how nodes
work and why they're useful. Enough access to basic image processing
functions is included that the capabilities are very deep and extensive,
and because of Blender's support for HDR formats, such as OpenEXR, and its
lack of limitation on resolutions, it is squarely in the professional
compositing camp, albeit at the less-sophisticated end of the spectrum
(as one would expect from a brand-new project). It is advancing quickly.
In future versions, more user-friendly keying tools and color tools
are planned, and hopefully there also will be more direct access to
the translation and garbage matting functions, which at the moment are
obtuse and inconvenient. Until such tools emerge, I highly recommend
that anyone wanting to use Blender as a workhorse compositor invest in a
book that teaches how compositing works, both in theory and practice.
The best available is The Art and Science of Digital
Compositing (The
Morgan Kaufmann Series in Computer Graphics).

Using Blender for composite work has significant advantages as well,
since it's an integrated part of a 3-D content creation suite, the particle
systems, fluid systems, procedural textures and all the traditional 3-D
modeling and animation tools are at the compositing system's disposal,
which is supremely useful for any number of highly complicated shots
that normally would require using several programs in conjunction to
pull off correctly.

Here's hoping the Project Peach, the currently in-process sequel
production to Elephants Dream, gives us more such innovations that push
the compositing system to the next plateau. Until then, there is much
to explore, learn and use.

Open-source compositing has finally arrived. Enjoy!

Dan Sawyer is the founder of ArtisticWhispers Productions (www.artisticwhispers.com), a small audio/video studio in
the
San Francisco Bay Area. He has been an enthusiastic advocate for free and
open-source software since the late 1990s, when he founded the
Blenderwars
filmmaking community (www.blenderwars.com). Current
projects include the independent SF feature Hunting
Kestral and The Sophia Project, a
fine-art
photography book centering on strong women in myth.

HI, I really liked your coments on node based compositing facilities being ofered in Blender. Its true that I am a newbie for video editing and all the other software stuff but that can not stop me from being a huge fan of open source . Lately I v been researching the different capabilities of open source multimedia content ceation softwrae and closed source memebers of the same family t to figure out whether we (OPen Source Lovers ) have beaten the closed source giants. It made me realize that even though vector graphics,raster graphics ,and audio editing and even 3D content creation has already givin promises to be catching up fast.video compositing has ben a bit laging behind.cinerrella seems to do the job right but its stubbornly aiming only Linux platform. For it to beat those Windows based counterparts it l have to battle in the samwe ground.
jahshaka is looking good but where have al the developers gone who were there earlier for it. Open source guy we need to unite to start the battle with cloced Sources!!!

Trending Topics

Upcoming Webinar

Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report

August 27, 2015
12:00 PM CDT

DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.