Sunday, September 23, 2012

Video Art: Experimental Animation and Video Techniques in Linux

Animation and video editing in Linux can be treacherous territory. Anyone
who has tried working in these media probably has experienced the
frustration of rendering a huge file for an hour only to see the program
crash before the export is finished. A bevy of tools and applications for
manipulating video exist for Linux, and some are more mature than others.
The most mainstream of GUI applications have been covered quite a bit
in other Linux-related articles on the Web and in print, including in
previous issues of Linux Journal. Some of these names may ring familiar
to you: Kino, PiTiVi, Openshot, Cinelerra, Kdenlive and Open Movie Editor.
Although I refer to these nonlinear editors (NLEs) from time
to time here, the main purpose of this article is to introduce some video
effects and techniques you may not have thought of before. If you are
producing a film or animation in a conventional format, such as a DVD
or a Web video, you most likely will want to employ a suitable NLE at
some point in your process. Many of the ideas I present in this
article are experimental.

Video Editing

LiVES
LiVES is primarily a VJ (video jockey) tool for performing live
audio-visual effects, but it also can encode and export video via its
MPlayer back end. The interface has two modes: clip editor and multitrack
editor. The clip editor view is more suitable for live VJ sets, while
you'll probably lean toward the multitrack view if using LiVES as your
NLE.
Figure 1. LiVES in the Clip Editor View
LiVES is highly extensible. In addition to the built-in effects, you
can apply custom RFX (rendered/real-time effects) plugins. Several of
these scripts are available for download from the LiVES Web site. You
also can share LiVES' real-time effects with other applications using the
frei0r effects API.
The number of options and the advanced effects in LiVES are comparable
to those of Cinelerra, but I strongly recommend LiVES over the
latter. Cinelerra is indeed a powerful video editor, but the interface
is antiquated and difficult to use. Although LiVES can seem foreign to
new users, it is not hard to become acquainted with it.
ZS4
ZS4, formerly known as Zweistein, is a unique—and quite
strange—video
editor and compositor. The developers of ZS4, who go by the name
"t@b",
are a duo of musicians who use their own software to create music
videos. They are hard to pinpoint on the Web, as they use several sites
for different purposes.
I admit that I was confused by the existence of both zs4.net and
zs4.org, as well as the Viagra advertisement lines that appeared in
Google search results at the zs4.net domain. The two sites both contain
download links for ZS4 as well as some other software.
If you plan to use ZS4, I recommend downloading the t@b Media Converter
and/or installing Avidemux, as ZS4 is picky about importing video
files. Most videos are not compatible out of the box, so it may be
necessary to convert them to a format ZS4 can work with.
Working with ZS4 can be frustrating at first because the interface is
far from intuitive. Menus are not where you would expect them to be,
and you might find yourself aimlessly clicking your cursor in different
places to accomplish a simple task, such as dragging a media file into the
timeline. The media viewing windows are vaguely labeled
"rectangles". To
show or hide a track, you click on + or - instead of the typical
open- or closed-eye icon.
It took me years to gather the patience to learn my way around this
program. So yes, the GUI needs some serious work if it ever is going to
reach a mass audience, but it doesn't seem like mainstream appeal is a
major concern for the eccentric-minded developers.
So, why tell you about a bizarre-looking application that hasn't been
updated in years when there are plenty of other video editors for
Linux? Well, for all ZS4's graphical quirks, it can accomplish some very
interesting compositing effects.
Figure 2. Tiling Effects in ZS4

Animation

GIMP
The famous GNU Image Manipulation Program can create animations as well
as still images. Because it is a full-featured image editing program,
you can use it to create an animation entirely from scratch.
In order to import a prepared image sequence into GIMP, click
File→Open as Layers... or press Ctrl-Alt-o. The Open Image dialog
allows you to select multiple files, which then will appear as
layers.
Figure 3. An animation in progress
that I made by tracing reference photos of faces from the Psychological
Image Collection at Stirling (PICS).
In the example shown in Figure 3, I imported a series of reference
photos into GIMP and traced over them in cyan and then in black. I
eventually deleted the reference photos and blue layers, leaving only
the black-lined drawings that I planned to use for my final animation.
To finish my animation, I exported the layers as a GIF and specified
animation parameters in the export dialog. Because I wanted to use the
animation in a video, I had to turn the animated GIF into a video file. I
ultimately chose to do this by way of screen recording, but that is not
the only option.

From Stills to Movies

Let's say you have a sequence of images, or perhaps an animated GIF, that you
want to make into a video file. There are several ways to go about this.
Stopmotion
Stopmotion started as a student project under the Skolelinux/Debian-edu
organization in 2005. Although it hasn't been updated since 2008,
I find it to be a handy tool for anyone working with frame-by-frame
animation. You might have trouble finding Stopmotion in your
distribution's repositories if you aren't using a DEB- or RPM-based
package manager, but you can, of course, compile it from source on any
distribution; that's how I set it up in Sabayon Linux.
Stopmotion is simple and to the point, with a nice drag-and-drop
interface. It's not designed for heavy post-production or for drawing
and adding effects to frames. Rather, the point is to give users an easy
way to arrange images sequentially and export them into a video file.
The video import and export options are limited only by your imagination
(and your knowledge of the command line). If you know how to use FFmpeg
and/or MEncoder to convert image sequences to video, you can pass your
desired command-line arguments to Stopmotion, which is essentially a
GUI for those programs. Stopmotion also gives you several choices of
video capture commands for grabbing video from your Webcam or another
attached device.
One cool feature I didn't know about until I read the user's handbook
was the option to add sound. You can set a sound clip to start at any
given frame by double-clicking on it. The audio I added to my sequence
didn't play in the exported AVI, but maybe you'll have better luck.
If you want to perform more-advanced editing on your individual frames,
Stopmotion has a button to open a selected frame in GIMP. You
also can export your data into Cinelerra for video editing.
Figure 4. Animating a Sequence of Faces in Stopmotion

Command Line

There are several ways to turn frames into motion via the command line.
jpegtoavi
jpegtoavi is a simple C script that does exactly what its name
suggests—converts a sequence of *.jpg files into an AVI movie. If your
images are not in the JPEG format, you first can convert them using the
convert command from ImageMagick:

convert image.png image.jpg

If you need to convert a batch of images in a folder, ImageMagick
grants you about a gazillion different methods. One of these is to
cd to that directory and do:

convert *.png image.jpg

The new filenames will be numbered automatically.
Once you have your folder of sequenced JPEG files, you can employ
jpegtoavi. A basic usage template from the man page is:

jpegtoavi -f fps width height img1 [img2 ... imgN]

Although jpegtoavi is nice for simple tasks, minimal documentation exists. I
was surprised to find that none of the main Web sites hosting downloads
of the software provided any type of wiki or instructions beyond what
already was in the man page and README.
You can do more-advanced encoding with FFmpeg and MEncoder, both of which
are heavily documented on-line and in their man pages. These
programs both rely on libavcodec and have many overlapping uses, but the
command formats are different. For this article, I cover only FFmpeg.
This will convert a folder of GIF files sequenced as
"image-001",
"image-002" and so forth into an MP4 movie file with a framerate of 10
frames per second and a reasonably high bitrate of 1800:

ffmpeg -r 10 -b 1800 -i image-%03d.gif movie.mp4

Make sure your files are named properly, because encoding will stop
early if the program encounters a gap in the number sequence.

Animated GIFs

If you're a citizen of the Internet, you've no doubt noticed the recent
proliferation of animated GIFs on sites like Tumblr.com. Now that
more people have access to high-bandwidth network connections, the GIF
art form is not so limited in resolution and number of frames as it was
in the 1990s when tiny GIF animations originally rose to popularity in
Geocities and Angelfire home pages. Modern GIF animations often display
entire scenes from movies.
So, are you ready to pimp out some mad GIF
skills?
With ImageMagick, it's easy to fashion an animated GIF from a sequence
of non-GIF images:

cd /path/to/image/folder ; convert *.jpg animation.gif

The mother of all command-line GIF manipulation programs though is
Gifsicle. Your images must already be in the GIF format to use it.
To create a GIF that animates just once, do:

gifsicle image1.gif image2.gif image3.gif > animation.gif

In most cases, you'll want your animated GIF to loop endlessly. You
also may want to specify parameters, such as framerate. Try this for a
dithered animation that loops at 10 frames per second:

You also can use Gifsicle in reverse mode—that is, to extract
the individual frames from an animated GIF. Just use the
--explode argument:

gifsicle --explode animation.gif

Now, go out (and by "out", I mean to your nearest terminal) and explore
all the neat tricks you can do with Gifsicle!
Here's one more to wet your feet:
Take a ready-made animated GIF with a white background and make it
transparent:

Abstraction

For most of us, the notion of animation brings to mind deliberate,
structured sequences. In this section, I introduce
some less-traditional ways of creating mind-blowing, computer-generated
effects without having to know languages like Processing or Pure Data
(both of these are very powerful, but not everyone who wants to animate
knows how to code).
In my own work with video, screen recording tools have been
indispensable. Sometimes I use them to capture animations I make in
Pencil, because the movie export feature is broken in the version I
use. Other times, I just want to capture some cool imagery on my screen
without worrying about proprietary copyrights, so I take screen recordings
of free software.
My preferred screen recorder is the bare-bones, command-line version of
recordMyDesktop. Your distribution's repositories also might provide
the graphical front ends GTK-recordmydesktop and QT-recordmydesktop,
but I find those to be buggy and prone to crashes when recording long
scenes. You can record your entire screen with:

recordmydesktop screenrecording.ogv

The recording will start as soon as you enter the command, and it
will stop when you press Ctrl-c. Read the man page for more
options, such as recording a specific window (tip: find a window's ID with
xwininfo).
Electric Sheep
If you aren't familiar with the trippiest screensaver in the world,
go on-line and look up some images of Electric Sheep. The software
artist Scott Draves created Electric Sheep as a dynamic, collaborative
fractal flame animation that runs on and by thousands of computers
worldwide. Networked data determines the mutations of the various
"sheep" in the animation, and users can vote on and contribute their own
sheep. And because it's all free, anyone can use the images generated in
this android dream.
So how do you take a screen recording of a screensaver? Well, guess
what: Electric Sheep is a binary. Just enter
electricsheep
into your terminal and watch the magic in MPlayer.
If you want to create your own sheep, check out the program
Qosmic.
Figure 5. Using recordMyDesktop to Capture Electric Sheep
XaoS
XaoS is a real-time, interactive fractal zoomer that will
capture the eye of mathematicians and VJs alike. You can change
the fractal formulae and colors with many different parameters and
filters. My favorite is the Pseudo-3D filter, which extrudes lines to
generate what looks like a surreal landscape. Using the left and right
mouse buttons, you can zoom in and out as if flying a plane over the
"terrain".
Figure 6. XaoS with the Pseudo-3D Filter Applied
Fyre
Fyre is a program that generates and animates Peter de Jong
maps. You don't need a screen recorder to make animations with this; you
can enter key frames and render an AVI file directly from the program. As
you can see from the screenshot shown in Figure 7, Peter de Jong maps make for some neat,
abstract images.
Figure 7. Animating in Fyre

Alphas and More to Look Out For

Unfortunately, there is not enough space in this article or in my brain to
cover all the new video-related Linux software that's in development. In
lieu of a complete list, I'll provide you with the names of a few projects
that I expect to be worth checking out for both developers and end users.
Auteur
Auteur is one cool new kid on the block. I first heard of this project
in an episode of the podcast "The Bad Apples" (which has since been
re-branded as "GNU World Order"), produced by Seth Kenlon, aka Klaatu,
who is also a developer on the Auteur team. Klaatu noted the absence of
a truly solid nonlinear video editor for Linux, so he set out to make one
with all the features he felt existing software was lacking. The Web site
currently says that the project is frozen due to lack of programmers—so
programmers, why not help out with a promising alpha?
Figure 8. Testing Out Auteur
VLMC
The folks behind the VLC media player have a nascent project called VLMC
(VideoLAN Movie Creator). The latest releases are still basic and not
quite stable, but knowing what the VideoLAN team is capable of, I am
sure this will mature into a serious nonlinear video editor. They
currently are looking for contributors.
Pencil
Pencil is a traditional 2-D animation program,
which, although still in beta, already fills a gaping hole in the sphere of
Linux animation tools. It allows drawing in both vector and bitmap formats
as well as importing images and sounds. My trials with Pencil have been
basic but mostly satisfactory, although the video export feature appears
broken in Linux. I have compensated for that and made some cool videos
anyway simply by taking a screen recording during animation playback in
Pencil. There is an active community of Pencil users who post animations
on the Pencil Web site's gallery. Pencil is similar to Synfig Studio,
but I find the interface easier to navigate.
Figure 9. An Animation I Made in Pencil
Puredyne
Puredyne is a multimedia Linux distribution based on Ubuntu and Debian
Live, specifically designed for real-time audio-visual processing. Many
of the tools and APIs I haven't had the verbal real estate to cover in
this article (such as FreeJ, Gephex and DataDada) are included either in the
distribution itself or in optional modules.
And, there you have it, animators and filmmakers. I hope this article inspires a
cool music video or two!