The Ins and Outs of Video
Compression

1) What is video compression?

Way back in the day when digital video was first being born,
video was stored in its pure, uncompressed format, much like digital
audio (for those who don't know, CDs were pure uncompressed digital
audio data, while newer standards such as Dolby Digital, MiniDisc, and
MP3 are compressed). This meant that it took a LOT of space. For
comparison on how much space video takes over audio, no matter how its
stored, take for example the FM radio spectrum. TV Stations, like Radio
stations, have a fixed frequency range which they operate on, however
unlike FM we are never told the exact frequency at which they operate,
our TVs are just hardwired with what frequency values correspond to
channel 2, 3, 4 and so on. The ENTIRE FM spectrum exists in a
tiny little gap between TV channels 4 and 5, which themselves (this is
for a single channel) take something around 5 times more bandwidth than
the whole FM band. Crazy, huh?

So one can imagine that back when digital video first came
out, it was damn hard to store. Nowadays we have 100GB RAID arrays
being common in not-so-expensive consumer setups. 10 years ago, 10GB
was an ENORMOUS amount of space!! Hard drives didn't hold more than
500MB tops, but uncompressed digital video still took up the same space
it does now. This lead to people storing digital video to tape and
doing things with it that way, but this made Non-Linear video-editing
impossible.

Then along came video compression. Probably the earliest, and
truly successful video codecs (CODEC stands for
COmpressor/DECompressor) was MPEG1. I'm sure you've all heard
of MPEG (The Motion Picture Experts Group) in terms of its compression.
But did you know that the actual standard for MPEG1 video is over a
decade
old? Not only that, but the MPEG2 standard used in DVDs is more than
half a decade old!

But how does compression work? Well like with any kind of
compression there are two kinds: lossless and lossy.

2) Lossless Compression

Lossless compression, as the name implies, means that after
compressing the video, and then decompressing it, you wind up with the
exact same data as you put in. This is comparable to something like ZIP
or RAR (In fact, the most common lossless codec out there, Huffyuv,
basically uses the same compression algorithm as ZIP on each frame of
video to achieve its compression). Lossless has the advantage that no
matter how many times you compress it, you still haven't lost any video
data. The bad part is that most often you don't save nearly as much
space as you would with other lossy compression algorithms. There are
many different lossless codecs these days, each are described later in
the guide to making clips.

3) Lossy Compression

This is the form of video compression most people are familiar
with. 95% of all video codecs out there are lossy, meaning that when
you compress the video and then decompress it, you do not get back what
you put it. Now, this isn't as bad as it may sound. Obviously if you're
compressing something like a text document, you don't want to lose any
of the data, but with something like a picture, even if a few bits and
pieces aren't quite right, you can still make out the generally gist of
the image. Same thing with audio. Famous lossy codecs include MPEG1,
MPEG2, MPEG4 (AKA DivX3.11, DivX4/5, XviD, Quicktime MPEG4), DV (and
its variants DVCAM, DVCPRO, DVCPRO50, Digital-S, etc), Wiindows Media
Video, RealVideo,
Sorenson, Indeo, and the classic Cinepak.

Now with that distinction out of the way, we can discuss how
to compress the actual video itself, of which there are also 2 basic
ways of doing this. These two methods are called intra-frame
and inter-frame compression.

4) Intra-Frame Compression

As the name suggests, Intra-frame compression relies on only
the single, specific frame it is working with to compress the data
within it. This means that you are basically encoding each separate
frame as its own picture. In the case of something like MJPEG (an
algorithm which uses only intra-frame compression) you are encoding
each and every frame using a JPEG compressor (which should be familiar
to anyone who has ever worked with images, it has the extension jpg).
This means that while you can't take advantage of the information in
previous and forthcoming frames, you have the ability to recreate each
frame without the need for the others. This comes in very handy when
editing video (which is why editing with a pure intra-frame codec is a
must unless you use special hardware).

5) Inter-Frame Compression

As should be obvious by now, Inter-frame compression relies on
information in preceding and occasionally forthcoming frames to
compress an image. The most well-known way of exploiting this data is
by exploiting the fact that the majority of a video image isn't always
moving. Take a newscast, for instance. Usually the only thing moving is
the anchor's body, while the rest of the set is staying perfectly
still. Why should we bother to store all of the data that makes up that
background for every single frame? Here's an illustration:

We start with this frame. Now we take a look at what is
actually changing in between frames:

Here are frames 2 and 3, with the only parts that have changed
since the previous frame shown below the actual frame.

Here we have frames 4 and 5 with the same arrangement.

Notice that the only things in the frame that changed was the
chair moving (and redrawing the background in the area that the chair
had left since the previous frame). Almost all inter-frame compression
is based on exploiting this fact about a video image. The disadvantage
to this is that if you want to check out frame 5, you can't actually
see what the real frame looks like without first looking back at frame
1, then applying the changes in frames 2, 3, 4, and 5. This leads to
the notion of keyframes. A keyframe is a special frame which
exists on its own - in other words it doesn't rely on any other frames
to store its data. Thus seeking to a particular frame usually involves
first going to the preceding keyframe and then applying each successive
partial-frame until we reach the desired one.

This presents a problem for video editing - if we want to cut
the video on a specific frame, we are unable to because most programs
(Premiere, Media Studio Pro, VirtualDub, most any video software) will
only cut at a keyframe. This makes your editing options very limited
unless you create lots and lots of keyframes, in which case you then
lose the benefits of inter-frame compression in the first place!

Bottom line: Use Intra-frame compression-only algorithms for
editing, and inter-frame compression algorithms for final distribution
and archival.

One final thing - depending on the codec, filesize can be
determined one of two ways (see a trend forming here?). Either the
filesize is determined by bits per second or by bits per
frame. All inter-frame compression algorithms are based on a
bits/second factor (although this can be variable throughout the file
depending on if you use what's called variable bitrate), and
many intra-frame compression algorithms (with DV and MJPEG being
notable exceptions) are based on bits/frame.

What does this mean? Well, it means that when you encode
something in MPEG and specify that it will be in 1.5 megabit per
second, it doesn't matter whether the video is in 29.97 fps, 25 fps,
23.976 fps, 15 fps, or whatever - it's still going to take up the same
amount of space. The 15 fps one will look sharper, but obviously will
be less smooth than the other ones. If you try the same thing with a
codec like Huffyuv or Intel Indeo, you will see that this does not hold
true. That's because these codecs encode each frame and store it
separately, regardless of how big or small the frames before or after
it are. Thus, you will get the same image quality through all of them,
but the smoothness and filesizes will change.