Too Busy For Words - the PaulWay Blog

Wed 15th Jan, 2014

On the way home from LCA, and on a whim, in Perth I started adding
support for LZO compression to Cfile.

This turned out to have unexpected complications: while liblzo supports
the wide variety of compression methods all grouped together as "LZO",
it does not actually created '.lzo' files. This is because '.lzo' files
also have a special header, added checksums, and file contents lists a
bit like a tar file. All of this is added within the 'lzop' program - there
is no external library for reading or writing lzo files in the same way
that zlib handles gz files.

Now, I see three options here:

Abandon it. No-one uses .lzo files anyway.

Copy and paste code from lzop into cfile, bloating it out considerably
but supporting .lzo files.

Create a separate 'liblzop' project and produce a library that supports
reading and writing of .lzo files compatible with lzop. Something that the
inventor of the LZO algorithms hasn't done at all.

Yeah, I'm going for option one there.

LZO is a special case: it does a reasonable job of compression - not
quite as much as standard gzip - but its memory requirements for
compression can be miniscule and its decompression speed is very fast.
It might work well for compression inside the file system, and is
commonly used in consoles and embedded computers when reading compressed
data. But for most common situations, even on mobile phones, I imagine
gzip is still reasonably quick and produces smaller compressed output.

Now to put all the LZO work in a separate git branch and leave it as a
warning to others.