Introduction

This article presents two zipped STL iostream implementation based on the library zlib (see download link above) and bzip2 (see download link above). This means that you can easily manipulate zipped streams like any other STL ostream/istream.

To give you an idea, consider following snippet that prints "Hello World":

As you can see adding zipped buffers into your existing applications is quite straightforward. To summarize, let's see some quick facts about zipstream and bzip2stream:

STL compliant,

any-stream-to-any-stream support,

char, wchar_t support,

fining tuning of compression properties,

support custom allocators (New!)

Why another wrapper? Why not use gzstream?

Writing wrappers around the zlib library is popular on CodeProject. If you search for 'zip' you will find at least 14 articles on the topic. Moreover, if you crawl on the web and especially on zlib home page, you can find dozens of other wrappers.

So why another wrapper? Well, none of the wrappers are fully STL compliant. Ok, this is not true since gzstream (see download link above) implements fstream-like STL streams. However, gzstream has three drawbacks:

It does not allow buffer to buffer compression since it is based on gzip i/o methods: only file to buffer or buffer to file are supported,

it is licensed under LGPL which makes it difficult to use in commercial apps,

it does not support wchar_t

Last reason to write this wrapper: it is a good exercise to understand and implement iostreams.

Wrapper architecture

The three drawbacks of gzstream pushed me to re-implement an STL wrapper for zlib (and later on, do some cut and paste to get bzip2 working).

This wrapper takes a user defined i/ostream to write or read compressed data. This approach is quite flexible since the user can give any stream (istringstream, ifstream, or custom stream) to store or load the compressed data.

Internally zip_stream acts as a triple buffer: the streambuf object in itself, zlib library and the user-defined stream. For example, during the compression process, the buffers are used:

first buffer: the data to compress is buffered into a streambuf object,

second buffer: when overflow is called, the first buffer data is sent to zlib which also buffers the data internally. If zlib outputs data, it is sent to the user-defined stream,

third buffer: the user-defined stream is buffered.

Some care must be taken when flushing: you must use the method zflush that will first flush the streambuf, then flush the zlib buffer, then flush the user-defined stream. Note that you should avoid flushing as it degrades compression.

Implementing iostreams

Since I'm not an STL expert, I will very briefly discuss this part. There's room for a tutorial on this topic...

To implement custom iostream, you need to take the following steps:

implement a custom my_streambuf, inherited from streambuf. You need to override the virtual methods sync, underflow and overflow. sync and overflow are used in output streams and underflow is used in input streams.

implement a custom my_ostream, inherited from ostream. It will use my_streambuf

implement a custom my_istream, inherited from istream. It will use my_streambuf as stream buffer.

Class quick reference

All the zlib classes are in the zlib_stream namespace and all the bzip2 classes are in the bzip2_stream namespace.

The two main classes of the zlib wrapper are basic_zip_ostream and basic_zip_istream which implement respectively compression and decompression and behave like classic basic_ostream and basic_istream.

Classical typedef are also provided for these classes:

zip_ostream, zip_istream for char streams

wzip_ostream, wzip_istream for wchar_t streams.

The bzip2 classes have similar names, just replace zlib by bzip2: basic_zip_streambuf becomes basic_bzip2_streambuf.

Reference

License

These zlib and bzip2 wrappers are licensed under the zlib/libpng license.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

Share

About the Author

Jonathan de Halleux is Civil Engineer in Applied Mathematics. He finished his PhD in 2004 in the rainy country of Belgium. After 2 years in the Common Language Runtime (i.e. .net), he is now working at Microsoft Research on Pex (http://research.microsoft.com/pex).

Comments and Discussions

As I'm not using the std namespace, in zipstream.ipp, I had to change ios and ios_base to std::ios and std::ios_base when using zip_istream to compile. Suggest either insert a using namespace std somewhere or qualify ios and ios_base fullly.