9. vsl: Binary I/O

Chapter summary:
This section describes how to save and restore objects using a binary stream.
It details how to add the appropriate functions to each class to make use
of this facility.

All objects in VXL should be able to save themselves to a stream (eg a file)
and restore (load) themselves from a stream (file). The main functions provided
for this purpose are vsl_b_write(os,object); and vsl_b_read(is,object&);.

The binary IO for the core libraries (vbl, vil, vgl and vnl) is implemented in `clip-on'
libraries which live in the io subdirectories of each library (thus the
declaration of the function vsl_b_write(vsl_b_ostream&,vnl_vector const&);
lives in the file ‘vnl/io/vnl_io_vector.h’.

However, it is recommended that I/O for other libraries be provided by writing
b_write(os); and b_read(is); functions in each class. See the
`Design Notes' section below.

9.1 Supported Platforms

The binary I/O code is known to work across the following hardware/OS/compiler
combinations, but probably also works on most other platform/compiler combinations:

Intel - Linux - gcc-2.95 and gcc-3.0

Intel - WindowsNT - vc++

SGI - MIPS - MipsPRO CC

Sun - Solaris - gcc-2.95

DEC alpha - OSF - gcc-2.95 and gcc 3.0 (64 bit!)

Thus
binary files produced by any of the above should be readable by any other of the above.
There is of course a minor exception: large numbers (like integers larger than 4294967295)
saved on a 64-bit platform cannot be read on a 32-bit platform.

It is recommended that the default extension name for your binary files is .bvl.
This extension does not appear to be used by any other program. In many cases however, you
will want to pick a new extension to indicate the contents of a file. For example, we store
active shape model objects with ending .asm.

The classes vsl_b_ifstream and vsl_b_ofstream are simple wrappers around
real vcl_ifstream and vcl_ofstream objects. These wrappers ensure that
you open a file with CR/LF conversion turned off, and they should also allow lots of
common misuses to be caught at compile time.

The functions vsl_b_write(os,X) and vsl_b_read(is,X) are defined for all
reasonable cases, including all inbuilt types, most classes in vcl and the classes in
the core vxl libraries.

When you write a new class, you should add the appropriate functions to allow easy
use of binary I/O (see below).

Or for simplicity we provide the utility functions which would allow you to write:

// Make application aware of possible classes that it might see in the file
vsl_add_to_binary_loader(my_derived());
vsl_add_to_binary_loader(my_derived2());
...
my_base *b = 0;
vsl_b_ifstream bfs("data.bvl");
vsl_b_read(bfs,b);
// b now points to the correct class which has been created
// on the heap and filled with the data from bfs
...

Note that the read function will only work if the application has been made
aware of each of the possible derived classes that it might come across in the
file. This is done using calls to vsl_add_to_binary_loader(my_derived())
(see appendix for details).

To reduce the pain of doing this, many libraries have a function that adds all
the relevant derived classes (eg xxxx_add_all_binary_loaders() where
xxxx is the library name).

9.2.3 Which files do I need to include/link?

In general the vsl_b_read and vsl_b_write functions use Koenig Lookup - that is the location
of their declaration depends on their parameters.

The vsl_b_stream objects and vsl_b_write and vsl_b_read functions
for fundamental data types are declared in <vsl/vsl_binary_io.h>. If you want to
load or save a vcl_vector, the appropriate vsl_b_write and vsl_b_read
functions will be in <vsl/vsl_vector_io.h>. Likewise for most of the other vcl classes.
The vsl library contains the implementation of all of this.

When reading/writing by baseclass pointer, you need to include vsl/vsl_binary_loader.h.

If you want to load or save a vgl_point_2d, you will need to include
<vgl/io/vgl_io_point_2d.h> and similarly for all other Level-1 VXL libraries. You
will need to include the vgl_io library.
For Level-2 libraries, the situation varies. If binary io has been defined at all for a
level-2 library, it might be included in the library itself, e.g. the io functions for
vpdfl_gaussian are declared in the same file as the Gaussian, vpdfl/vpdfl_gaussian.h>.
Alternatively, it might be in a clip-on library in the same form as the Level-1 libraries
above.

9.2.4 How to save templated objects

The situation for templated objects is the same as above, except that you need to ensure that
the appropriate (templated) vsl_b_read and vsl_b_write functions are explicitly instantiated.
This instantiation is achieved by placing a file in the relevant "Templates" folder.

An example template file, is shown below. It enables saving of a 2d array of "hjk_model"s
(a completely made up plain class).

9.4 Error Detection

IO is often prone to errors beyond the control of the programmer. In
particular, files can be come corrupted, given to programs that can't read
a new format, read on platforms that do not support large enough numbers.

vsl attempts to detect as many error conditions as possible.
It prints an error message to vcl_cerr and sets the fail bit
on the input stream. Any objects that were being loaded when the error
occurred should be consistent at least as far as being able to delete the
object safely.

During the opening of a binary input stream, vsl also checks
for a schema version number, and magic number that confirm that
the stream was written by vsl.