The power of GDAL virtual formats

08 March 2012

GDAL is most likely the most powerfull GIS toolset out there, with very geekish features that may seem complicated at first, but become extremely powerful and simple at the same time once you master them.
One of these features I most like is the virtual format concept, valid both for raster data sources (GDAL) and for vectorial data sources (OGR).
Basically GDAL gives to the user a very simple mechanism to create virtual formats from some different sources.
Let’s analyze this feature, with a couple of samples derived from real world scenarios I have been envolved in the last weeks.

Raster GDAL Virtual Formats

GDAL gives the possibility to create GDAL virtual formats: it is possible to create a GDAL dataset “composed from other GDAL datasets with repositioning, and algorithms potentially applied as well as various kinds of metadata altered or added. VRT descriptions of datasets can be saved in an XML format normally given the extension .vrt.”
In the virtual GDAL dataset it is possible, by simply using some xml, to manage, alter, and add important elements of a raster, like the spatial reference, the six values for the affine geotransformation, the metadata, the mask band.
For each band of the resulting raster, it is possible to manage important parameters like the NoDataValue, the ColorTable the vertical units, just to name a few ones.

In a recent project I had a netCDF dataset, composed of 104 subdatasets (weather forecast variables), each of them composed of 10 bands (each band with the forecast for one of the next 10 following days).
I wanted to have a single virtual raster managing the values of 3 of the 107 variables for each forecast day, for using it within MapServer (and at some extent in QGIS).

This is what I have been using in xml for creating the virtual raster composed of 3 bands with the values of the 3 variables in each band for the 7th day of the forecast:

At this point this is effectively a raster for GDAL and all the software and tools based on it.
For example it will be easily possible to query it with gdalinfo or manipulate it with some other GDAL command line utilities, to display it with MapServer or desktop software like QGIS or even to access it programmatically with the GDAL API.

The GDAL command line utility gdalinfo will produce the following output:

QGIS will correctly display it using the GDAL Virtual Raster driver, and finally it will be accessible programmatically using the GDAL API: for example this is how in Python is is possible to get the metadata information of the first band:

Note that he gdalbuildvrt GDAL command will build a vrt from a list of datasets, and it is very handy if you want to generate the vrt file without too much effort.

Vectorial OGR Virtual Formats

In OGR, the vectorial “part” of GDAL, there is also the possibility to compose virtual formats.
This format is an excellent possibility if you need to derive spatial layers from formats that are not directly spatially enabled like flat files and flat database tables with spatial information not stored in common formats like wkt or wkb.
You can create a virtual format mapping the original data source with an xml file: after creating this file it will be possible to read (and if some basic condition is met even to write!) the original datasource without using interchange formats.

A couple of situation I have met in my latest projects: in the first one I had to read a flat table in oracle storing point geometries, with the geographic information stored as two double fields (x, y), and at the same time filtering the results on a CATEGORY field value.
This is how I ended up creating the virtual driver to the table (using the GeometryField xml subelement for defining where the geometric information is stored):

As for virtual format for raster, also for OGR virtual layers there is the possibility to use the GDAL API.
For example here I access to the virtual raster and check how many features are stored in it: