ITK/Git/Develop/Data

This section is currently out-of-date and may contain errors but is retained for historical reference. For peer-reviewed, authoritative guidance on ITK, see The ITK Software Guide. Additional guidance can be found via the mailing lists.

This page documents how to add test data while developing ITK.
See our table of contents for more information.

This means that the git-gerrit-push script pushed the topic and uploaded the data it references.

Options for gerrit-push:

--dry-run: Report push that would occur without actually doing it

--no-topic: Push the data referenced by the topic but not the topic itself

Building

Download

For the test data to be downloaded and made available to the tests in your build tree the ITKData target must be built.
One may build the target directly, e.g. make ITKData, to obtain the data without a complete build.
The output will be something like

Local Store

It is possible to configure one or more local ExternalData object stores shared among multiple builds.
Configure for each build the advanced cache entry ExternalData_OBJECT_STORES to a directory on your local disk outside all build trees, e.g. "/home/user/.ExternalData":

$ cmake -DExternalData_OBJECT_STORES=/home/user/.ExternalData ../ITK

The ExternalData module will store downloaded objects in the local store instead of the build tree.
Once an object has been downloaded by one build it will persist in the local store for re-use by other builds without downloading again.

Discussion

An ITK test data file is not stored in the main source tree under version control.
Instead the source tree contains a "content link" that refers to a data object by a hash of its content.
At build time the the
ExternalData.cmake
module fetches data needed by enabled tests.
This allows arbitrarily large data to be added and removed without bloating the version control history.

The above workflow allows developers to add a new data file almost as if committing it to the source tree.
The following subsections discuss details of the workflow implementation.

ExternalData

While CMake runs the
ExternalData
module evaluates DATA{} references.
ITK sets
the ExternalData_LINK_CONTENT option to MD5 to enable automatic conversion of raw data files into content links.
When the module detects a real data file in the source tree it performs the following transformation as specified in the module documentation:

Recover Data File

pre-commit

While committing a new or modified content link the
pre-commit
hook moves the real data object from the .ExternalData_MD5_${hash} file left by the ExternalData module
to a local object repository stored in a .ExternalData directory at the top of the source tree.

The hook also uses Git plumbing commands to store the data object as a blob in the local Git repository.
The blob is not referenced by the new commit but instead by refs/data/MD5/${hash}.
This keeps the blob alive in the local repository but does not add it to the project history.
For example:

git gerrit-push

The "git gerrit-push" command is actually an
alias
for the
Utilities/Git/git-gerrit-push
script.
In addition to pushing the topic branch to Gerrit the script also detects content links added or modified by the commits in the topic.
It reads the data object hashes from the content links and looks for matching refs/data/ entries in the local Git repository.

The script pushes the matching data objects to Gerrit inside a temporary commit object disjoint from the rest of history.
For example: