Overview

caGrid is released as a source distribution, and, as a graphical installer. Those leveraging caGrid are strongly encouraged to still use a caGrid release, however, when developing applications that use caGrid libraries, it can be difficult to know which caGrid libraries (or external libraries caGrid needs) you need for what you want to do with caGrid. As of caGrid 1.2, this information is formally managed and documented using the popular dependency management framework call Ivy. Ivy is a sub-project of Ant, and provides a number of custom Ant tasks focused solely on creating an enterprise dependency manager. caGrid transparently makes use of Ivy for its build process to manage its internal and external dependencies. Those who do not want to use Ivy are unaffected by this fact, but for those who do, it provides great advantages in determining exactly which libraries are needed for which tasks.

Ivy Introduction

The Ivy Documentationitself is quite good, and those new to Ivy are encouraged to review it. However, the main idea is that dependencies are described by each project in an Ivy file. An Ant-based build process (such as caGrid's process) is developed to use Ivy, and configures Ivy's settings, which control things like how Ivy should find dependencies. Then, when a given project is built by the Ant build process, first its dependencies are resolved and retrievedfrom a repository into the project (such as into its lib directory). The project can then be built as it normally would be, and optionally could publish itself to such a repository. A graphical representation of this process is shown below.

The use of Ivy does not necessarily need to affect your project's normal build process. In fact, only the main caGrid build process uses Ivy, and each caGrid project build process is completely independent of Ivy.

Depending on caGrid Using Ivy

There are several options for leveraging the power of Ivy when developing a project that depends on caGrid. This includes different ways of integrating Ivy directly into your project's build framework, just using Ivy to bootstrap your build by using it to initially copy what you need, or even just looking at the detailed reports Ivy generates and copying libraries "the old fashioned way". In the following sections, some of these options are discussed in the framework of a running example. One item of note is that, at least currently, it is expected that you have the libraries from a Globus installation on your classpath; these are not modeled in the Ivy dependencies (though this may change in the future).

A Simple Example Project that Needs caGrid

For the purposes of demonstrating Ivy, a sample project which uses the Discovery APIs of caGrid is explained below. The full source code for the example can be downloaded here or can be checked out from the Demos module of caGrid's source control system.

The sample project contains a single source file, contained in the src folder, and its Ant build process (defined in build.xml) expects to find the libraries it needs in the lib directory, and in the lib directory of the Globus installation (it expects the GLOBUS_LOCATION environment variable to be set). The build process compiles into the build directory and contains a target to run the main method of the source code (which makes various discovery queries). Thus far, this is a fairly typical project and makes no reference to Ivy or a caGrid installation.

If we run this project as is, it will attempt to compile the project and generate a number of compiler errors. If we were to place all of the caGrid dependencies needed to compile this project's source code, it would run correctly. This brings us to the topic at hand; how to depend on caGrid.

Integrating Ivy into the Project

As mentioned above, there are numerous ways Ivy can be used to solve our problem of depending on caGrid. If we actually want to use Ivy directly in our project, we need to describe to Ivy what our dependencies are. The approach taken in this example is to isolate the Ivy-related portions of the build (such as retrieving dependencies), from the "standard" build targets and build process (those defined above). In this way, we will actually create additional Ant build files which import our standard build, and add the Ivy support. We will use these additional targets when we need to leverage Ivy, but can use the standard build file for the normal compile and run process. The advantage to this approach is that we can actually just exclude the Ivy-related files if we were to create a distribution of our project which has all the jars it needs in its lib directory. However, if we wanted to actually integrate Ivy into our build process, we could have just as easily done that as well.

The first step to integrating Ivy into our project is to actually define the project in terms of what it is, how you can depend on it, and what dependencies it has. This is done by creating an Ivy file (ivy.xml)for the project. A detailed explanation of these files is out of scope for this discussion, but suffice it to say the one used for this project is quite simple. It just indicates that version 1.2 of the discovery module of caGrid is needed. For more information about Ivy files, see the Ivy documentation. Additionally, the caGrid release itself contains hundreds of Ivy files which can be used as examples. You can also look at these online in thecaGrid remote Ivy repository.

Now that we have defined the project and its needs in a way Ivy understands, we need to tell Ivy what we want it to do for us; this is where we have choices. In the cagrid:Ivy Introduce section we saw that Ivy retrieves project dependencies from a repository. We have to tell Ivy what repository it should used to locate the caGrid dependencies our project needs. The sample project actually demonstrates two different ways to do this (explained in the next sections).

Using Ivy to Extract Libraries from a Local caGrid Installation

In the first example, we will configure Ivy to copy caGrid dependencies from a local build of the caGrid source distribution. We will refer to this installation's location with the environment variable CAGRID_LOCATION. As mentioned before, the approach we will be taking for integrating Ivy is to create another Ant build file, which extends our project's regular build process. In this second build file, the most important target we will create is the ivy-resolve target, which will instruct Ivy to copy the dependencies we need into our lib directory. We will also define the supporting targets of ivy-clean which will remove these copied dependencies, and ivy-report which will tell Ivy to create us a report of what it has done for us. We will create an all target, which simply calls all three of these targets, and make it the default for the project.

Aside from the aforementioned Ant targets, there are some other aspects of the build file which need explanation. The first is that we define the new Ant tasks Ivy provides with an Ant taskdef and define a namespace (ivy) for these new tasks. This process is explained in the Ivy Ant documentation. Also of importance is how we tell Ivy about our repository. This is done by configuring Ivy with the ivy:settings task. The use of this task is defined in [the relevant section of the Ivy documentation, but of notice is that we are just pointing to the settings file which caGrid uses in its build process. This settings file actually needs two variables to be set, which we can do with simple Ant properties. The first is the repository.dir property, which is where the external libraries caGrid needs are stored, and the second is the integration.repository.dir which is where caGrid projects publish themselves when built so other caGrid projects can depend on them.

Once Ivy is configured, you will note that the actual Ant targets are quite simple. To get the jars we need, we just need to call ivy:retrieve. It is worth noting that we are taking advantage of some of the default settings of Ivy, but these can always be overridden. For example, Ivy by default will copy jars into the lib directory, but that can be changed].

To actually run this process and copy the dependencies we need into our lib directory, we just need to type:ant -f local-ivy-build.xml
An example output of this can be see below:

Using Ivy to Download Libraries from the Remote caGrid Repository

In the second example, we will configure Ivy to copy caGrid dependencies from a remote Ivy repository accessible over the Internet. We will no longer require a local caGrid installation or environment variable as was necessary for the previous example. As mentioned before, the approach we will be taking for integrating Ivy is to create another Ant build file, which extends our project's regular build process. We will be creating an alternative (remote-ivy-build.xml) to the local-ivy-build.xml used in the previous example. We will use all the same Ant targets names and processes however. Actually, the only difference in the local and remote examples, is the way in which we tell Ivy about the settings it should use. Whereas, in the last example we pointed to settings in the caGrid release, in this example we will point to settings provided by the remote repository itself; all we need to know is the URL of these settings.

Note this time it was significantly faster, and that it told us it didn't need to retrieve anything new. You can experiment with this, by removing a few of the jars from the lib directory, and doing another ivy-resolve and you'll see it only downloads what it needs.

You can now run the main build file just as before, and you should see the same results.

Looking at Ivy reports as a guide to manually copying libraries from a local caGrid installation

The preceding two examples showed how Ivy could be used programmatically in your build process to copy dependencies into your project. If for some reason you do not have the ability to integrate with Ivy in this way, you can use the output of the Ivy reports Ivy can generate (as shown in the preceding examples), to look at any given project's dependencies, and manually copy the dependencies you need (though it is recommended you allow Ivy to do this for you, even if you don't do it as part of your build process).

Shown below is a portion of the report generated by running the ivy-report target of the ivy builds files:

You can see in the Dependencies Overview section, a tree-like representation of the transitive dependencies are shown. In our example, the example project only depends on caGrid Discovery, which in turn depends on the metadatautils, metadata, and core caGrid projects. These projects in turn have dependencies which are also retrieved. You can use this information to understand which libraries you may need for a given use of any caGrid project, and why you need what you need. The full report contains the specific details of each module's artifacts. You can find Ivy reports for every caGrid project in each project's ext directory.

Ivy Summary

This section has shown a variety of ways in which the power of an Ivy-enabled caGrid build can be leveraged. However, it has only scratched the surface of what Ivy can do, and those who find utility in the mechanisms described here are encouraged to look further into Ivy. Those who are doing extensive development work depending on caGrid will likely find it a very convenient way to stay integrated with both released versions of caGrid, as well as supporting nightly or continuous integration versions; switching between the two can be as simple as pointing to a different Ivy settings file. It also ensures your project will be more "future proof" in that when new caGrid releases are made available, you can automatically update your projects to leverage them.