After creating a web service client, you can already use functions that Data Avenue provides by invoking operations such as copy, list, mkdir, delete, etc. In order to use the Data Avenue API first you need to request for a "ticket" (also known as API key) by filling out the ticket request form.

The code of a directory listing below illustrates how simple is to use Data Avenue web services:

Creating a Client

Typically, the web service client is simply created by invoking DataBridgeClient.getClient(wsdlUrl), where wsdlUrl is the URL of Data Avenue web services' WSDL file.

NOTE: Data Avenue server uses self-signed certificate. The lines below are no longer needed, but ensure that you use "unsecure client" (DataBridgeClient.getUnsecureClient(wsdlUrl)) and the latest client lib of version 2.0.2.

NOTE: If using Java6, in spite of the above certificate import, host key verification must be turned off, which is done if creating client via DataBridgeClient.getUsecureClient(wsdlUrl). It is due to the missing SNI functionality in Java6, but the communication will still be secured.

Uniform Resource Identifiers (URIs)

All entities that Data Avenue can handle are identified by uniform resource identifiers (URIs) of the form: protocol://host[:port]/path, where path contains (ends with) either a file or a directory name. In the case of directories, paths should end with a slash symbol: '/'. If omitted, port is the protocol default (e.g., defaults to 80 in the case of HTTP). Available protocols can be obtained dynamically by invoking client's getSuppotedProtocols method (see later), which are: http, https, sftp, gsiftp, srm and s3, at the time of writing this document.

Credentials

Accessing different storage resources may require authentication data, called credentials, which must be provided for Data Avenue, which will behaves as a mediator. Depending on the selected protocol, you can choose among different authentication types, which require different authentication data, called credential attributes to be specified. For example, a site may require HTTP Basic Authentication, so you choose "UserPass" authentication type, and set "UserID" and "UserPass" attributes. The list of protocols, authentication types and credential attributes are available here. All credential attributes must be specified as strings, so credential data available in files locally (e.g., x509up proxy file) must be read up and set as string in credentials.

We use "Globus" authentication to create a new directory in a storage element accessible via gsiftp in the example below:

Sessions

At accessing a specific storage, tickets and credentials data need to be passedat the very first call only, which are then maintained in your web service "session" by Data Avenue. This way you may omit these parameters in subsequent calls (null values can be passed) as long as your session is alive. After a specific time (5 minutes) of inactivity, your session will expire (you get SessionExpiredException), when you have to set ticket and credentials data again.

NOTE: There are two exceptions when credentials have to be passed: copy/move operation and alias creation. Copy tasks create new, individual session as well as aliases (download/upload), since their lifetime may exceed user session (may take several hours to complete).

Operations

There are a number of functions, operations provided by Data Avenue, all available as client methods for API users. These functions consist of four groups: meta-information retrieval, synchronous- and asynchronous operations (commands), and "alias-related" functions.

Returns the list of directory contents (file and subdirectory entries). Details about files and subdirectories (such as type, last modification date, size) can be queried using the appropriate DirEntry methods.

Asynchronous commands

These are called asynchronous operations, as these tasks, data transfers may take a longer time to complete (possibly hours or days), so these methods calls returns transfer identifiers ("handles") associated with background tasks initiated by the call. These ids can be used to poll transfer state or cancel the task later.

Copies the specified source file to the destination location. The method returns the identifier of the transfer, which can be used in getState or cancel operations (see below). overwrite flag indicates to replace the previous file with the same name at the destination location. destUri may be a directory; in this case, the name of the source file will be used as file name on the destination location. Note source- (srcCreds) and destination credentials (destCreds) are specified to authenticate on different source and destination resources, respectively.

Moves the specified source file to the destination location (source will be deleted after copy). The method returns the identifier of the transfer (task).

The following methods are not "asynchronous", but they are related to managing asynchronous tasks.

TransferDetails getState(String ticket, String id);

Returns state details about the transfer having the specified identifier. TransferDetails record returned contains status (CREATED, SCHEDULED, TRANSFERRING, DONE, FAILED, CANCELED), time (created, started, ended), size and progress information (size, bytes transferred), failure message (if the task has failed for some reason) information.

void cancel (String ticket, String id);

Stops an active transfer, and set its state to CANCELED.

Aliases

The aliasing-technique implemented by Data Avenue provides HTTP tunneling possibility to access (read or write) files on different storage elements. When creating an "alias", you specify the file on a storage –accessible with a specific protocol – along with the appropriate credentials, and you will get an ID back, from which an HTTP URL can be constructed, available on the blacktop: https://data-avenue.eu/blacktop2/<ID>. Reading this URL (by HTTP GET), or writing this URL (by HTTP PUT or POST), respectively, will read or write the resource, mediated by Data Avenue – without the need of the client (wget, curl, HTTP form) to be aware of the specific protocol.

NOTE: you may use aliases for a longer period of time (days, weeks). Therefore it is very important to provide credentials that are still valid at the time of their latest potential use (creation time + lifetime)!

Creates an HTTP alias for the specified resource. read flag specifies whether this file will be read (HTTP GET) or written (HTTP PUT/POST). The method returns an ID; appended to blacktop URL gives the full HTTP URL, i.e. the alias of the resource. The alias will be accessible for lifetime seconds. (archive option is reserved for future development, use false value.)

The example below creates an HTTP alias for a file "myfile" accessible on an SFTP server and then downloads its content into a local file "downloaded":