REST interface can be used with command line tools as well such as curl. For example:

curl -k https://data-avenue.eu/blacktop-dev/rest/version

returns Data Avenue version (-k is to ignore certicate verification).

Data Avenue Authentication (Key)

Each REST call must contain a special header field, named "X-Key", containing the access key (a.k.a. "ticket") required to use Data Avenue services. It can be requested on page: ticket request form.

You can use curl's -H (--header) option set the key for the request:

curl -H "X-Key: de305d54-75b4-431b-adb2-eb6b9e546014" ...

To avoid saving your key into command history, you can create a file key with a single line "X-Key: de305d54-75b4-431b-adb2-eb6b9e546014", and use cat in curl:

curl -H "$(cat key)" ...

X-Key header will be denoted as <key> for short in the following.

(Storage) Credentials

In order to allow Data Avenue to access the storage of your choice, you have to temporarily delegate your credentials for successful mediation. Such credentials are passed via HTTP header elements. The most common authentication type is the username-password authentication (access key/secret key pair, in the case of S3), but you can also authenticate using x509 proxy (e.g., GridFTP). In the latter case, proxy need to be base64 encoded in the header. These credentials have to put into headers named "X-Username", "X-Password", "X-Proxy" as appropriate.

To pass username and password with curl, use -H option:

curl -H "X-Username: username" -H "X-Password: password" ...

To add a proxy, stored in file x509up, to HTTP header you can use the base64 encoder inline:

curl -H "X-Proxy: $(base64 --wrap=0 x509up)" ...

Note that credentials passed over non-secured HTTP can be eavesdropped. Use secured Data Avenue endpoints (starting with URL https://), and avoid using curl's -k (--insecure) option, unless you are absolutely sure in that the network link between you and the Data Avenue server is protected.

Storage credential headers will be denoted as <credentials> for short in the following.

Data Avenue REST operations (file, directory, attributes, transfers) require passing the URI of the remote file on which some operation needs to be performed. The URI is passed in "X-URI" HTTP header field.

With curl you can do this like:

curl -H "X-URI: s3://aws.amazon.com/mybucket/file2download"

/directory (resource)

GET

Returns the list of directory enty names (files and subdirectories) of the spefified remote directory (X-URI header) in the form of JSON array (application/json response).

If you also add header "X-Accept-Redirects: yes" and the storage resource supports direct URLs (pre-signed URLs, in the case of Amazon S3), file up- and downloads are redirected to the storage resource bypassing Data Avenue, so data transfers can be much more efficient. Use -L (--location) option of curl to allow redirection.

Using redirects in the case of S3 requires using the PUT HTTP method. Some S3 storages also require setting empty Content-Type header or using "binary/octet-stream". It is thus recommended to use curl command's --upload-file option (--data-binary sets "application/x-www-form-urlencoded") and to omit the Content-Type header setting.

/attributes (resource)

GET

Returns the attributes of the specified remote file or directory (X-URI header). The result is a JSON object with keys: name, size (in bytes, in the case of file), date of last modification (epoch in millis), permissions (in linux-like format, e.g., xrw-rw-rw, in owner, group, all users permissions order), and comment (optional).

Returns the set of attributes of the set of specified remote files or directories in the specified remote directory (X-URI header). The set of entries of interest should be given in entiry body as a JSON array. The result is a JSON array of JSON objects with keys descibed in GET. This operation is used typically together with directory listing (GET /directory) to get entry details.