How to configure and use Node API for parallel transfers

IN THIS ARTICLE:

Overview

The 3.6 release of Aspera Enterprise Server includes a REST API for initiating parallel transfers. This new feature is an extension of the asperanode /ops/transfers API. Parallel transfers can move content between single servers or clusters, and to/from block or object storage.

Details

In this KB article, we describe how to interact with this new API. We are working on building this transfer initiation capability into various applications, but for the current version, it is only exposed as an API. The best way to interact with this API, is via the curl command line utility. This article walks you through setting up a server system and interacting with the API. In the cluster to cluster scenario, the Node API is designed to distribute the sessions over the cluster on the initiating side.

Configuration

Datacenter (on premises) system

1. Install Aspera Server (3.6) on Linux 64 bit

rpm -ivh aspera-entsrv-3.6.version-linux-64-release.rpm

2. Create a user & API access

Create a system user / password

# useradd xfer# passwd xfer(NOTE: You will be prompted to provide a password)

NOTE: You can also specify transfers_multi-session in the JSON transfer spec. The default retry duration is 20 minutes, but if you want a longer retry period, you can specify it by setting transfers_retry_duration.

Cluster system

NOTE: The easiest way to setup a cluster in the cloud, is by using the Aspera Cluster Manager. Once you have launched your cluster, you need to create an Access Key and give that access key permissions to an S3 bucket. The server will need to have a few options enabled, as can be seen in the tuning options shown below.

To disable splitting of files altogether, you would configure the setting as follows:

<multi_session_threshold_default>0</multi_session_threshold_default>

Transfer initiation

The transfers API can be exercised via the Linux command line using the curl command. The API can be used to send and receive single files, file lists, or directories. Here is an example of the command syntax:

for spec.json - use the name of the spec file that you create (see examples below in Transfer specification examples)

for API_USER:API_PASSWORD - use the Node API user account credentials that were created in step #2 above

For IP_ADDRESS - use the IP address of the system initiating transfers

Transfer specification examples

This spec file is essentially the transfer recipe. Examples for sending, receiving, directories, file lists, and individual files are provided. Use your favorite text editor to create these files, for example: