SMILA/Documentation/Using The ReST API

Most functionality of SMILA is already accessible via an HTTP ReST API using JSON to represent data.
This means basically that you can control and monitor a running SMILA server using your web browser (at least with some small plugins).
Also, it is relatively simple to talk to SMILA programmatically or using scripts. On this page we will recommend some tools and
best practices to work with SMILA.

Feel free to extend this page if you find interesting tools that you want to recommend to other SMILA users.

Contents

Basics

An HTTP ReST API consists of a set of URLs that can be invoked using standard HTTP requests. One URL refers to a "resource" in the system, which may be a service or some element managed by a service. Each resource may support several HTTP methods for different operations, but not each resource will support every method. In SMILA we use the following methods, usually with one of the described semantics:

GET: get the content, definition or description of the resource; get statistic data or state information about the resource

POST: execute the main functionality of the resource (e.g. execute a pipeline); add or update a sub element of the resource

PUT: set the content, definition, or description of the resource

DELETE: clear the contents of the resource; remove an element from the resource.

Most resources should at least support the GET method which is the method used by the web browser when you enter the URL, and the result should contain links to resources associated with this resource. So you can explore the system state with your web browser.

GET and DELETE requests never have a request body, parameters must be passed by using ...?param1=value1&param2=value2 in the URL. With POST and PUT requests the data is passed in the request body as a JSON object, i.e. a JSON string like this:

{"param1":"value1","param2":"value2
}

In SMILA, this corresponds to the metadata part of a record. In POST requests it is also possible to add binary attachments, see Documentation of the SMILA HTTP server for details. Finally, some resources accept or produce so-called "JSON bulks". This means that it is possible to send or get a bunch of records in a single request. The JSON of the record metadata must then be printed on a single line, and the records are separated by newlines. However, it is not possible to add attachments to such records.

Interactive Tools

Web Browser Add-Ons

For most web browsers there are free add-ons available that support pretty-printed viewing of JSON documents and the interactive invocation of ReST commands.

JSON viewer: By default browsers do not know how to handle JSON results and will suggest to download them to a file. There are add-ons that display the JSON right in the browser, nicely formatted and highlighted and make contained URLs clickable, which makes it easy to explore the system state

ReST Client: ReST Client allow to compose other requests than GET requests with the browser. They allow to enter the resource URL, select the request method, and add the request body content, and then you can execute the request and view the result. There are lots of these add-ons out there, we can recommend the following:

cREST Client: Easy to use and keeps a nice history of the executed requests so that it is easy to repeat sequences of commands for test purposes.

Advanced REST client application: A bit harder to use, but the only one we currently know of that lets you create "multipart" requests containing record attachments: In the following screenshot we have created a request to submit a record (first file field contains name of a JSON file) with an attachment (second file field) to the "indexUpdate" job:

Shell scripting

Basically, it is possible to work with the ReST API using generic HTTP tools like cURL. However, for improved usability you might want to check out resty. This is a simple wrapper (for bash and zsh) for cURL that allows you to write ReST requests like shell commands:

Using cURL to insert a record with some metadata and a binary file as an attachment. The parameter name "metadata" is fix, but the parameter name "content" could change in your scenario. In that case please check your pipeline for the name of the parameter where the content attachment would be processed.

# send a metadata record and a binary attachment to a etlImport job
> curl -F "metadata=@record.json" -F "content=@binary-file.pdf" http://localhost:8080/smila/job/etlImport/record

Programmatical Access

It's quite easy to use the SMILA API from external programs. In Java, basically everything you need is available in packages java.net. But it's easier using the SMILA data model classes and JSON utilities and a HTTP library like the Apache HttpClient. Below we provide some simple examples for doing requests with it.

There are also free Java libraries available to make working with JSON ReST APIs easier, e.g. Resty (yes, same name, but another tool ;-). However, we do not have experience with them.