Forum

Tango Feature Request 4: Defining a standard Tango REST API

During the 29th Tango Meeting we announced the possibility to make a standard Tango REST API.This feature request has been written in a more formal way below as a tentative for helping the discussion.Please give any feedback about the definition of this TFR and I will update this page as soon as I can.

I will open a new topic in the forum to discuss especially about the API .

Cheers,Vincent

Code name

ScopeThe scope of this feature request is focus on the definition of a REST API for Tango.This will include the definition of the HTTP request (URL, content, …)

ContextSeveral web application have been develop for Tango and a REST api has been often used.Here is a list of different existing implementations:

Problem to solveThere are many way to define an API using the architecture REST. Also there are different constraints among them the Stateless is the most important. RESTful is defining a set of rule to guarantee a compatibility between all of the Restful software. How to handle command with RESTful compatibility can be one of the topics among others.Also Tango already defines a standard API on how to access devices, attributes, properties… That can be matched with a Tango Rest Api.

All the web client application won't be compatible if all the different Tango REST server have a different interface.

Goal and milestones:The goal is to collect the different experiences to define a stable first version of the standard. First of all a proposition with different option will be released then to collect the feedback from the overall community. This will define a alpha version of the TANGO REST APIFinally a document will release before the Tango meeting the 30rd for approval from the board. This will define a beta version of the TANGO REST API.

Output:The first version of this API is expected to be release with:

A complete description of the API written in a document, online manual,…
A set of compliance test with a possible automatic test set
(Optional)Some possible reference implementation for the only purpose of education and given as an example. Different language can be proposed.
Release some implementation guidelines, i.e. caching and optimizations advises. For instance, implementation must be clever enough to combine several requests, i.e. several clients read the same attribute. This is implemented in mTangoREST server.

Benefit:A standard API provide an interoperability between the different implementation of the web server and the client. The leverage of the standard will allow to create an active ecosystem.The expectation is to grow the web development of Tango in a coherent way for the users.

Possible later improvement:Performance test: Can be part of the compliance test

On the server side byte array received from a remote Tango device is wrapped with BufferedImage without copying the array and is immediately written into the response.

This approach does not support TIFF images. So the server side must ensure to convert all images into JPEG first.

2. Commands

Command execution is always a GET request, even with an argument:

GET {API_ROOT}/device/sys/tg_test/1/DevString=Hello%20World%20!!!

And this was a major question when I first started to design the API. GET was chosen just for simplicity – one can execute commands directly from the browser's address bar.

Only GET is supported.

3. Events

GET {API_ROOT}/device/sys/tg_test/1/long_scalar_w.change

Events implementation is based on the Comet specification, i.e. server blocks request till it gets a notification from the remote Tango device. This approach may utilize all available connections in heavily events oriented applications. Therefore client and server must implement multiplexing/demultiplexing of events' requests. Plus it might be useful to have a timeout so that if server does not get a notification within the specified period of time - it releases the request and responses with an old value. In this case client may resend the request.

Other supported event types are: periodic, user, archive.

Only GET is supported.

4. Family/domain/device browsing

As Tango hierarchy perfectly matches REST's idea of a resource tree this case is straight forward:

GET {API_ROOT}/devices

lists all the devices defined in the Tango DB attached to this mTango host:

Caching in terms of HTTP response header is not supported. But mTangoREST server performs a binary caching of the response for 200 ms.

This was a quick overview of the REST API implemented in mTangoREST API. For more details please refer to the page.

Apart from API specification I think it is important to release some implementation guidelines, i.e. caching and optimizations advises. For instance, implementation must be clever enough to combine several requests, i.e. several clients read the same attribute. This is implemented in mTangoREST server.

And do we really need so much time?

In other words currently we will only get a beta version in an year. I believe this task can be accomplished much faster: in three month release alpha version, then perform a beta before next Tango meeting so board can agree on Release Candidate 1.

IngvordApart from API specification I think it is important to release some implementation guidelines, i.e. caching and optimizations advises. For instance, implementation must be clever enough to combine several requests, i.e. several clients read the same attribute. This is implemented in mTangoREST server.

And do we really need so much time?

In other words currently we will only get a beta version in an year. I believe this task can be accomplished much faster: in three month release alpha version, then perform a beta before next Tango meeting so board can agree on Release Candidate 1.

Thanks for your feedback.Great idea for the implementation guidelines. I will add that to the first post.Effectively we can have a stable version before the next Tango meeting. An approval from the community can be done online and just have a official presentation to the board.

<device-prefix>/ping_device. NOTE from now on <device-prefix> = tango/rest/{host}:{port}/Device/{domain}/{family}/{name}

<device-prefix>/get_device_info

Attributes:<device-prefix>/get_attribute_list;<device-prefix>/plot_attribute/{attr_name} [Reads attribute and returns its values as array or set of arrays depending if attribute is of type SPECTRUM or IMAGE.]; PUT <device-prefix>/read_attributes

First of all this is not a REST API meaning that it does not follow REST specification. For instance, a tree structure of the resources ain't satisfied, i.e. instead of <device-prefix>/attributes we have <device-prefix>/get_attribute_list. API overflowed with redundant case specific commands, like 'plot_attribute', 'put_property', 'command_inout' and so on.

Return value: "measure date: 14\/06\/2015 15:00:30 + 137ms\nquality: VALID\nRead:\t3.14\n” seems very strange to me saying the least. This is a string client must parse! Other issue is the date – it should be simple timestamp, because it is much more convenient for the client then to do something with it.

Errors handling:API does not return anything more than “Attribute double_xxx not read. Unable to connect with device sys\/tg_test\/1” [NOTE: there is no such attribute 'double_xxx'], i.e. in case of any error raised by a tango proxy on the serer side a client will get “Unable to connect”

Tango related features:

API exports any Tango data base to the outside world. This is very questionable approach, hence very interesting if not 'Device' anchor in between, i.e. GET http://localhost:8080/tango/rest/localhost:10000/sys/tg_test/1 seems much more elegant in this case. Another issue with this anchor is that it is capitalized, i.e. server won't match anything if one asks for …/localhost:10000/device/sys/…

Tango hierarchy is not supported: GET restfultango/rest/localhost:10000/device/sys/tg_test returns 404

Commands execution always requires argin, even if it is a command(void). Moreover in case of void it requires special argin=DevVoidArgument

No events.

No TangoAccessControl integration.

Conclusions

API definitely provides some interesting ideas could be brought into Tango REST API specification. Like access to properties and control of the Tango proxy.

P.S. next week I will look through GOTAN REST API. Meanwhile I will appreciate if other involved people look through mTangoREST, GOTAN and Solaris's RESTful Tango so we can start to discuss common things and how to combine them into a single Tango REST API

The only thing from my point of view worth to stress is that API is a little bit messy in terms of using http methods. For instance, attribute write must be PUT not POST, as it does not create anything new, just writes a new value, i.e. updates an attribute. Read attribute must be GET and so on.

Anyway next week I will prepare a draft document that defines REST API for Tango.

It has been useful for me to look at this since I am involved in a project where we are going to use a rest api as well. It looks a well though api, I just have a few comments/questions:

Device Commands: In the forum it is specified that commands are still to be decided if they are GET or PUT requests (with a first choice for GET), however in the proposal api it is being used POST. I like the idea of being able to test easily directly through the browser, however I feel more inclined to think a command as a PUT request since you are acting on a device, i.e., triggering the execution of whatever actions the command has.

When talking about partial answers, the GET request ~hangs from the device only? The following requests would still be valid?

GET /mtango/rest/device/sys/tg_test/1/ampli?fields=argout
GET /mtango/rest/device/sys/tg_test/1/ampli/info?fields=unit, format
GET /mtango/rest/device/sys/tg_test/1/info?fields=classname

GET /mtango/rest/database/info (answer: number of devices exported, etc.)
PUT/GET /mtango/rest/database/commands/export?argin=device_name

Properties: when writing new values for several properties, any property not specified will be deleted. I feel it is a bit risky and prone to property losses. I would suggest to leave the rest of the properties unchanged, a safer approach.

Device Commands: In the forum it is specified that commands are still to be decided if they are GET or PUT requests (with a first choice for GET), however in the proposal api it is being used POST. I like the idea of being able to test easily directly through the browser, however I feel more inclined to think a command as a PUT request since you are acting on a device, i.e., triggering the execution of whatever actions the command has.

I agree - PUT seems a little bit more natural, as it updates the "state" of the device, i.e. executes a command.

mikegu

When talking about partial answers, the GET request ~hangs from the device only? The following requests would still be valid?

There will be a generic filter implementation for any response, i.e. one can always request only that portion of the response which is actually needed. In terms of implementation this will be a servlet filter on the server side (at least I think to implement it in this way).

Tango Database is just another Tango device. You can access it using the same API.

mikegu

Properties: when writing new values for several properties, any property not specified will be deleted. I feel it is a bit risky and prone to property losses. I would suggest to leave the rest of the properties unchanged, a safer approach.

I agree that this approach is quite misleading and counterintuitive. I can not image any scenario where it can be useful.

In fact in the next edition of the proposal I will remove PATCH request at all. Apart from introducing unnecessary complexity there is also a problem - it might not be implemented everywhere. Like in raw Java HTTP library.

The community

Tango Controls is a toolkit for connecting hardware and software together. It is a mature software which is used by tens of sites to run highly complicated accelerator complexes and experiments 24 hours a day. It is free and Open Source. It is ideal for small and large installations. It provides full support for 3 programming languages - C++, Python and Java.