Getting started

This page describes how to install and interact with a standalone version of the Warp 10™ platform. It also describes WarpScript™ IDE tools.

The standalone version of the Warp 10™ platform uses LevelDB as its storage layer, this version is suitable for managing
a few tens of millions of time series and few hundreds of billions of datapoints. If you have larger needs,
a distributed version of the platform is also available which uses Apache HBase as a scalable storage layer.

Warp 10™ is a server. It uses http requests on different endpoints, one for each task. In this introduction, we will use only two endpoints :

While discovering Warp 10™, you will hit some restrictions (number of data in a FETCH, maximum number of operations). All the Warp 10™ configurations are in etc/conf-standalone.conf.

Step 5. Run Warp 10™

As warp10 user:

./warp10-standalone.sh start

You may also launch Warp 10™ as root with sudo ./warp10-standalone.init start

At the first start, this script will create the LevelDB database and all the stuff surrounding it.
An initial set of tokens will be provided and some useful commands to start playing with your Warp 10™ instance.

Useful resources:

Warp 10™ standalone config conf-standalone.conf has been generated in the etc directory

Initials tokens are available in the initial.tokens file in the etc directory

Logs are available in the logs directory

Data are stored via LevelDB in the leveldb directory

Snapshots of LevelDB data can be performed via the init script. More information here.

Step 6. Generate Warp 10™ tokens

The Warp 10™ platform is built with a robust security model that allows you to have a tight control on who has the right
to write and/or read data. The model is structured around the concepts of data producer,
data owner, application, within WRITE and READ tokens.

By default, at the first start, Warp 10™ init script will generate both a READ and a WRITE token for the test
application io.warp10.bootstrap for a test user that is both the producer and the owner of the data.

Then, you can generate your own write and read tokens by executing the worf command via Warp 10™ init script.

As the warp10 user, generate a token for MyTestApp with a 10 year lifespan :

In order to interact more precisely with the user/token/application system, you may need an interactive access to Warp 10™'s
Worf console. More information can be found here.

Step 7. Prepare data in GTS data format

Data is sent into the platform via HTTP POST requests to the Warp 10™ API.

API Endpoint

The HTTP endpoint used to send data is http(s)://host:port/api/vX/update, where vX is the version of the API you want
to use (currently v0). In order to be accepted by the platform, requests to this endpoint need to be authenticated,
by using a X-Warp10-Token HTTP header with your write token.

Data format

If you have your own data, you can now convert them into GTS format. If you do not have, you can use our example datasets.

One GTS is one time series.

A GTS is defined by its CLASS and LABELS value.

Knowing this facts, up to your use cases, you must choose wisely what you will put in names and labels.

Data is sent in the body of the POST request, one data point per line. Each line follows the GTS input format:

TS/LAT:LON/ELEV CLASS{LABELS} VALUE

Where:

TS

*Optional* Timestamp of the reading, in microseconds since the Unix Epoch.
If omitted, the timestamp associated to the data point will be the one of the platform when the point is pushed.

Class name of the reading as a URL encoded UTF-8 character
string. The encoding of character `{` (Unicode LEFT CURLY BRACKET, 0x007B) is MANDATORY.

LABELS

Comma separated list of labels, using the syntax `key=value` where both `key` and `value` are URL encoded
UTF-8 character strings. If a key or value contains `,` (Unicode COMMA, 0x002C),`}`
(Unicode RIGHT CURLY BRACKET, 0x007D) or `=` (Unicode EQUALS SIGN, 0x003D), those characters MUST be encoded.

VALUE

The value of the reading. It can be of one of four types: `LONG`, `DOUBLE` (you must have a dot decimal separator), `BOOLEAN` (character T or F), `STRING` (anything url encoded between single quotes)

Data model example

This is a car maker example:
Each vehicle records time, position, vehicle speed (km/h), engine rotation speed (rpm), and engine torque (Nm). You plan to look for the time a customer spends on the second gear ratio, and try several models of gear wear, up to torque applied. You can choose to have three different class names:

VehicleSpeed

EngineRpm

EngineTorque

And labels that will help you to identify the GTS:

VehicleSerialNumber

GearboxType

Because you know the formula will depend on the gearbox, and the gearbox type will never change in the vehicle life. Adding gearbox type as a label will allow you later to select only this gearbox type to perform a computation.

Step 8. Pushing data

In the default configuration, Warp 10™ listens on 127.0.0.1 only.

If you plan to access it from another machine on your network, you must change standalone.host to listen on any interface. Edit config conf-standalone.conf in the etc directory with your favorite text editor, and restart Warp 10™.

Pushing data with curl

If you're on a *nix system, you can push the data using cURL.
The Warp 10™ platform listens to the port 8080 of your container. In the setup we have mapped this port to
the port 8080 of the host machine, so you can make your request to 127.0.0.1:8080.

If everything is OK, you should receive a HTTP 200 OK with the most recent datapoint of every GTS in JSON format.

When using Docker on Mac OS or Windows, there is no binding between Warp 10™ API address and the host (docker is
ran throw a Virtual Machine). To reach Warp 10™ you need to replace 127.0.0.1 by the real IP address of the container.
To get it, use a simple docker-machine ip default>, the container address is also shown in the Settings/Ports page of
your container. If you used the shared volume between the container and the host, you can access to the virtual machine
using docker-machine ssh default> and inspect the repertory /var/warp10. Don't hesitate to check on
docker-machine documentation.

Pushing data with python

This small python program will take GTS text file as arguments, and will push them using requests library.

Step 8. Quantum interface

Warp 10™ can be extended with plugins and extensions.
Quantum is a plugin delivered and preconfigured with the standalone or docker version.
Warp 10™'s Quantum is a web application aiming to allow users to interact with the platform in an
user-friendly way, offering an alternative to command-line interaction.

Open http://localhost:8090/ in your favorite browser (if needed, replace localhost by the IP of the Warp 10™ host).

Once backend configuration is done, you can execute WarpScript on your Warp 10™ instance. For example, the very first WarpScript you type in this tutorial, using curl, to read all data you pushed:

//store read token in a variable
'hMeJHyhK5wiHur_dsO8kccnzWLAVQO7eOfvOVhXAlmftexalPBy6cENIHwv8JDx10BSBlW0Baz.oOj1uONYYY3H5zdD_WTGePMC_cb2lRr1m_IfUgt.sZYtTdlvU0zbB' 'myreadtoken' STORE
//fetch the most recent point of every GTS this token can access
[ $myreadtoken '~.*' {} NOW -1 ] FETCH

Step 9. VSCode IDE for WarpScript™

Visual Studio Code is a source code editor developed by Microsoft for Windows, Linux and macOS. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets, and code refactoring. It is free and open source. We developed a plugin for WarpScript files (.mc2 extension).

//store read token in a variable
'hMeJHyhK5wiHur_dsO8kccnzWLAVQO7eOfvOVhXAlmftexalPBy6cENIHwv8JDx10BSBlW0Baz.oOj1uONYYY3H5zdD_WTGePMC_cb2lRr1m_IfUgt.sZYtTdlvU0zbB' 'myreadtoken' STORE
//fetch the most recent point of every GTS this token can access
[ $myreadtoken '~.*' {} NOW -1 ] FETCH

If the extension is correctly loaded, you now have syntax highlighting, autocompletion, hover help, variable completion, local macro substitution...

By default, VSCode extension will execute WarpScript on localhost API (http://127.0.0.1:8080/api/v0/exec). To change the Warp 10™ server address, open the VSCode preferences (Ctrl+Shift+P, type "settings", press enter), search for "warpscript", and copy paste a customized line from the left pane to the right pane.
If you plan to use multiple instances of Warp 10™, you can also specify an endpoint in the very first comment line of the file. VSCode will interpret it.

Press Ctrl+Alt+E to run the warpscript.

JSON result will be displayed on the right part of the editor. In this example, we count the number of GTS returned by the FETCH command. Up to context, JSON can context GTS or PNG images. WarpScript extension will also open GTS Preview or Image Preview tabs.