README.md

jKool Streaming & Query API

jKool Streaming & Query API allows you to stream events, metrics, transactions as well as execute queries against jKool streams. You will need a streaming "access token” in order to stream & query your data. This token is associated with a repository assigned to you when you sign-up for jKool. Other language bindings can be generated with the Swagger Code Generator using the Swagger yaml file found it the "swagger" folder. Please be aware the the Swagger yaml file is documenting every field that can be passed via Restful API. When using this Java Helper API, many fields will be defaulted or automatically computed for you.

Concepts and Terminology

You can find very comprehensive documentation on jKool Data Types and Concepts in our jKool Streaming Guide. There are four types of timeseries data types that can be streamed to jKool. They are:

Event: Represents a basic time series element containing time, message, severity and other fields associated with event.

Activity: Represents a group of events and other activities (e.g. transactions).

Snapshot: categorized metrics (name, value, type) at a "point in time". Snapshots can be associated with events, activities.

This Git repository contains a Swagger yaml file. Open this file in a Swagger Editor and you will have detailed documentation of each field that comprises the above mentioned data.

How to build

To use this sample code please do the following:

Run mvn install on the project. This will generate jkool-client-api-<version> jar file. This jar file can be found in the target directory. Be advised that when running from the command line (as documented below), run from the build directory that Maven will assemble. This build directory will be at the same level as the directory you run Maven from.

Streaming to jKool

Streaming allows developers to send time series data such as events, metrics, transations, logs over secure jKool cloud interface. You will need your jKool access token that you received when you signed up for jKool. This token ensures that the your data goes to the repository associated with the access token.

JKStream jkSend =newJKStream("yourtoken");

Create an event and populate the fields you wish to stream. For example:

Please note that this example code depicts streaming in real-time. Therefore the start date of the event will default to the current date/time and the end date will default to the start date plus the elapsed time. You can however control start/end dates. For example:

The Client API formats the entity into JSON format and streams it to jKool over default https protocol.

Running jKool Queries (Synchronously)

In addition to streaming, data can also be retrieved from jKool via Rest. To do this, make use of the jKool Query Language (JKQL). Please see JKQL Documentation. Use the JKQuery to run JKQL queries synchronously. Simply pass in your access token along with the JKQL query. Below is an example:

Running jKool Queries (Asynchronously)

Developers can also invoke JKQL queries asynchronously using callbacks. To do this, make use of the JKQueryAsync. Below is an example. This example makes use of two connection handlers: 1) for tracing connection events and 2) for retrying connection during failures.

Next execute your query. All response will be delegated to all default callback handlers, because no callback has been associated with this query :

// run query in async mode without a callback (use default response handlers)
jkQueryAsync.callAsync("get number of events for today");
...
jkQueryAsync.close(); // close connection

Alternatively you can execute a query with a specific callback instance. All responses associated with this query will be routed to the callback instance specified in the jkQueryAsync.callAsync(...) call.

MyJKQueryCallback.handle() is called when for every response to the query -- there maybe one or more responses depending on the query. MyJKQueryCallback.done() is called when the handle will never be called again. This happens when the query is cancelled using JKQueryAsync.cancelAsync() call or when all responses associated with a specific query have been delivered.

jkQueryAsync.callAsync() returns a query handler (instance of JKQueryHandle), which can be used later to cancel subscriptions.
Cancelling an active query subscription attempts to stop any streaming traffic associated with a specific subscription.
Cancellation is also issued asynchronously and any responses that are still in transit will be routed to the default response handler specified by addDefaultCallbackHandler() call.

// run query in async mode with a callbackJKQueryHandle qhandle = jkQueryAsync.callAsync("get number of events for today", newMyJKQueryCallback());
...// attempt to cancel subscription to the query results
qhandle.cancelAsync(qhandle);

JKQL queries can aslo be executed using prepared JKQL statements as follows:

Subscribing to real-time event streams

Developers can also subscribe to live data streams using JKQueryAsync class. Subscriptons are based continous queries submitted by the client and run on the jKool servers. The results of the query are emmitted as data becomes available and streamed back to the client call back handler instance of JKQueryCallback. See example below:

The code above is equivalent to the JKQL statement subscribe to events where severity > 'INFO'. MyJKQueryCallback() gets called as the query matches incoming streams. All pattern stream matching is done on the jKool server side. subscribe query runs on real-time streams only and never on past data. Use get queries to get past data.

Running jKool searches on message content

JKQueryAsync class provides a helper method to run pattern macthes against event message content. See below:

The code above is equivalent to the JKQL statement get events where message contains "failure", where 10 is the maximum number of matching rows to return (default is 100). The example above can be implemented as:

Streaming with Python

Data can also be streamed natively (without helper classes) into jKool using Python. To do so, you will need to use the Python "Request". Details on the Python Request can be found here - http://docs.python-requests.org/en/latest/user/quickstart/. Below is an example of using it to stream into jKool:

Query jKool using Curl

Rest can be used to retrieve data natively (without helper classes) out of jKool uing Curl. Below is an example. Please note that you also have the option of putting the token in the header instead of specifying it as a parameter.

Note on time stamps

Time stamp fields such as time-usec, start-time-usec and end-time-usec are measured in microseconds (usec), between the current time and midnight, January 1, 1970 UTC. Most language environments don't return such time in microsecond precision, in which case you would have to compute it by obtaining current time in milliseconds and convert to microseconds (e.g. System.currentTimeMillis() * 1000).