Integrations and APIs

Whether you want to build event-driven data integration flows or incorporate Integrator into your own automation workflow, you can use the developer features below to achieve your goals. All the APIs in Integrator are REST-based.

Calling Integrator API endpoints with JWT auth

Step 1. Create an API user. Note: you can actually use any user, as long as they have a role that can execute flows.

Step 3. Use the access token, received in Step 2, to call Integrator API endpoints. The access token must be submitted as a header parameter, as in: Authorization:Bearer access-token, or query parameter, as in: ?Authorization=Bearer%20access-token.

Important: access tokens in Integrator are short-lived and self-expiring. An access token gives you access to the APIs for approximately 10 minutes. It is recommended that you refresh the access token before each call to the API.

Calling Integrator API endpoints with basic auth

Step 1. Create an API user. Note: you can actually use any user, as long as they have a role that can execute flows.

Step 2. Use the user name and password for the user created in Step 1, to call an API endpoint.

To execute flow by name synchronously:

EXAMPLE (with no parameters): https://app.etlworks.com/plugins/flows/rest/v1/flows/flow-name/my-flow/run, where my-flow is the name of the flow to execute.

EXAMPLE (with a parameter): https://app.etlworks.com/plugins/flows/rest/v1/flows/flow-name/my-flow/run/?patientid=123, where my-flow is the name of the flow to execute, patientid is a parameter and 123 is the value of the parameter. There can be multiple & separated parameters, following this pattern: ?parameter1=value&parameter2=value.

{"flowId":flodId,
"messageUuid":"UUID of the message if flow was executed by sending payload to the custom endpoint",
"auditId":"ID of the record in audit table if flow was executed by name or by ID",
"status":status-code,
"exception":"exception-stack-trace"
}

Possible status codes: 0 - success, 1 - error.

RESPONSE CODES: 200 for success, 401 and 403 for not authorized, and 500 for an internal error.

Get flow status

This API endpoint is used to retrieve current status of the flow, even when flow is currently running.

messageId=UUID. Example: https://app.etlworks.com/plugins/dashboard/rest/v1/flows/1234?messageId=15a20488-0224-4ae1-a52c-9ee65a69f2c5. If the value of the optional parameter messageId is not empty, the response will contain a status of the flow execution, associated with the particular message ID. The message ID is returned as a part of the response when the flow is executed by sending payload to the custom endpoint.

auditId=long. Example: https://app.etlworks.com/plugins/dashboard/rest/v1/flows/1234?auditId=1234567. If the value of the optional parameter auditId is not empty, the response will contain a status of the flow execution, associated with the particular audit ID. The audit ID is returned as a part of the response when the flow is executed by name.

REQUEST BODY: none

REQUEST CONTENT TYPE: none

RESPONSE: JSON document in the following format:

{
"auditId": long, // unique audit ID
"parentAuditId": long, // parent audit ID if there were auto-retry attempts when executing the flow. Null if there were no auto-retry attempts
"retries": integer,// number of retries or null
"flowId": long, // flow ID
"flowName": "string", // flow name
"eventId": long, // scheduled event ID
"eventName": "string", // name of the scheduled event or null if flow has not been scheduled
"messageId": "UUID", // message ID when flow has been executed by sending payload to the custom endpoint, otherwise null
"tenantId": long, // tenant id if flow was executed by super-admin user, otherwise null
"status": "string", // status of the flow execution: success, warning, error, running
"triggerType": "string", // flow.execution.scheduler or flow.execution.on.demand or null
"startedBy": "string", // who triggered the flow: scheduler, or user login name
"started": long, // start time in milliseconds
"ended": 1531158790495, // end time in milliseconds. Only available if flow has been already executed
"duration": "string", // human readable duration of the flow. Only available if flow has been already executed
"code": "string", // last executed SQL statement in case of error, or null. Only available if flow has been already executed
"exception": "string", // exception stack-trace or/end messages logged by the user using etlConfig.log(message). Only available if flow has been already executed
"warnings": "string", // warnings or null. Only available if flow has been already executed
"metrics": [metrics], // flow metrics or null. Only available if flow has been already executed
"fileMetrics": [file metrics], // file metrics or null. Only available if flow has been already executed
"errorLocationInSql": integer, // line number in last executed SQL in case of error, or null. Only available if flow has been already executed
"fileName": null, // currently not used
"records": 1 // currently not used
}

RESPONSE CODES: 200 for success, 401 and 403 for not authorized, and 500 for an internal error.

Audit-trail

Each event, such as a login, flow execution, etc. is captured in Etlworks Integrator's audit-trail log.

PARAMETERS: exception=false|true. Example: https://app.etlworks.com/plugins/audit/rest/v1/audit?exception=true. If the value of the optional parameter exception is true, the exception stack trace will be included in the response.

REQUEST BODY: none

REQUEST CONTENT TYPE: none

RESPONSE: JSON document in the following format:

[
{
"id": number, the audit record id
"an": string, the application name
"md": string, the event name
"ur": string, the user name
"tn": string, the tenant name
"rt": boolean, if true the event was successful
"in": boolean, if true the record has additional information
"ex": string, the exception stack trace
"hip": string, the host IP
"cip": string, the client IP
"dn": long, the duration in milliseconds
"dt": string, the start date+time in milliseconds
},
]

PARAMETERS: exception=false|true. Example: https://app.etlworks.com/plugins/audit/rest/v1/audit?exception=true. If the value of the optional parameter exception is true, the exception stack trace will be included in the response.

REQUEST BODY: JSON document in the following format:

{
"app": string, the application name
"method": string, the event name
"login": string, the user name
"fromDate": string, the start date for the filter in Unix epoch format
"toDate": string, the end date for the filter in Unix epoch format
"duration": long, the duration in milliseconds
"clientIp": string, the client IP
}

REQUEST CONTENT TYPE: application/json.

RESPONSE: JSON document in the following format:

[
{
"id": number, the audit record id
"an": string, the application name
"md": string, the event name
"ur": string, the user name
"tn": string, the tenant name
"rt": boolean, if true the event was successful
"in": boolean, if true the record has additional information
"ex": string, the exception stack trace
"hip": string, the host IP
"cip": string, the client IP
"dn": long, the duration in milliseconds
"dt": string, the start date+time in milliseconds
},
]

RESPONSE CODES: 200 for success, 401 and 403 for not authorized, and 500 for an internal error.

User-defined APIs

Integrator can be used as an Enterprise Service Bus, serving requests from third-party applications. Basically you can build an API, without writing a single line of code, which can be used by other applications to send payload to Integrator's API endpoints.

Below is a typical example of an event-driven flow:

The third-party application updates its own database.

The update triggers an event, which sends a notification to Integrator and includes updated record(s) as part of the payload.

There is an HTTP listener, configured in Integrator, which listens to the incoming HTTP requests.

There is also a flow, which has the listener as the source (FROM) connection. The flow is scheduled and ready to receive a payload.

Once the payload is received, Integrator automatically executes the flow and passes the payload as a source.

The source data is transformed and placed into the destination. For example, information about an updated record(s) is sent to a data warehouse.