Import API

The Stitch Import API is a REST API that allows you to push arbitrary data into your data warehouse. Once the data enters the Import API, it’ll be processed and sent through Stitch like data from any other integration.

The Import API accepts JSON or Transit and returns JSON for all of its methods. Each method uses a standard HTTP verb (GET/POST) and standard HTTP response codes for returning statuses.

Before you get started, consider where the data that you want to push currently lives: is it already in a database or SaaS integration we support? A few of our SaaS integrations allow you to sync data at the tabular level; all database integrations (except for Mongo) can be synced to the columnar level. In addition, you can define Replication Methods for tables that come from a database integration.

In short, unless you want complete control over data replication, it’s usually easier to use the native integration instead of using the Import API.

Note that if someone does get ahold of your key, they won’t be able to access your data - they’ll only be able to send it. If at any time your API access token is lost or compromised, you can revoke it and generate a new one.

Example Request Header

The API access token is used to authenticate to the Import API via bearer auth, as demonstrated in the example below:

Return Codes

the 4xx range indicate a bad request - for example: invalid or omitted credentials, a required field is missing, etc.

the 5xx range indicate an error on our end. We recommend checking our status page for reported outages if you receive a code in this range. If nothing has been reported and these errors persist, please reach out to our support team.

Your app should handle each of the following return statuses gracefully:

200 - OK

Success. New data was added as a result of the request.

201 - Created

The request was accepted and will be processed later; new data will be added once the request is processed.

202 - Accepted

The request was accepted, but cannot currently processed due to an internal error. Data will be automatically re-processed.

400 - Bad Request

The request includes malformed JSON/Transit, did not provide an array of records, or contains more than 20,000 records.

401 - Unauthorized

The request did not have a valid API access token. You should generate a new token and re-try your request.

403 - Forbidden

The request had a valid API access token, but is not permitted. You should check that the client_id field in your request contains the correct client ID.

405 - Method Not Allowed

The request has an HTTP method that is not supported by the endpoint. POST and GET are supported by the Import API. Check that you’re using the correct HTTP method.

413 Request Entity Too Large

The size of the request body was over 4MB.

415 Unsupported Media Type

The content header of the request did not specify JSON or Transit.

422 Unprocessable Entity

The request is missing a required parameter. Check that the array of records in the request contain all of the required request body fields.

503 Service Unavailable

The Import API is experiencing problems. Try again later.

504 Gateway Timeout

The Import API could not process the request in time. Try again later.

Methods

The Import API supports three methods:

Status - The status endpoint can be used to determine if the Import API is operating correctly. This endpoint doesn’t require any authentication.

Validate - The validate endpoint can be used to validate requests but will not persist them to Stitch. We recommend using this endpoint for development and testing.

Upsert - The upsert endpoint is used to push data into your data warehouse.

Status

The status endpoint can be used to determine if the Import API is operating correctly. This endpoint doesn’t require any authentication.

Example Response

Validate

The validate endpoint can be used to validate requests but will not persist them to Stitch. This endpoint will validate your credentials, meaning you can check your token or client-id while developing and testing. If either credentials are invalid, a 403 Not Authenticated message will be returned.

The behavior of this endpoint mirrors that of the upsert endpoint, with two exceptions:

If the request is valid, a 200 OK response will be returned.

Regardless of whether Stitch is functional, a 503 Service Unavailable response will never be returned.

Required Request Body Fields

This field contains the name of the destination table, ex: users. See Defining Tables for more info.

sequence

This property tells the Import API the order in which data points in the request body should be considered. See Defining Sequence for more info.

action

This field contains the action for the endpoint. For the Upsert endpoint, it will always be upsert.

key_names

This field defines the Primary Key and will contain an array of field names that uniquely identify the row that the record belongs to.

data

This field contains the data to be upserted into your data warehouse.

If the request was successful, the response will have an HTTP status code of 201 Created and an empty body.

Define Tables

When you push data to an arbitrary table name using the Import API, the table will be generated dynamically in your data warehouse. When creating requests, keep the following in mind:

The Import API doesn’t enforce any limitation on the hierarchy of your tables.

There aren’t any commands to create or destroy a table in the Import API.

You should create one table for each type of data point that you’ll push. For example: if you have customer and product data, you should create customer and product tables.

Each data point pushed to a single table should have the same data structure. For example: if a customers table contains customer_id, name, and email columns, every customer record pushed into this table should contain those columns.

You can push more than one table using the same API access token. Think of it this way: one schema for each API access token. All tables pushed using the same API access token will be housed in the same schema in your data warehouse.

Define the Sequence

Sequence properties communicate the order in which data points should be considered – newer data points can replace older ones, but not vice versa.

Every data point pushed to the Import API must have a sequence property.

A simple solution is just to use the current timestamp, but before doing so, consider the following:

Are the rows being considered frequently updated?
Rows that are updated every few milliseconds can result in failure if records with identical key values are pushed simultaneously. This means that records with the same key values cannot be sent during the same clock resolution.

For example: if the resolution is measured in milliseconds, records with identical key values cannot be sent during the same millisecond.

Are the records coming from multiple sources?
If records from multiple sources will be sent to the Import API, the time clocks of these sources must be synchronized. This is especially important if different sources are pushing rows to the same table.

Sequence Example

Take a look at the following example. The first three requests are all for one record (id 10), while the fourth is for a different record (id 22).

If the requests were received in this order:

Requests 1 and 2 would continue to Stitch, but not Request 3. This is because Request 3 has a lower sequence value than Request 2.