Importing Data

There are two ways to import data into your Sanity project.

The recommended way of importing data is to use the Command Line Interface. You can run sanity dataset import --help for a quick summary of syntax and options. Your other option is to use one of our client libraries and handle it yourself.

Import using the CLI

The Sanity import tool operates on newline-delimited JSON (ndjson) files. Basically, each line in a file is a valid JSON-object containing a document you want to import.

Documents should follow the structure of your data model - most importantly, the requirement of a _type attribute. The _id-field is optional, but helpful in case you want to make references, or to be able to re-import your data replacing data from an old import. _ids in Sanity are usually a GUID, but any string containing only letters, numbers, hyphens and underscores are valid.

During import, all references are automatically set to weak, then flipped to strong after all documents are in place. This ensures that you can import documents that reference other documents in any order you like.

Assets (images and files) are stored using references in Sanity. To make it easy to import these and refer to them within your documents, you can use a special _sanityAsset-property where you would normally put a _ref. For instance, lets say you want your document to end up like this:

Note that you need to prefix the asset URL with a type declaration, either image@ or file@.

If your asset is on the Internet use image@https://example.com/path/to/rogue-one-poster.jpg instead of image@file:///local/path/to/rogue-one-poster.jpg.

Gotcha

File URIs are absolute so include the entire path.

Once you have prepared your ndjson-file, you can run the import using the Sanity CLI:

sanity dataset import <file.ndjson> <targetDataset>

E.g.:

sanity dataset import my-data-dump.ndjson production

Gotcha

This import fails if an incoming document already exists. A couple of options allow you to amend this:

--replace Overwrite existing documents. If you specify _id in the imported data, this flag can be very useful. It will let you reimport stuff that you got wrong in an earlier pass.--missing Only create documents which don't exist, leave the rest alone.

Import using a client library

If you prefer not to use our CLI import tool, you may of course do the import yourself with help from one of our client libraries.

There are some common pitfalls to keep in mind:

Concurrency. While you may have thousands of documents to import, you shouldn't trigger thousands of requests in parallel. This is going to exceed API rate limits and might fail. We advise you to use a queue with a reasonably low concurrency.Use a library to keep your import below our API rate limit:

API usage limits. Importing large data sets can quickly cause a lot of requests, especially if you import a single document per request. It is usually a good idea to send multiple mutations within a single transaction.

Mutation size limits. While it's a good idea to do multiple mutations per transaction, you need to make sure that the size of the request is within our limits, in terms of byte size.

Mutation visibility. Clients use the visibility mode of sync by default, which means that it will wait for the documents to be searchable before returning. This should not be necessary when importing large data sets. Instead, use deferred. If you have a lot of documents, it can take a little while for them to be searchable, but the import job will move along much faster.

References. If you are referring to one document from another, they either need to be imported in the right order, or the reference needs to be flagged as weak by setting the _weak property to true. After importing, you probably want to remove the weak property in order to prevent referenced documents from being deleted.

Assets. Since assets (files, images) in Sanity are stored using references, you'll need to upload the assets first and put the returned document ID in your reference.

With this in mind, do check out our client libraries documentation to see how to perform mutations.