MongoDB Overview

MongoDB is a document database with the scalability and flexibility that you want, with the querying and indexing that you need. Read more about MongoDB. Etlworks Integrator can read and write data stored in MongoDB.

Working with MongoDB

Working with MongoDB is similar to working with files. The MongoDB document id is similar to a file name, and the document associated with the document id is comparable to the file's content.

Connecting to MongoDB

In addition to the MongoDB connection, you will need to create a JSON format. If you intend to only read data from MongoDB, a JSON format with the default settings will work just fine. Below, we will explain how to create a format for writing into a MongoDB.

Reading data from MongoDB

Step 1. Create a MongoDB connection. Optionally, enable Remove Object ID when Reading. If this option is enabled, the system will remove the document ID when reading the document. It can be useful if the document is a JSON array and you just need to read the content of the JSON array.

Step 2. Create a JSON format. The default settings for the JSON format will work.

Step 3. Create a flow, where the source is a file and the destination is anything.

Step 4. Select the MongoDB connection created in Step 1, and the JSON format created in Step 2 as a source (FROM).

Step 5. Enter the source into the FROM field. The following options are available:

A fully qualified document name - the document id as it is stored in MongoDB.

A wildcard document name, for example, sales_orders*. The system will read all the documents which match the wildcard, and will combine them into a JSON array, where each document is one row in the array.

A request to MongoDB in the JSON format. The system will read all the documents which match the request, and will combine them into a JSON array, where each document is one row in the array. Here's an example of a request: {"first":"Simba"} .

Step 6. Select a destination connection, format (if needed), and enter the destination into the TO field.

Writing data into MongoDB

Step 2. Create a JSON format. If the source document to store in MongoDB is an ARRAY, for example, a database table, CSV file, JSON array, etc., set the First Object Type to variable and the Root Object Name to whatever name you want to use as a name for the root element. This field is required.

Other Use Cases

Copying Documents into MongoDB

To copy existing documents, for example files, into MongoDB, use Copy Files flow type, where the destination is a MongoDB connection.

Important: the source document must be in the JSON format, which can be serialized to BSON, which is used by MongoDB.

Downloading Documents from MongoDB

To download documents from MongoDB as files, use Copy Files flow type, where the source is a MongoDB connection.

When specifying the source (FROM), the following options are available:

A fully qualified document name - the document id as it is stored in MongoDB.

A wildcard document name, for example, sales_orders*. The system will read all the documents which match the wildcard, and will combine them into a JSON array, where each document is one row in the array.

A request to MongoDB in the JSON format. The system will read all the documents which match the request, and will combine them into a JSON array, where each document is one row in the array. Here's an example of a request: {"first":"Simba"} .