We are just giving MongoDB a test run and have set up a Rails 3 app with Mongoid. What are the best practices for inserting large datasets into MongoDB? To flesh out a scenario: Say, I have a book model and want to import several million records from a CSV file.

I suppose this needs to be done in the console, so this may possibly not be a Ruby-specific question.

Edited to add: I assume it makes a huge difference whether the imported data includes associations or is supposed to go into one model only. Any comments on either scenario welcome.

2 Answers
2

Assuming you have an existing database in SQL, the easiest way to migrate that data is to output your SQL data as JSON strings, then use the import tool for each collection.

This includes denormalization and nesting/embedding - so don't migrate a relational model to MongoDB, you should consider also refactoring your data model to leverage MongoDB features.

For example, a common task is to merge articles and tags to an articles collection with the tags embedded as an array. Do that in your export script, so all MongoDB sees is nice clean JSON coming in through the import :-)

You can still import all your tables as collections, but you're missing out on some of the true strengths of MongoDB by doing that.