Ruby + Graphs with Neo4j

Batch Importer – Part 3

At the end of February, we took a look at Michael Hunger’s Batch Importer. It is a great tool to load millions of nodes and relationships into Neo4j quickly. The only thing it was missing was Indexing… I say was, because I just submitted a pull request to add this feature. Let’s go through how it was done so you get an idea of what the Neo4j Batch Import API looks like, and in the next blog post I’ll show you how to generate data to take advantage of it.
First thing I had to do was update the pom.xml file to include the latest version of Neo4j, and the Lucene index dependency.

We pass in an indexType variable to decide if this index will be exact or fulltext. We create the index with lucene.nodeIndex. We then read the values to be added to our index from the file passed in. Once we capture the key and values from our file, we simply pass them along with our node id to the add method.

Like this:

Related

6 thoughts on “Batch Importer – Part 3”

[…] If you recall, I’ve had three blog posts about the Batch Importer. In the first one, I showed you how to install the Batch Importer, in the second one, I showed you how to use data in your relational database to generate the csv files to create your graph, and just recently I showed you how to quickly index your data. […]

[…] needed to import this into Neo4j. I followed the examples from Batch Importer Part 2, and Batch Importer Part 3 to do some ETL, but first I needed to load the data into Postgresql so I could match up the two […]

Batch importer is great but I’m having some issues getting the right type casting for edge properties (https://github.com/jexp/batch-import/issues/6). Any ideas what might be going wrong? I saw Michael saying somewhere that the importer script just skips the missing property values but I now suspect it might be affecting the typecasting. Would be great to have this resolved since currently I cannot do any queries in Cypher that use comparison operators for edge properties.

[…] we are now ready to import it into Neo4j using the Batch Importer. Much has changed since my last blog post about the batch importer. Michael Hunger has made our life easier by allowing us to specify a way to look up nodes by an […]