Choose a platform: I picked nodejs, for the only reason that I am proficient in it

Choose a database: I took part to a similar project where I used readDB, commonly called mongoDB. It was painfully slow when upserting lots of data, so I chose couchbase instead, a key/value store with a document query language.

After a while I needed “full text” search, so I added elasticsearch, simply because I knew how to use it. It requires a few (ugly) scripts to automate its synchronization with couchbase.

And suddenly came the epiphany! Or to be more precise, I asked for a review and after detailing the context, my colleague simply asked: “why don’t you remove couchbase?”. He was spot on: the dataset fits in memory and it is readonly.
It was replaced with a simple import to load the JSON in memory ready to be queried by the application.
But it got me thinking, couldn’t I remove the second database too? And I learned that there are indeed multiple libraries to search into local documents: I just used lunr and could therefore remove both databases.

The application is now more simple and reliable:

The code base has been reduced: 122 untested lines of script, 227 lines of configuration

The application is now deployed with one image: less reasons to fail

No more scripts to load the data in the database (the file is read by the application)

Nonetheless, after the refactor, the application was a bit slower even if everything was done in memory. Having said that, response time is still below 20 milliseconds, so the advantages brought by this removal still exceed the loss of a few milliseconds.

This resulted in a pull request with twice as many deleted lines as added lines and my takeaway would be:

Following KISS principles is more difficult than it seems: you can always question yourself but you should more likely seek for feedback from your peers. And, instead of choosing a database straight away, I would now choose a place to store my data.