TinyDB: past, present & future

It’s almost 1000 days since the first commit of TinyDB, my small document-based Python database. And within these 1000 days the project gathered more than 700 stars on GitHub. So I thought I’d share my thoughts on this project’s origins, its current state and plans for the future.

The past

Given its origins, TinyDB is an unexpected success. In summer 2013 I was given the task to select photos for a summer camp to be handed to the kids. From previous experience I knew that I was facing about 5.000 to 6.000 photos from different photographers. I was trying to avoid a big clusterfuck or manually sorting the pictures into their appropriate directories (something like <day>/<time>-<photographer name>-<consecutive number>.jpg) so I figured I’d write a small Python program that would take care of that. It’s really nothing too fancy: just look at the file name and the EXIF information and move the picture to the right location – if it hasn’t already been copied!

So I needed some kind of database. Plain text files would be too cumbersome to maintain, SQLite would require me to design a database schema, and anything more complex was out of the question in the first place. That day TinyDB was born. It was really just a fancy Python dict with a nice query syntax and a JSON storage – prototyping the first version didn’t take much longer than a day.

Not much later I polished up the code, wrote some documentation and released it on GitHub and PyPI and almost forgot about it.

But against my expectations, people started using it.

The present

Fast forward to today. As I’m writing these lines, TinyDB has more than 700 stars on GitHub. The Python Package Index states 2321 downloads in the last month (altough that has to be taken with a fair grain of salt1). These are huge numbers for something that started as a quick side project. And has grown to a serious Python library. Numerous bugs have been found and fixed. We had a major redesign cleaning up old cruft. Taking all of that into account, I think TinyDB is more or less feature-complete. But what’s up in the future then?

The future

There are two things I’d like to see in the future: a TinyDB ecosystem and a TinyDB community. There already are some extensions, mainly written by @eugene-eeo and me. But there is room for more. I’d like to see more 3rd party storage engines, useful middlewares and helpful table classes.

Regarding a TinyDB community, the only place for discussion right now are the GitHub issues. There is no proper place to discuss future developments, to ask questions about TinyDB’s usage or to show off new extensions or cool projects that use TinyDB. To that end I’m trying an experiment: I’ve set up a forum which from now on shall act as the central place to discuss everything related to TinyDB.