Microdatabase Project Idea

This is a project plan that could be followed
to create a microdatabase that would be interesting
for beginners.

Basic Design

The client API would have three parts:

A constructor for an object
associating the database with a json file on
the server. Doesn't do much, just remembers
the URL.

mydb = new JsonDb('mydb.json')

A simple key-value store that saves its
values textually in the JSON file. (The
methods should return Promises, which in the future
will work with an await an extension to
CoffeeScript. See the
await syntax project.)

The Javascript string would just be a function
definition written in standard Javascript,
the same thing you get when you use fn.toString()
on a function in Javascript.

Phase 1: build a nodejs server for get/set

In this phase, we would build a basic node.js
server that uses Express and listens to the
/data URL, implementing the
get and set API.

When setting data for the first time,
The request file should be created as a
UTF-8 text file and initialized with
an empty JSON object document '{}'.
Properties should be able to be added
and queried via the server.

Phase 2: implement a client library and multiple users

In this phase, we would implement multiple
users, using the convention that the username
is the first component of the domain name in
the Host header,
and when you request "http://david.pencilcode.net/data/myfile.json", it saves the data
in "david/myfile.json".

And we would also implement a client library
and create some tests.

Phase 3: link a javascript-in-javascript
interpreter.

To safely execute Javascript transactions,
we must prevent them from running too long or
doing dangerous things. So we want to
interpret transactions inside a Javascript
interpreter within Javascript.

So the second phase is to embed an interprter
like this one,
and use it to implement transactions.

In this phase, we would add tests
to verify that basic things can be done
with the interpreter, but that dangerous
things, such as accessing node.js globals,
cannot be done.

Phase 4: implement resource limits

To make it safe to provide the database
for free to all students, it must automatically
limit each database to a small amount of
storage (say, 1mb), and a small amount
of CPU per transaction.

Exceeding these limits should return
an clear error.

In this phase, we would add tests
of the resource limits.

Phase 5: implement an in-memory cache

If several queries or transactions come
in for the same file, the file shouldn't be
re-loaded from disk each time. Instead,
it should be retained in RAM for a few minutes.

After a short time without access,
databases should not be kept in RAM.

Phase 6: implement passwords and "stored procedures"

In this phase, we would add a couple
methods to allow a database to store
specific allowable transactions, and
limit access so that users need to
provide a "secret password" to use a
particular transaction.