I have an alternative use-case for Cayley that I would like to see adopted into the main project. I’m using Cayley entirely in the browser by transpiling to JS using GopherJS, storing the graph in “memstore”. It works surprisingly well, with my largest graph so far having 325k quads and using 1.85Gb of memory (including browser overhead) on Safari. The generated minified JS for the app I’m developing is 2.8Mb (482Kb gzipped). To get it that “small” I had to remove the “net” dependency fro…

This is interesting from multiple points of view:

We can run a Cayley engine that works entirely in browser, and even offline.

The data can be stored in memory, Local Storage, Indexed Storage (local) or PouchDB (local+remote).

Data on the client can be kept in (readonly) sync with Cayley server by streaming replication log back to the client.

And all we need to add the initial support is to add a build script, and maybe switch from current UUID lib to allow the code to compile without “net” dependency.

Then we can try to implement one JS-specific backend using KV abstraction (it’s the easiest way right now) and store the data in Local Storage, for example.

@elliott5 Can you please share the build steps needed for building and minification of the code?

Oh, that’s nice, I expected something more involved… Like installing tools from npm, or something

OK, I’ll try to compile it, and if everything goes well, we’ll need to figure out what API we want to expose from it and make a separate file under ./cmd/ that will expose this API to JS. After that we will include it in our build process. I’m also working on a new Cayley UI, and it might be interesting to include this API implementation to it.

The only issue I can see is that PouchDB does not support transactions.

The alternative would be to use the JS IndexedDB API directly, accepting that support is poor in some browsers. Indeed non-existent for v2.0 in IE/Edge. Using PouchDB has the advantage of choosing which browser database option is likely to work best.

@elliott5 I just checked Kivik, and it seems like it will fit a nosql API naturally. The subset of operations needed to make it work should be reasonably small, and it will be great if you could make a prototype for CouchDB/PouchDB.

In worst case, we can always fallback to KV API and/or IndexedDB.

Also, it will be great to hear some feedback on NoSQL API - if there is a set of operations that is hard to implement in any of these DBs with current design, it’s a good time to change API for better compatibility. I’m currently working on this part to be able to include ElasticSearch backend as well.

Transactions are a good point, but they are also weak in Mongo. In most case we only need an atomic increment on one document field to make node ref count work. Multi-quad transactions might work properly in Mongo, but if any of JS DBs can properly implement it - I will expose hooks for it.

This might not work well for DocIterator - it might be passed around and should use the context of the last caller, not the first one. I would say it make sense we can add contexts to Insert, FindByKey and EnsureIndexes and leave other builders as-is, since they already pass context in Do, One, Next, etc.