The following benchmarks are meant to give a ball park approximation of
the performance to be expected out of the box, without special tuning, on
commodity hardware.

Test conditions

Except where indicated, the benchmarks are run on an EC2 m1.large instance
with 7.5GB memory and two cores, running Ubuntu 12.04. The database is placed
on ephemeral storage, and the original filesystem is used as-is with no extra
tuning (this means EXT3 without any extra mount option). In practice, better
performance can be expected given some tuning and for instance a RAID10 setup
(the tests below are largely disk-bound).

A separate instance is used as the load generator.

Some statistics relative to the disk and overall performance of the instance
(redis random reads and writes) can be found at the end of this page.

obigstore is run with the following options:

-write-buffer-size 16_000_000: set the buffer size to ~16MB

-max-open-files 40000 to allow the benchmark to run (LevelDB's
documentation states that about around 1 file per 2MB is required).

Random hotspot writes

In this benchmark, 2^13 "hotspots" are chosen randomly amongst all 2^128
16-byte keys. Writes are performed in these hotspots as follows:

with probability 50%, a new key is added semi-sequentially

with probability 50%, a random key from this hotspot is overwritten.

The payload is a random 32-byte string with a ~50% redundancy.
Up to 64 concurrent operations are allowed; each performs 20 writes.

Fsync'ed writes

By default, obigstore performs group commits and always calls fsync() before
acknowledging each individual write to the client.

The dispersion observable in the insertion rate is due to obigstore's
throttling mechanism, which limits the insertion rate to avoid write stalls.
It would be possible to change the throttling algorithm to keep the rate
as constant as desired (i.e., disallowing bursts) as long as it is below the
sustainable one, as shown in another benchmark below.

The latencies are relatively high due to the large sustained insertion rate
and concurrency factor:

Non-fsync()'ed writes

It is possible to run obigstore in -no-fsync mode (which can cause the
loss of up to write-buffer-size data in the worst case, but not data
corruption since LevelDB always fsyncs when it moves to a new file). This
allows to assess the cost of fsync()ing:

Sequential insertion

In this benchmark, 200 million sequential 16-byte keys with random 32-byte
payloads (50% redundancy) are written. The load is generated with

which represents a ~75% CPU usage --- the task is clearly IO-bound. Note
that obigstore uses three threads when writing: one for request handling, one
to perform the actual write and another for compaction.

Latency measurements are performed with a concurrency level of 50 (2-writes
per request), and a rate approaching 15000 writes/s:

Reads

Range reads

1e8 pairs with 16-byte sequential keys and random 32-byte (50% redundancy) are
inserted in a database. 512-key ranges are requested with bm_read -range.
Ranges are read at a speed exceeding

295000 columns/s

Random reads

1e9 pairs with 16-byte sequential keys and random 32-byte (50% redundancy) are
inserted in a database. bm_read performs lookups for keys distributed randomly
in ranges of increasing size, using up to 5 concurrent requests each of them
performing 20 lookups (this corresponds to client-side joins). obigstore is
run with the -assume-page-fault option, which instructs it to run each read
request in a separate thread so as to avoid blocking reads.

The lookup rate as a function of the working set size (in keys) is shown below:

Three terms contribute to the lookup time:

request processing overhead

data retrieval from FS buffers or disk

block decompression

(3) is determined by the block size (-block-size), as larger blocks entail
higher latency. LevelDB (and thus obigstore) uses snappy to compress
data block-wise, and decompression speeds typically exceed several hundred
MB/s. The decrease in speed shown in the above graph between 100000 and 1e6
keys is a consequence of the working set no longer fitting in LevelDB's block
cache (set to 8MB by default and unchanged).

The lookup rate falls abruptly when the working set no longer fits in memory,
at which point it quickly becomes seek-bound and (2) dominates.