It's a girl

For the past few months, we have been doing a lot of work on performance for Voron. Performance for a storage engine is a very complex problem, composed of many competing issues.

For example, sequential writes vs. random writes. Or the cost of writes vs. the cost of reads. In this post, I am going to focus specifically on the single scenario we have focused most of our efforts on, sequential writes.

Sequential writes are important because they present the absolute sweet spot for a storage engine. In other words, this is the very first thing that needs to be fast, because if that ain’t fast ,nothing else would be. And because this represent the absolute sweat spot, it is perfectly fine and very common to explicitly design your system to ensure that this is what you’ll use.

We have been doing a lot there. In particular, some of our changes included:

That would be around 4 pages per tx, at 4521 tx/s. So roughly 70 MB/s in data, excluding journal writes. Which would amount to more than 140 MB/s disk transfer rate if the journal pages are uncompressed, or somewhere between 80-110 MB/s if the journal pages are compressed.

Ayende, I think what Matt meant that just based on the graphs it is not clear how much (or even if) Voron has improved. Because first graph shows one test suite with one version of Voron, and the other shows a different test suite with a different version of Voron.

The only thing the graphs show is that ESENT is ~2x faster on the new test suite than the old... and I don't think anyone cares about that. What this series of blog posts is about is improving Voron and we all want to see how much faster Voron is now vs. Voron in the past. The graph as they are now, conceal this information, very odd to see.

Alexei,
I understand now, and it make sense, but we don't care very much about comparison to the past version.
Sure, it would be nice to do a comparison, but It would mean doing some non trivial work to run the old version of the test against the new version of the code, etc.
What I care about it comparing to Esent.