Large Scale Distributed Systems

Announcing octane

December 21, 2016

Performance has always been one of my interests and I’ve been wanting for a long time to write a high performance networking library to use for when high throughput and low latency is a priority. I really like libuv however by default it only offers N:1 I/O threads to event loop threads. This makes it hard to scale to modern many-core systems unless you deploy multiple processes per system. I want something I can scale to the physical CPU cores available in a single process.

Today I’m announcing Octane which extends libuv to provide this scalability. I’ve chosen to add octane to the Techempower plaintext HTTP benchmarks so that it can be measured against the worlds fastest HTTP servers in an independent benchmark. My PR should be completed and merged in time for round 14.

Ahead of round 14 I wanted to post my own benchmarks so I can give a bit of an idea of the kind of performance octane offers.

Benchmark hardware

Both client and server machines have identical hardware.

20 Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz physical cores.

128GB of memory.

20GbE network.

Throughput results

In round 13 libreactor was the clear winner of the plaintext benchmark so I benchmarked against it.

Figure 1. Throughput requests per second.

libreactor

While libreactor is fast the CPU utilization was 75% idle during this benchmark even though network throughput was 3x less than octane can reach. This indicates to me libreactor is not scaling to use all the performance available on the machine and the network.