Lobsters recently had posts on flatbuffers and SBE. Less recently, we also had a post on cap'n proto but it didn’t get any love.

On Kenton Varda’s point about SBE probably being a little faster by removing random access and variable-length integers, he pointed out that a lot of Martin Thompson’s focus with SBE was removing branch mispredicts on an HN thread, and that’s probably where the speed difference will come from.

I’m currently working on a pure Java implementation, and I’m in fact writing the tests in Scala, which seems to work quite well.

For Scala code that uses capnproto-java, the main non-idiomatic thing I’ve hit is that unions don’t get to take advantage of sum types and pattern matching. I’m by no means a Scala expert, though, so I’d be interested to hear about any other potential impedance mismatches.

How does building up messages work? For idiomatic Scala I’d expect an immutable COW style, which gives you an awkward choice; either instantiating an object instantiates the serialized version (which means serializing copies of all the partial instances) or you only create the serialized version once everything’s been built (which means an extra copy stage). Maybe some kind of effect style is the right way to go, building up a monad that knows how to create the serialized instance and then running it to actually serialize? I can’t imagine you could do that in Java though.