Our CTO, Heikki Nousiainen followed with a presentation on benchmarking raw Kafka performance across four different clouds to demonstrate its capabilities.

While we weren’t able to get a copy of Rovio’s presentation deck, what follows is a summary of Rovio’s presentation. We’ve also provided a summary of ours as well with a copy of the slide deck at the end of this post.

Let’s jump in.

A tale of two Kafkas

Heiskanen kicked the meetup off with his presentation, Story of Two Kafkas, a reference to their ongoing transition to a newer pipeline. A user of Kafka since version 0.7, Rovio’s daily load is impressive:

3 billion events,

1 Terabyte of data

Without getting too deep into their architecture, suffice it to say that their Kafka node count is high. In spite of its size, when asked how big a team should be to manage Kafka, he responded,

“One really good Kafka guy on development side who is available 24 hours.”

His wink and a nod response provides insight into the difference between setting up Kafka and running it in a production cluster, as one of his slides summarized;

Setting up Kafka is simple, but for running a production cluster, you need monitoring, orchestration, and capable people.

That said, Rovio’s choice to go with Kafka is obvious:

Kafka’s the best tool available and doesn’t pose vendor lock-in, unlike AWS’s Kinesis and Google’s PubSub.

Additionally, there are a lot of good Kafka tools and connectivity libraries available, as he quipped, “If you feel you need to implement something yourself, you’ve not googled enough.”