The Highly Variable Network Performance of Amazon EC2

We recently moved a bunch of our app server nodes from Rackspace to Amazon EC2. Since then, we have noticed highly variable amounts of network performance. We monitor DripStat using DripStat itself, so we can show you exactly how this looks like.

Architecture Overview

To give an idea of how things work on our backend:

Every JVM that is connected to DripStat, pings our ‘Data Collector’ servers every minute.

The data collectors verify the data and perform some light processing on it.

That data is then put on a Kafka queue to be indexed and queried.

Our Kafka nodes are located on Softlayer in the Washington region.
The EC2 nodes are located in US-east region.