Performance Monitoring Tools

Before we get to tuning our cluster, it helps to describe what we might actually want
to monitor as well as any potential pain points.

Prior to switching over to
the new Storm Kafka client, which leverages the new Kafka consumer API under the hood,
offsets were stored in ZooKeeper. While the broker hosts are still stored in ZooKeeper,
this is no longer true for the offsets which are now stored in Kafka itself. This is a
configurable option, and you may switch back to ZooKeeper if you choose, but Metron is
currently using the new defaults. With this in mind, there are some useful tools that come
with Storm and Kafka that we can use to monitor our topologies.

ToolingYou can use the Storm and Kafka tools to monitor your topologies.