Performance Monitoring and Analysis

Version 9

Created by drummonds on Mar 26, 2008 1:12 PM. Last modified by jeanchen on Jan 5, 2011 10:43 PM.

Introduction

A common question that crosses my desk is, "how do I analyze and correct performance?" There are a variety of techniques for doing this and many sources of information spread around the interweb, so I'm going to collect a few thoughts here.

Monitoring

Guest-based performance monitoring is an inaccurate and unhelpful means of evaluating performance in virtual deployments. See Guest-based Performance Measurement for more inforamtion. Monitoring and analysis of VMware ESX Server should be performed with esxtop and VirtualCenter.

esxtop

esxtop is the tried-and-true means of collecting every performance stat needed and making it available in a way that is conducive to analysis. The best source of information on launching esxtop can be found in the Resource Management Guide (page 159). It's worth nothing that the directions in that guide came from ESX Server 3.0.1. Since then, the developers have been kind enough to simplify the process of including all performance counters. This is done with the "-a" switch. So,

esxtop -a -b > analysis.csv

runs esxtop in batch mode and prints all performance counters. Let me give you the quick "do-s" and "don't-s" of esxtop:

With esxtop on ESX Server 3.5 and newer, always include the "-a" option to display all counters.

With esxtop on versions of ESX Server prior to 3.5 always follow the resource management guide to enable all counters. The storage latency statistics, for instance, are not displayed by default!

Always start your VMs before running esxtop in batch mode. If you start them after starting "esxtop -b", then esxtop will only produce VM data based on the VMs that were running at the time of its start.

VirtualCenter

VirtualCenter doesn't provide all of the performance counters you might need to analyze performance. But, it provides more than you might think! The default setup for VirtualCenter performance counter collection is fairly minimal. But this can be expanded by reconfiguring VC's performance counter collection. This is done as follows: