In this talk at the 2013 JVM Language Summit, Oracle's
Aleksey Shipilev talks benchmarking, that necessary evil. Shipilev addresses the basic questions, like what is a benchmark? He then explains microbenchmarking, and how software performance engineering is different from software engineering. Shipilev explains how software performance engineering is more of a natural science, with experiments, whereas vanilla software engineering is more of a formal science. See the video of the presentation and slides below.