Eric Lippert's blog

Main menu

Post navigation

Benchmarking mistakes, part one

The good people over at Tech.Pro have asked me to write a short series of articles for their site on the subject of mistakes I’ve made and seen others make when writing benchmark performance tests of C# code. The series will be pitched at the beginner level.

Wow – I never knew about `Process.TotalProcessorTime`. Seems like for performing benchmarks this might be ideal – especially in the world of power saving devices (like laptops) where we might be getting benchmarks clouded by power optimizations without necessarily realizing it. Interesting about the stopwatch being unavailable in some cases – I was definitely not aware of this.

Oh yes, that StopWatch thing!
I’ve once wasted some hours looking for an assumed bug, but in the end it was StopWatch fault.
On our development and test servers all was well, but on production it looked like our Cache needs 12 seconds (sometimes only) instead of a few milliseconds.
The solution was found as the StopWatch was compared to DateTime.Now – there simply was no slow cache, but the StopWatch was simply insane.
Never had expected this :-(

I strongly suspect that it’s a non-issue by now if you have modern hardware and/or an operating system newer than Windows XP. Everything I’ve read on the subject leads me to believe that QueryPerformanceCounter and QueryPerformanceFrequency work correctly in the face of variable clock rates and multiple CPUs. If Stopwatch is implemented in terms of those two functions, then this “problem” has been dead for five or more years.

Here the problem with Stopwatch occured on Windows 2003 Server (fully patched, serverworks board with an Opteron).
Another problem are virtual machines on W2008R2, where the clock is slower than reality – I found no good solution up to now. The Host time is fine all the time.