4 Responses to “Performance Comparisons (a Helper Class)”

you put very much thought into how to make the measurement accurate. you increased the thread priority, collected garbage and did a sleep(0) between runs. very clever. here are 3 additional ideas a had:

you could do 10 runs of the test and discard the 3 slowest runs. this would discard cpu-hickups, disk activity and cosmic rays. i would not discard the fastest runs.

the third idea might be a little crazy but maybe it is useful to generate the test runner with reflection emit or codedom to eliminate the delegate overhead. this would however necessitate that the method to run is passed as a methodinfo.

>> Process.GetCurrentProcess().ProcessorAffinity = new IntPtr(1); //i dont think this increases accuracy
Well – my intention behind this was that (on multicore CPUs) the OS might move the thread to another core. It may be useless – but I think that it does not do any harm🙂

>> you could do 10 runs of the test and discard the 3 slowest runs. this would discard cpu-hickups, disk activity and cosmic rays. i would not discard the fastest runs.
I am not discarding the fastest runs. As said in the article you should always look at the fastest test run. So I see no advantage in ignoring any test runs.

>> the third idea might be a little crazy but maybe it is useful to generate the test runner with reflection emit or codedom to eliminate the delegate overhead.
Hmm – sounds interesting but also a little bit oversized. In my opinion, this overhead is no problem as it occurs in all tests. So all results get a little bit “worse”. The absolute values usually do not express much; more important is the comparsion between them.