Programmer and Compiler Optimization

Martin Fowler: Another thing you have to bear in mind is that performance optimizations are quite
implementation and release dependent. When you get a new version of Java, you really
ought to undo every performance optimization you ever made, then reapply them to make
sure they still make the program go faster. Often you'll find that a performance
optimization you did in a previous version of the virtual machine (VM) or optimizing
compiler will actually make you slower now. A performance optimization that helped
before may actually defeat the optimizations the new compiler or VM is doing.

Bill Venners: It's not easy to remember what mutations you made to
improve performance in the past.

Martin Fowler: Undoing and reapplying optimizations is really what
you ought to do. I know it's not easy. It argues for keeping careful track of what changes
you make for optimization. The thing to remember is the performance hit of old
optimizations can sometimes be a big deal.

Craig Larman told me a story I still love. Craig gave a talk on performance optimization
at JavaOne, and he mentioned two well-known techniques: object and thread pooling.
Object pooling is reusing the same objects rather than creating new objects. Thread
pooling is basically the same thing with threads. After the talk, two guys came up to him,
both of which were designers of high performance VMs. One VM was Hotspot, and I think the
other VM was JRocket.
The one guy said, thread pooling works well, but you don't want
to do object pooling on our VM because it will actually slow it down. And the other guy
said the reverse. You can use object pooling, but not thread pooling, because it will slow
us down.

So you could make a performance optimization in one VM, and then bring in Hotspot,
and it will actually slow Hotspot down. You've got to be very wary of that. Object
pooling is a good example. A lot of people are very enamored with object pooling, yet
half the time people are not measuring to that to find out whether object pooling is any
good. Object pooling was very important in the early days of Java, because garbage
collection wasn't terribly good. When you've got generational garbage collection, object
pooling becomes a lot less effective, because short-lived objects can be collected very
cheaply. It's the long-lived objects, such as ones you might pool, that are expensive to
garbage collect.

So the rules keep changing. That's why you've got to be very careful to profile. If you
think you can predict from the source code what the machine is doing, you've got no
chance. When you're in a world of optimizing compilers and VMs, you have to profile,
because the compilers and VMs are doing things that you can't even imagine. So don't
predict, just measure.