Measuring CPU performance cross platform

Page 1: Benchmarking

<b>My views on this whole CPU naming schema</b>.

Many people, and companies display concern over the ratings of processors and how fast they are. Just a few years ago, you could easily determine the relative speed of a CPU by just looking at the MHz rating. Those days are long gone.

Remember in 2000 when the Pentium 4 hit the market, everyone was ranting and raving about its performance. Some people said it was stellar and some said it took a step behind of the Pentium 3. Wether Intel made the Pentium 4 just for pure clock speed is mere speculation, but I remember reading that the engineers wanted a stronger FPU (Floating point unit). The FPU control the number crunching aspect of the CPU and as we have seen with the Athlon, a stronger FPU means a faster CPU. The Pentium 4 must rely on SSE2 enhancements, to a point, to regain the performance of the lost FPU power. But the Athlon doesn't need any of those fancy enhancements and was just as fast as Pentium 4s at much lower clock speeds. AMD knew it wouldn't gain a fair amount of market share by throwing a 1.4Ghz Athlon against a 1.6Ghz Pentium 4. What they did was make a naming scheme of how much performance would compare to the Pentium 4 CPU, I believe it is called Giggahertz. What AMD is doing is not bad at all, trying to compare its performance to its competitors and allowing customers decide which they want.

While I'm all for competitive natures of these companies, AMD and Intel alike need something different that pure Ghz ratings. That is just to old now. What can be done? Create an industry standard way of measuring processor performance, not just by pure MHz, but by the entire processor, including the FPU. If speed is measured by Instructions per second (Strong on AMD) times Clock speed (Strong on Intel side), then it should be easy to determine this by a benchmark. Sure a benchmark can be easily used, but there comes in some problems. Modern processors have extra instructions that allow it to perform faster than it would otherwise. This includes SSE, SSE2, 3DNow, and 3DNow Professional. What do you benchmark with? I don't see how you can benchmark a CPU with those instructions available to it, but knowing that not all programs will be able to use those instructions. Let's Say I have my Pentium 4 1.6a Ghz and I benchmark it with SSE2 enhancements, and I take an Athlon and benchmark it with 3DNow enhancements. How can you compare those results? You can't, and as CPUs begin to become more complex, this benchmarking system must be improved. What happens if you compare an Alpha to a x86 chip? How do you benchmark that? In the future, we all will have 64 bit CPUs, but in the transition time, how will we know what is a better buy when we want to purchase a what I call 'morphing CPU'? How do you benchmark an x86-64 chip? Do you do the 64bit and 32bit aspects? And what will use those 64bit instructions?

So you see, you can't just saying CPU x is as good as CPU y because it performs the same in program z. I'm not trying to bash AMD or Intel, but just relay some thoughts on this whole CPU thing. <a href="mailto:aronschatz@aselabs.com">Flames can be sent here</a>