What It Takes To Test A Product

This site may earn affiliate commissions from the links on this page. Terms of use.

In my checkered career, I’ve participated in or observed many discussions about product performance. Generally, the most passionate discussions revolve around PC components such as CPUs, core logic and graphics cards. Curiously, although products like printers get benchmarked (pages per minute, pages per ink cartridge), users don’t seem to get overly passionate about these numbers — though image quality certainly raises a few hackles.

Recently, some readers have been curious about our benchmarking and testing methodologies, so I thought it would be a good idea to share our process and our philosophies on product testing. If you’ve read a wide array of articles across the Internet and in publications, you may have noticed that different publications often get different results on what seems to be the same or similar sets of tests. We can’t speak for other sites, but thought it would be worthwhile to share how we test products, at least when it comes to CPUs, motherboards and whole systems.

So let’s talk about product testing and benchmarking, and let’s talk about my own personal bias. Yes, I have a bias, though it’s not towards a particular company or product. My bias is a systems bias. Some very smart people have great fun digging deep into a particular architecture — how does the cache perform? What’s the effect of a pipeline stall? What is the triangle throughput for a graphics card for zero pixel triangles or 25 pixel triangles? In fact, our very own Dave Salvator has to test like this, because of the very nature of what he’s testing — specific subsystems, like graphics and audio cards.

I spent part of my career — both as a technology writer and working in the industry — caring about that sort of minutiae. It’s important stuff, particularly if you’re an architect, or want to second guess what the architects have done. But over the years, as I’ve listened to and corresponded with users, I’ve become more and more a believer that it’s the results of the entire system that matter.

This site may earn affiliate commissions from the links on this page. Terms of use.

ExtremeTech Newsletter

Subscribe Today to get the latest ExtremeTech news delivered right to your inbox.

Email

This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our
Terms of Use and
Privacy Policy. You may unsubscribe from the newsletter at any time.