The machines are measured using something called the Linpack benchmark, which is a "dense system of linear equations."

But critics have said Linpack is too limiting and doesn't reflect real-world performance. Alessandro Curioni, head of the computational sciences department at IBM's Zurich research lab, tells the BBC, "we need a more practical measurement that reflects the real use of these supercomputers based on their most important applications."

"One change coming with the new benchmark: It's likely to throw some cold water on a recent trend, the addition of special-purpose 'accelerator' processors such as graphics chips that are good for only a subset of supercomputing chores.

The HPCG test has entered beta testing and will be adjusted in the coming years, said Jack Dongarra, a University of Tennessee researcher, the author of Linpack, and one of the organizers of the Top500 list. Linpack won't be removed, but it'll have to share the limelight with HPCG, he said.

" 'It will be a while before we have enough data to populate the Top500 list, but that is the intention,' Dongarra said. 'Keep in mind that the Linpack Benchmark took many years before it was widely accepted. We hope this new benchmark reflects modern applications better than Linpack.' "

It's not immediately clear how the change in the benchmark will affect the rankings, but we should know in about six months when the new list comes out.