Here are the new benchmark tests we will be using in our reviews

Synthetic benchmarks should be approached with care, because putting too much trust in those numbers may result in unnecessarily high (or low) expectations of a device's performance. Indeed, it is real-world usage that counts, but still, benchmarking shouldn't be ignored altogether. After all, there are some tests that can actually give us a pretty good picture about certain aspects of a phone or a tablet.

We think you know where this is going. Ladies and gents, allow us to introduce our new benchmarks! Let us elaborate: we've come up with a number of somewhat more scientific tests that we'll use to study the devices' displays, system performance, camera speed, audio output and... that's right - battery life! The results will be easily visible in the respective device's review. Here's a glimpse of how the new benchmarks are going to look!

When it comes to displays, we'll be providing you with quite a lot of information. For example, you'll be able to see a display's maximum and minimum brightness, contrast, color temperature, Delta E, color gamut and more. We've tried to make it all easy to use and understand.

The CIE 1931 xy color gamut chart represents the set (area) of colors that a display can reproduce, with the sRGB colorspace (the highlighted triangle) serving as reference. The chart also provides a visual representation of a display's color accuracy. The small squares across the boundaries of the triangle are the reference points for the various colors, while the small dots are the actual measurements. Ideally, each dot should be positioned on top of its respective square. The 'x: CIE31' and 'y: CIE31' values in the table below the chart indicate the position of each measurement on the chart. 'Y' shows the luminance (in nits) of each measured color, while 'Target Y' is the desired luminance level for that color. Finally, 'ΔE 2000' is the Delta E value of the measured color. Delta E values of below 2 are ideal.

The Color accuracy chart gives an idea of how close a display's measured colors are to their referential values. The first line holds the measured (actual) colors, while the second line holds the reference (target) colors. The closer the actual colors are to the target ones, the better.

The Grayscale accuracy chart shows whether a display has a correct white balance (balance between red, green and blue) across different levels of grey (from dark to bright). The closer the Actual colors are to the Target ones, the better.

Up next, we'll continue to check performance of
Android phones by running some of the most popular performance
benchmarks out there, but we'll be displaying the results in a new, more
appealing type of visualization.

We'll be testing camera speeds
in a couple of ways. The first one is our custom "Taking a pic" test,
which will measure the time needed for a camera to launch, focus, take a
picture, and get ready to take the next one. This test will also have a
separate result for HDR photo-taking. We'll also post the scores we've
got using the cross-platform Camspeed benchmarking app.

Moving on to the audio output,
there are two things we're going to measure in this regard. The first
is the output power of the 3.5 mm headphone jack (in volts), and the
second one is the loudspeaker's loudness (in decibels).

Audio output

Finally, we've created a custom battery life
benchmark to test phones with. It's a special script, designed to
replicate real-life usage. The results for each handset or tablet will
be provided in hours and minutes.

Battery life

We measure battery life by running a custom web-script, designed to replicate the power consumption of typical real-life usage. All devices that go through the test have their displays set at 200-nit brightness.

You are correct, but Reviewers should understand that you can ONLY rate quantitative characteristics such as display measurements, sound loudness... etc... like what they just update here

And things like "how pretty" or "we like this skin better" is complete subjective opinion, which is finally up to each user or buyer, and should NOT be rated... or at least it should follow certain parameters

So yeah this should help lessen both the bias in reviews and the hating in the comments

But some aspects of a phone's appeal are subjective and can't be captured by a benchmark. Aesthetic is one. The "feel" of a handset is another. The quality of the outbound and inbound voice signal isn't really entirely subjective, but there are so many variables involved that it might as well be. The verdict on such subjective criteria tends to evolve from the aggregate. But still, your point is taken. Having hard data on the real capabilities removes much of the subjectivity.

I wish, but fanboys are fanboys. This is only one aspect of a review, and while the benchmarks are going to be cleaner and interactive, there is still so much more making up a review. But, we can hope it helps. ;-)

Very nice. But just as a phones real test is how it works in real life, the real test of these imaginative benchmarks will be how well they parallel our actual experiences. My take on benchmarks and companies designing their device to "cheat" them is that if your benchmarks measure actual benefits to the end-user, then cheating them simply makes your device better for the end user. Cheat all you want.

There's something like this on another popular review site, but I like this one better because you can add phones (so you can compare easier). Pretty neat.

One suggestion I can make is to use a theoretical limit for a pixel that is off (Display measurements and quality). You can use 0.000001 nits for all AMOLED screens so that we can have a figure to compare with. It's not the real result yes, but will give us an idea of how AMOLED screen from one phone can compare to an AMOLED screen of another.

the only problem is that they don't seem credible. look at the battery numbers. sony phones get good scores in comparison to others that are known for having better battery longevity. i don't think that's accurate.

All content (phone reviews, news, specs, info), design and layouts are Copyright 2001-2016 phoneArena.com. All rights reserved. Reproduction in whole or in part or in any form or medium without written permission is prohibited! Privacy . Terms of use . Cookies . Team