Samsung Edges Q'comm in New Android Test

SAN JOSE, Calif. — The Samsung Exynos has nudged ahead of Qualcomm's Snapdragon, according to initial tests of the latest SoCs with a new Android benchmark released today.

The AndEBench-Pro created by the Embedded Microprocessor Benchmark Consortium (EEMBC) shows Samsung's Exynos 5 Octa 5420 narrowly beating the Qualcomm Snapdragon 800 MSM8974AA v2 in overall device and 3D graphics performance. A similar Qualcomm part in a Google Nexus 5 system had a higher overall score than the Samsung part, but it ranked lower in the 3D performance category.

AndEBench-Pro is a suite of tests measuring the performance of CPU, GPU, memory, and storage subsystems, as well as a platform test that gauges GUI rendering, XML parsing, image manipulation, data compression, and cryptography tasks embedded in actual workloads. It was the product of an EEMBC workgroup led by Ronen Zohar, a principal engineer at Intel. Members represented most application processor vendors, including Imagination Technologies and Nvidia.

The suite marks a significant advance over the group's AndEBench v1, a CPU-only test released in 2012 and upgraded last year that uses just one workload. By contrast, the AndEBench-Pro suite uses 18 workloads just for its CPU tests.

Markus Levy, EEMBC's president, said the new benchmark is meant to be useful to both end users and device makers. It also has an implementation in Mandarin, since "a very, very large majority of users are in China," he said. "Unfortunately, Android benchmarks in the market have been developed by entities which might be naïve about how to develop a structurally sound benchmark, and furthermore their motivation is unclear. Given the expertise that has gone into developing AndEBench-Pro and the transparent availability of source code, EEMBC is convinced the benchmark is structurally sound."

More than 5,600 users submitted scores using the original AndEBench on more than 1,000 devices. The benchmark remains available on the EEMBC website, but those scores are not directly comparable with tests using the new AndEBench-Pro.

AndEBench-Pro can be downloaded for free from Google Play and the Amazon Appstore for Android. The app is free for personal, noncommercial uses, but an EEMBC license is required for any commercial uses.

Professional device reviewers should contact the EEMBC directly to obtain a version of AndEBench-Pro that will enable them to change benchmarking parameters and gain undisclosed scoring information to aid in their device analysis. The EEMBC plans future versions of the benchmarks, and it encourages anyone interested to join the consortium and working group developing them.

The EEMBC lays out in a technical document details of the test suit. For example, it notes the CPU test uses Base and Peak CoreMark-HPC Scores.

For the Base mode, the EEMBC built workloads using the default compiler included with the official Google Native Development Kit. In Peak mode, the workloads reflect the best performance that can be expected from the CPU, using compilers and compiler settings optimized for each CPU.

The 3D test generates a one-minute game sequence as an exercise routine and then measures the number of frames generated per second. It performs all rendering off screen to a 1,080-progressive pixel display. After the rendering, the frames are scaled and displayed on the actual device's screen. Higher-performing GPUs will be able to generate more frames and thus achieve higher scores, according to the EEBMC document.

The AndEBench-Pro memory bandwidth test uses several serial access patterns based on the Stream benchmark to test memory read and write performance. The test is run once in single-threaded execution and once in multi-threaded mode. The two results are aggregated using a geometric mean and reported in MBytes/second.

The memory latency test initializes a 64M block of memory as a linked list with the elements of the list arranged in random order inside the allocated block. The test measures the time it takes to traverse N nodes in the linked list. Each step of the traversal requires fetching the next node pointer from memory, thus testing the memory subsystem's latency. The score is reported in number of nodes accessed per second, so that higher scores reflect lower memory latency.

The Samsung Exynos has nudged ahead of Qualcomm's Snapdragon, according to initial tests of the latest SoCs with a new Android benchmark released today.

Interesting to know that Samsung Exynos has nudged ahead of Qualcomm's Snapdragon. I am curious to know if these are the final numbers or initial numbers because it is mentioned as initial tests in the article ?

Based on the screenshot, "device score" looks like an overall score that actually includes 3D, and if it is to be of any use, this overall score should be taken by itself. Therefore, the older (?) snapdragon is better than the exynos, which is better than the newer snapdragon.

This basically means that either qualcomm are incapable of improving upon themselves, or that this benchmark's overall score is useless: I'm betting on the overall score being useless. better look each individual test score and get an idea of what type of computation each of these devices is better at than rely on an unproven overall score