Tests have limits, but the news seems (mostly) good for Samsung-made iPhone 6Ses.

Share this story

Many consumer electronics companies, Apple included, use parts sourced from multiple manufacturers to meet demand for their products. You can make and sell more iPhones if you're buying screens from two companies, and that also insulates you from risk if there's something wrong with one company's components. Parts sourced from multiple manufacturers will inevitably behave a little differently, but, as long as the differences are small, people aren't really going to notice or care.

This is the kind of manufacturing nitty-gritty that doesn't usually make headlines, but, in the case of the iPhone 6S, people have suddenly become concerned. Apple is sourcing an important system component, the Apple A9 system-on-a-chip, from both Samsung and Taiwan Semiconductor (TSMC). This is something that it normally doesn't do, since manufacturing processes from different chipmakers can have different performance and power consumption even when the chip's design is identical.

And that's just what some iPhone 6S and 6S Plus buyers have run into. Using an app that has since been pulled from the App Store, some users were able to determine which chip individual iPhones were using and found that the phones with Samsung chips had significantly lower battery life than the phones with TSMC chips in certain tests. The findings got enough attention that Apple offered a rare comment on the situation, claiming that the test being used wasn't representative of actual use and that in "real-world usage" the difference between iPhone models with any combination of components was no more than 2 to 3 percent.

We just happen to have two iPhone 6S models here, one with a Samsung chip and one with a TSMC chip, and we've been running a variety of battery life tests for the last few days to see if we could find any differences. Our results lined up pretty much exactly with the statement Apple provided, but we've laid out all of the details below.

The tests and results

As mentioned, we're using two brand-new iPhone 6S models locked to AT&T but with the SIM cards removed (we didn't have two active AT&T SIMs, so this was the easiest way to make sure both phones' cellular hardware was doing the same thing). The screen brightness in each handset was set to exactly 200 nits using a Spyder4Elite colorimeter, and the screen was left on at this brightness for the duration of all of our tests.

Now for the tests themselves. The battery life test that started this story off is the one included in Geekbench 3, which according to Primate Labs' John Poole does include some idle time but is generally very CPU intensive. According to the Xcode Activity Monitor instrument, the Geekbench 3 battery life test generally keeps the CPU pegged between 55 and 60 percent load for the entire time it runs, with occasional dips below 55 percent and peaks above 60 percent.

Update: To clarify exactly what Xcode's Activity Monitor is telling us, remember that every logical CPU core is tracked individually, so for a dual-core CPU like the A9 "full utilization" would be about 200 percent, 100 for each core. The Geekbench test is putting about 30 percent load on each core for a total of 60 percent. For comparison, the relatively light but modern iOS game Shooty Skies oscillates between 30 and 70 percent depending on how many objects are being drawn on screen.

Our WebGL battery life test similarly keeps the CPU (and the GPU) working continuously, but at a slightly lower level of load. CPU load for this test typically hovers between 45 and 50 percent, and the GPU Driver instrument says the GPU utilization is between 25 and 30 percent. We also ran the GFXBench GL 3.1 battery life test for good measure, which loops the "T-Rex" test 30 times while measuring performance and power drain. These two tests approximate the load that a 3D game might put on the A9.

Finally, most smartphone usage leaves the CPU and GPU idle for extended periods of time, which is where our Web browsing test comes in. It continuously loops some pages cached on one of our servers, loading one page every 15 seconds until the phone dies.

Each test was run at least twice to reduce the likelihood of outliers, and the results from both runs were averaged out to reach the numbers you see below (the Geekbench test was run three times).

There are two big takeaways from the results here. First, the Samsung phone did have consistently lower battery life results than the TSMC phone. The one exception was the WebGL test, in which the Samsung phone barely edged out the TSMC phone.

Second, even though that's true, the Geekbench test was the only test that caused what we would believe to be a significantdifference, one that we can definitely attribute to the SoC rather than the screen or the battery itself or some other system component. All three of the other tests showed the two phones scoring within two to three percent of each other, which just happens to be the same figure Apple quoted to the press last week. The heavier Geekbench test, on the other hand, showed the TSMC phone lasting an average of 28 percent longer than the Samsung phone.

So there are definitely circumstances under which the TSMC phone will last longer than the Samsung phone, but it's not a universal problem. A Samsung chip that's mostly idling or even one under modest CPU and GPU load, though, is going to behave in just about the same way as a TSMC chip. And the kinds of CPU-intensive work that the Samsung chip seems to struggle with just aren't that common on smartphones. Most of the time, iPhone 6S battery life should be similar no matter which chip your phone is using.

Limitations of the tests

It should be noted that no matter how many variables we try to control for, any testing of a sample size this small can only tell us so much about a problem that's this complex. Our two iPhones have different A9 SoCs, but we don't know what else differs between them—Apple uses multiple sources for many components, including displays, RAM and NAND, and even things like the Taptic Engine.

Others with Samsung and TSMC phones to compare have also found differences that largely echo our findings, which makes us inclined to believe that there is a difference between the two, but even if you add up every test from Reddit and Apple fansite forums and the Geekbench database you still get just a fraction of the 13-million-plus iPhone 6S and 6S Plus models that are already out there in the wild.

Apple is the only entity that can even come close to providing a sample large enough to be representative, and while it says that the differences between all models (not just Samsung-vs-TSMC, but all phones with all components) average out to 2 or 3 percent in "real-world usage," though the company wouldn't tell us how it makes that determination or what its internal battery life testing procedures are like.

So, we think our data is as good as it can be given our sample size, and our findings seem to confirm what both independent testing and Apple's own numbers are saying. But this also isn't the last word on the whole Samsung-vs-TSMC situation, and our understanding of the problem will continue to change as more people run more tests and as more chips and phones are manufactured.

We'll continue to keep an eye on this problem as the manufacturing processes for the phones and the chips themselves become more mature, and Apple and its suppliers have time to work out wrinkles. For what it's worth, the iPhone 6S with the Samsung chip is the one that I'll be using every day, and I'm heartened enough by these findings that I'm not worried about returning it so I can get a TSMC phone.

Share this story

Andrew Cunningham
Andrew wrote and edited tech news and reviews at Ars Technica from 2012 to 2017, where he still occasionally freelances; he is currently a lead editor at Wirecutter. He also records a weekly book podcast called Overdue. Twitter@AndrewWrites