Bay Trail Performance Preview: Android and Windows

For many years, Intel has struggled to be a competitive force in the mobile industry. They?ve tried with platforms like Moorestown, Medfield and Clover Trail. Each and every time, their SoCs were either too power hungry or their performance simply wasn?t up to par. And each and every time, Intel seemed to learn from their mistakes and improve upon their Atom-based SoCs. And now that we?re in 2013, we are finally seeing Intel?s first serious submission into the mobile market, Bay Trail.

Bay Trail in itself is a 22nm SoC based on the Silvermont CPU architecture and Intel?s own Intel HD graphics, a change for the company that has traditionally used 3rd party graphics IP in their SoCs. We have a much more detailed article about the architecture and different SKUs of Bay Trail here. In order to save ourselves and you some time, we?re going to cut out all of the architectural stuff since we?ve already covered it and will occasionally refer back to it when explaining some reasons for certain scores.

First and foremost, the opportunity we got to benchmark Bay Trail came to us thanks to Intel inviting us to their Bay Trail Editor?s Benchmarking Day. This event was organized and set up by Intel and they flew some press out for this event since it occurred a day before IDF 2013. Intel gave us the opportunity to benchmark both Android and Windows implementations of Bay Trail but forewarned us greatly that we would need to take into consideration that their Android implementation ?still needed work.?

The benchmarking day as a whole comprised of a briefing about benchmarks, followed by benchmarking, lunch, a tour of their experience lab and some Bay Trail gaming. This was intended to give us the best experience of how Bay Trail compares against other mobile SoCs. And from all of Intel?s marketing, Bay Trail is being squarely positioned as their tablet SoC to compete with Tegra 4, Snapdragon 800 and others. They will be bringing Merrifield later this year and earlier next year for smartphones, which should give Intel an opportunity to be competitive in the smartphone space, where they?ve been a non-factor.

Our goal with Bay Trail on Android is to see how it stacks up against the competitors. Since we?ve tested Qualcomm?s Snapdragon 800 on their development platform, it seems pretty fair to compare against Bay Trail?s own reference platform. Comparing both against Tegra 4 on SHIELD will also be interesting since SHIELD has active cooling and as a result, a higher thermal threshold than the other two.

First, we wanted to see how Bay Trail stacks up against other Android devices. For this, we?ve already done some past benchmarking on other review devices and at other benchmarking events, namely Qualcomm?s own reviewer benchmarking day. Now, for our testing we targeted some of our commonly used benchmarks for other reviews to test different performance factors for GUI, GPU, Whole System and other things. Intel expressed their desire for users to utilize whole-system benchmarks that test real user scenarios and experiences and personally, I felt like they were pushing Principled Technologies? benchmarks the hardest. Those benchmarks include the entire suite of XPRT benchmarks which include the BenchmarkXPRT as well as WebXPRT, TouchXPRT, HDXPRT and MobileXPRT. I have heard Intel talking about how important they considered these benchmarks, but during the short time that I had to benchmark Bay Trail (due to a whole multitude of issues) I did not get a chance to really test using these benchmarks. I would, however, like to do so in my actual review of a Bay Trail tablet when compared to other devices. Say, with an ASUS TF100 tablet. Nevertheless, they pushed this benchmark above all overs.

The first benchmark we ran was AnTuTu, this benchmark is essentially the most hated benchmark in the whole industry. Even though Intel touted that they are faster on AnTuTu in the past, they?ve since said that they don?t really know what?s going on inside of the benchmark or what they?re really testing for. Because of this, we actually won?t be including AnTuTu benchmark scores, but we just wanted to state that we did test it, and if people actually want to see it we?ll update the story with those benchmarks. Until then, we won?t bother.

Now, the next benchmark we attempted to run was Vellamo, which is a benchmark that was actually developed by Qualcomm internally and eventually released to the public. This benchmark was designed to help the company benchmark their own SoC?s performance against the competition in HTML5 as well as other performance applications. And as we?re seeing more HTML5 based apps rolling out, the importance of Vellamo gets ever greater. Furthermore, some of Qualcomm?s competitors have cried foul about this being a Qualcomm benchmark, but from my experience Qualcomm is not always the fastest as one would suspect. Unfortunately we could not get Vellamo to finish testing on Bay Trail as it would lock up midway through one of our HTML5 tests, which goes back to what Intel stated about the Android implementation of Bay Trail not being ready yet.

Next, we ran Basemark GUI, which is a Rightware Benchmark, designed to help users see how a certain SOC performs in Graphical User Interface scenarios. This is mostly a test of the GPU, but it also tests the smoothness of the experience and the overall frame rate. This benchmark has both an on-screen and off-screen test, and since most devices are limited to 60FPS, the off-screen test is more interesting than the on-screen test as a measure of overall GPU GUI performance.

As you can see from our testing, Bay Trail came in just a hair behind the Snapdragon 800 at 353.8 FPS while the Snapdragon came in at 364.3 FPS. This slight difference was great enough to be outside of a margin of error and to be considered a clear win for the Snapdragon 800, however, it does mean that Intel has effectively caught up to Qualcomm. Nvidia?s Tegra 4 based SHIELD actually beat both SoCs, but you have to consider that it is much less thermally constrained since it runs with active cooling, which means that it can afford to run at a higher clock. Due to this higher clock on the Tegra 4, it is able to outperform both SoCs. When you see our Clover Trail vs Bay Trail benchmarks, you?ll see why Intel hanging with Nvidia and Qualcomm is a big deal.

Next, we ran Basemark X, which is one of the most graphically intensive gaming benchmarks out there for mobile devices. In this benchmark, we didn?t expect high scores from any of the SoCs, so neither should you. Additionally, it runs on Unity 4.0 which is one of the most widely used game engines out there on mobile devices.

Here, you can see that the Bay Trail tablet actually didn?t perform the best on the on-screen test when compared to all of the other tablets. This is actually because the Bay Trail tablet is running at a 2560×1440 resolution while all the others are running at 1920×1080 or 1280×720. Since the on-screen test is actually testin
g the native resolution, the scores will vary the greatest but also gives a pretty good representation of the actual experience on that device. The Off-screen test is the test that actually tests the GPU?s performance and here we can see that it outperforms (10.3 FPS) the Adreno 320 in the Snapdragon 600 on the HTC One (7.4 FPS), but it still lags behind the Snapdragon 800 (16.1 FPS) and Tegra 4 (12.4 FPS) in the SHIELD. As a disclaimer again, the SHIELD will likely perform better than the rest of the devices primarily because of its greater thermal headroom due to being actively cooled.

In our cross-platform tests, we used 3DMark and Browsemark to test the different devices that we had for their user browser experiences and graphical performance.

In 3DMark, we were able to test a broader array of devices, however we ran into some issues with a new version of the Android 3DMark coming out the day of the benchmarking event. As a result, we could no longer run Ice Storm due to the frame rate being maxxed out in the new version, not giving us a score. As such, we were forced to run 3DMark Ice Storm Extreme and Unlimited. However, our Acer W510 with Clover Trail would not run the unlimited test and barely ran extreme.

As you can see from our benchmark results, in extreme the king of the hill was AMD?s Kabini with Radeon HD 8330 graphics (14249 pts.) followed closely by Qualcomm?s Snapdragon 800 SoC with the Adreno 330 (13523 pts). The Nvidia SHIELD with Tegra 4 (at 720P) ended up coming in with a score of 11,558 and the Bay Trail tablet with a score of 8354. This may make the Bay Trail tablet seem like it is the slowest of the bunch, until you look at the Snapdragon 800, which only scores 6330 at a much lower resolution, to boot. The interesting thing was seeing how the Clover Trail system performed, which was a measly 1610 points.

In order to get a more fair measurement, I also tested the Bay Trail tablet against the Tegra 4 that we had in our hands that day. We used the newly developed 3Dmark Ice Storm Unlimited test for this so that resolution and vsync would not be factors. I kind of wish the guys at Futuremark had released the Ice Storm Unlimited benchmark sooner so that we would be more prepared for Bay Trail benchmarking. Nevertheless, the Tegra 4 SHIELD scored 16434 points while the Bay Trail tablet scored 15,178 points, which indicates that the GPU on the Bay Trail tablet is incredibly powerful and is almost as fast as an actively cooled Tegra 4.

Then, for our final cross-platform benchmark, we tested Browsermark to see the overall browser performance, which we all know gets used a lot by mobile users. n our Browsermark testing, some of which we got thanks to Marko Chiapetta at Hot Hardware since we couldn?t get it to finish running on our Android device. Let?s just say that we had a LOT of fun trying to run lots of our benchmarks on Android with Bay Trail. Additionally, in our Browsermark testing we like to test both Chrome and default browser performance (device stock browser).

Looking at our scores, you can see that the Bay Trail tablet just barely missed the Snapdragon 800 MDP in terms of performance. Since the Snapdragon 800 MDP didn?t have a stock Android browser installed we didn?t get a chance to test one yet, but if you look at the scores you can see that the Bay Trail tablet almost caught up to the Snapdragon 800 in the stock Android browser. When you consider the point differential, it?s basically a tie. The Tegra 4 hung around a bit in the Chrome tests and certainly didn?t lag behind at all, meaning that we could expect a tablet like the Tegra 4 Note 7 to possibly deliver pretty competitive performance (if thermals aren?t an issue).

Then, we looked at some Windows-only benchmarks. For these benchmarks, we were most interested in seeing how Bay Trail stacks up in terms of performance against its predecessor, Clover Trail. After all, I believe many of you are interested to see what kind of improvements Bay Trail brings to Windows tablets. For this, we opted to test in AIDA64 and check out the memory, CPU and FPU performance tests that AIDA64 offers.

Here in the memory bandwidth test, we can see that with the addition of DDR3 and an improved memory controller, Bay Trail is nearing 3x the memory bandwidth of Bay Trail in Read, Write and Copy. This is important if they want to elevate the overall speed of the Valleyview SoC on the Bay Trail platform. However, memory latency was slightly reduced from 133 ns to 117 ns, clearly some room for improvement for Intel there.

If you look at the CPU performance, you can see that Intel has made significant improvements in terms of the Silvermont CPU architecture. As such, the smallest performance increase was 20% and the greatest we saw was over 100%. A performance increase of over 100% is nothing short of fantastic, but also tells you how incredibly weak Clover Trail really was.

As you can see, Intel didn?t put much focus on their FPU with Bay Trail and pretty much kept their performance about the same if not slightly faster or slightly slower. Keep in mind that not all of AIDA64?s benchmarks are quite optimized for Bay Trail, so we may have to come back and revisit these benchmarks once we start to see more Bay Trail devices and updates from FinalWire.

Overall, if you look at Bay Trail, you can see that Intel is finally serious about mobile. They clearly have a competitive chip that will really propel Intel based Windows 8 tablets into an acceptable performance category. After using Bay Trail I have really begun to understand how slow Clover Trail really is and how badly Windows 8 tablets need Bay Trail if they want to have long battery life and acceptable performance. It will be interesting to see how Bay Trail makes its way into tablets and how the final implementations make it to the market. Clearly the Android version isn?t ready yet, however, I expect that Intel could give Qualcomm and Nvidia a run for their money early next year when their Android version is ready. More competition is always better, thankfully Intel is pretty competitive.