Comparing tablet chip performance has never been easy. Mobile benchmarks have mostly been limited to just a single platform, but now a new test by a respected outfit—albeit one that’s frequently been accused of pandering to Intel—puts ARM-based processors from Apple and Nvidia ahead of Intel’s silicon.
BAPCo’s TabletMark V3, announced last week, shows that Google’s new Nexus 9 and Apple’s iPad Air 2 outperform an Atom-powered tablet from Lenovo.
The new benchmark is the first from BAPCo to support anything other than a Microsoft operating system. Previous iterations of TabletMark have supported Windows 8, Windows 8.1 and Windows RT, but now version 3 finally adds iOS and Android compatibility.
The test suite is designed to measure real-world browsing, email, photo and video chores on tablets, and it's already freely available in the Microsoft store and Android Play. The iOS version is finished, but it remains stuck in the famously slow Apple approval process.
In a recent press demonstration, BAPCo showed off the new TabletMark running on an Apple iPad Air 2 with its A8X chip; the Google Nexus 9 with its Nvdia Tegra K1; and a Lenovo Thinkpad 10 with a quad-core Intel Atom Z3795.
The real shocker is where Intel’s Bay Trail-based Atom Z3795 falls in line: last. Although it is slightly faster than the iPad 2 Air’s A8X in the browsing and email tests, it takes a distant third place in the photo and video workloads. The Tegra K1, meanwhile, actually represents Nvidia well by beating Intel’s Bay Trail chip.
Yes, what we have are two ARM chips from Apple and Nvidia acing Intel’s top budget mobile chip. To anyone who’s been following the benchmarking wars, this might come as a bit of a surprise, as BAPCo has long been accused of favoring Intel parts.
AMD famously took its ball and abandoned its BAPCo membership in 2011—this after accusing BAPCo of cooking the PC test suite SYSMark2012 so it wouldn’t reflect real-world usage models. At the time, AMD’s main beef was that SYSMark2012 under-utilized GPUs. AMD’s case was strengthened when VIA and Nvidia also dropped their BAPCo membership.
In its defense, BAPCo said it relied on advice from application vendors on whether to use GPUs or CPUs in specific tests. Adobe, for example, advised that using the GPU to render video could cause problems. What’s more, in 2011 few apps actually supported GPU workloads.
To BAPCo, that’s all water under the bridge. It maintains that Intel only gets one vote on its board, which includes more than a dozen members. BAPCo also said it hopes AMD and ARM vendors come back with the release of TabletMark V3—a benchmark, it points out, that AMD actually helped create the process for while it was a member years ago.
If you’re wondering how beefier “tablets” fared, wonder no more. I ran TabletMark on all iterations of the Surface Pros I had access to, as well as Lenovo’s new Yoga 3 Pro convertible, which runs Intel’s Core M CPU. Interestingly, from a cumulative numerical score perspective, the Surface Pro 3 fared the best, although the Yoga 3 Pro came in slightly faster in the photo and video tests. The overall result, BAPCo says, is a geometric mean between the two.
TabletMark V3 looks pretty solid as a cross-platform benchmark, but it won’t be the last word in performance testing. To make the benchmark run across all of the platforms, BAPCo had to write the apps from scratch to simulate photo, video, browsing and email. BAPCo said it used all available APIs on each platform, but many enthusiasts will still gripe that BAPCo should have used other methods. The benchmark is based on real-world use cases, but it remains something of a synthetic kludge in the end to get it to work across all the different operating systems.
That said, TabletMark V3 is the only benchmark with such widespread vendor support, and it's a pretty good tool for anyone who wants to assess tablet performance across multiple platforms.
BAPCo’s new test is already available in Google Play and the Windows Store for free. If you want to run it, you’ll need a 7-inch minimum screen size with a resolution of 1024x768, 2GB of RAM and 2GB of storage. The benchmark supports 64-bit ARM and x86 chips, and will run on Windows 8.1, Android 4.3 and iOS 7 and higher.
So where does your tablet fall? Download TabletMark V3, give it a whirl and let the bickering begin.
Clarification: Our story was updated to reflect that AMD started the process that TabletMark V3 is based on and did not actually create TabletMarket V3. BAPCo also said the overall score between the email/browsing and photo/video is a geometric mean.