Do Smartphone Benchmarks Really Matter?

August 5, 2013 - Written By Lucian Armasu

In the Android world we’ve gotten used to evaluating our smartphones based on how well they do in benchmarks. Initially the Quadrant benchmark was the most popular, but that has gotten less and less reliable as it stopped predicting the performance of phones that well anymore, and some companies were starting to game it anyway.

These days it’s AnTuTu combined with other specialized benchmarks, like the GPU benchmarks such as 3dmark, GFXBench and Basemark X. But as we’ve learned recently, even these can and will be gamed. So in a time when benchmarks are getting gamed and raw performance is starting to become less relevant for your phones, how much do these benchmarks really matter anymore?

I’m sure there are plenty of people who still care about them, but our phones can do almost anything instantly these days, and getting 2x faster every year won’t really be noticeable on a day to day usage of the device, so then benchmarks will be pretty much the only way to know your “next-gen phone” actually is faster. But if you can’t tell on your own, then why does it matter? It shouldn’t.

That’s why I think smartphone OEM’s should come up with new hardware innovations that they can add in their phones, and they shouldn’t just play it safe by increasing the specs of their components from last year. They should focus more camera performance for example, and by this I mean doing something radically new, to get much closer and much faster to DSLR quality than it would be doing if they just kept evolving them like they usually do every year. Make quantum leaps in camera performance, and sell that as a main benefit instead of processor speed. I think the mass market will be more impressed with that this point in time.

Making smartphones lasting a standard of 2 days, instead of the standard of one day right now, would also be a great way to think differently, and gain new customers. But these are just some of the more “obvious” tactics that they can use. They are the ones that know what else is being researched lately, and if they think something is the future, they should put the money behind it, and launch it in their phones. I think some of them will do this, and we’ll see a much more dynamic smartphone market next year, with everyone trying all sorts of different innovations.