The world of smartphone benchmarks has become a contentious issue, with some calling it "cheating" and others arguing that it's simply a reflection of the devices' capabilities. At its core, smartphones are designed to run efficiently, but when it comes to pushing their performance limits, manufacturers often use benchmarking tests as an opportunity to showcase their hardware's prowess.
It's worth noting that smartphones are at the other end of the scale from high-performance computers like gaming PCs or workstations. While those devices have cooling fans and can run on mains electricity, smartphones are limited by their batteries and lack of cooling systems. As a result, they must balance performance with battery life, making it increasingly difficult to achieve both goals.
When device makers design new smartphones, they make decisions about the processor, clock speed, and graphics processing unit (GPU) that will ultimately impact their performance. However, software also plays a crucial role in optimizing these settings. In fact, some smartphones can dynamically adjust their performance based on the apps running in the background. This means that even when an app is closed, its unique ID remains associated with the device's firmware.
In this context, it's possible for manufacturers to use software tweaks to artificially boost a smartphone's performance during benchmarking tests. By detecting which apps are running and adjusting the CPU and GPU accordingly, they can present a more favorable picture of their device's capabilities. While this might not be entirely transparent, it's also not necessarily cheating in the classical sense. After all, benchmarks are inherently artificial, designed to showcase a device's performance under specific conditions.
To illustrate this point, consider the car industry. We've seen manufacturers manipulate emissions and fuel consumption figures to make their vehicles appear more environmentally friendly or fuel-efficient than they actually are. Similarly, smartphones are subject to similar pressures, with manufacturers competing to produce devices that can deliver impressive benchmark scores. It's a cat-and-mouse game, where each side tries to outdo the other in terms of performance and capabilities.
So, what does this mean for consumers? While it's impossible to know exactly how much cheating is going on in the benchmarking world, it's clear that devices are being pushed to their limits in pursuit of high scores. This can lead to some impressive performances, but also raises questions about the accuracy of these results. Are we really seeing a true reflection of a device's capabilities, or are we just being presented with a curated version of its performance?
Ultimately, it comes down to perspective. While some might argue that manufacturers are cheating, others see it as simply a way to showcase their devices' potential. As someone who has explored the world of smartphone benchmarks in depth, I'm left wondering whether the benefits outweigh the drawbacks. Are we getting an accurate representation of a device's capabilities, or are we being sold a vision of what it could do rather than what it actually can?
The debate surrounding benchmarking is ongoing, and one thing is certain: the lines between truth and marketing speak are becoming increasingly blurred. As the smartphone industry continues to evolve, we can expect even more innovative solutions from manufacturers designed to outdo their competitors. Whether or not these efforts are ultimately beneficial for consumers remains to be seen, but one thing is clear – the world of smartphones will always be a high-stakes game of cat and mouse.
Download the Android Authority app and don't forget to go over to Android Authority.com because we are your source for all things Android
"WEBVTTKind: captionsLanguage: enhello there my name is Gary Sims from Andro Authority now uh Benchmark cheating has come up again in the news this time it was one plus that have been accused of cheating the benchmark scores on the OnePlus 3T they're not the first company to ever be accused of Benchmark cheating so the question is why do oems try to cheat on the benchmarks and how do they do it well let me explain so first a bit of about the types of people that buy smartphones I suppose broadly speaking there are three groups of people that buy smartphones the first group of person doesn't really care much about the specifications or the speed they'll go into the shop to buy the phone they'll ask the sales assistant they'll say yeah it's a good phone they like the price they'll buy it that's the end of the transaction dumb deal the second category of person are those that have a bit of Technology uh understanding they might understand what the specs mean they'll understand the difference between 4 GB of RAM and 2 GB of RAM 16 GB of internal storage 32 GB of internal storage they might understand screen resolutions that kind of thing and they'll make an educated uh decision based on what they understand about each of the phones they may also have read some stuff on on the internet but it isn't their era of expertise but they know enough about technology to make an educated purchase and then I suppose the third category of person is the one that is basically the geek the nerd that knows everything about smartphones reads all the reviews watches all the videos uh knows what's coming down the pipeline knows what was the best phones and things like they're the kind of person that you yes hi oh hi yeah I I'm making a making a video okay be quick okay yeah yeah it's a good phone it's a good phone what's the price oh excellent price yes great off contract brilliant okay by I've got to go okay thank bye bye bye bye bye bye sorry about that so they're the kind of person that you ring when you want to know what is a good phone to buy so the question is how do oems actually cheat well what actually happens is when a uh device is made there is a balance has to be struck between performance and power usage now really these two are enemies they're log ahads with each other let me give an example on your desktop you've probably got a an Intel or an AMD CPU it's maybe giving out between 50 100 maybe more watts of thermal uh heat it is got big heat sink on it it's got cooling fans of course it's running from the mains electricity now of course smartphones are at the other end of that scale they are run by batteries they mustn't get too hot they've got no uh fans on them and basically the more performance that you want to get from a mobile device the more heat it produces and the more battery life it uses so there's a balance between getting good battery life and getting um a good performance now when the device makers make a device they make various decisions in the hardware like the processor and the clock speed of the processor and the clock speed of the GPU and and other things like that and they also make uh some decisions in the software now the software can of course be tweaked dynamically while you're using the phone the phone is actually always changing the speed of the CPU the speed of the GPU it's throttling it's increasing there's a whole technology here which I've covered in some of my other videos now every app that runs has a unique ID and those unique IDs are part of the uh information that's put on the Google Play Store and so the uid the unique ID of every app Remains the Same so and22 has the same uid today and tomorrow and it had it last year and basically it's not very hard for an oem to put into its firmware something that detects when a particular app is running and if it is running it can tweak the software to say hey let's boost the CPU artificially during these few next few minutes let's boost the GPU now it's not the normal circumstances that the phone would use because if it ran at that speed the whole time the battery would go flat very quickly the phone would heat up very quickly but for a 90-second test for a 2-minute test it's not going to make any long-term difference and so the way they're cheating is basically when they see and22 running or geekbench running or some of the other popular ones basically the firm where says hey let's push push out the CPU let's push up the GPU and then it gives out different results than it would be if it was using its normal settings it normal performance settings for when it's running uh normal programs now of course there is this question I'll throw it out there I'll say from the beginning I think it is cheating but let me ask you is it really cheating because actually it's not like the firmware has somehow managed to send uh messages or inject data into the Benchmark and actually get it to display a full score that was the score that that device produced that is what that device is capable of doing but it's not what the device does dayto day because to reach that capability it actually has to um run the CPU much harder than it would under normal circumstances but let me look at it this way what about the speed things for cars people it will do not to 60 in you know 4 seconds or something like that well how many people actually pull away from traffic lights to go North to 60 in 4 seconds well you don't it's just too dangerous it's it's not what you do but actually people will say wow look that's the speed of that car so you know there are artificial statistics that exist in all Industries and maybe the benchmarks are an artificial statistic in this way in fact they are artificial in the sense they don't really reflect the workload than a normal app does I mean even playing quite complicated games doesn't actually reflect what you get in in the benchmarks I've done some other articles about this that shows you how the benchmarks use the CPU and GPU the graphs that are produced and they're very different to what you get under normal game so benchmarks are artificial to start with so if the smartphone is using them artificially well is that really a big issue I guess it is I I say it is cheating because people expect the phone to run that fast in normal usage and it doesn't but it's an interesting question I'd like to hear your thoughts in the comments below and so there you have it the oems are cheating on the benchmarks now this is not the only industry this is happening it's happening in the car industry we've seen that with emission we've seen that with fuel consumption figures this is going to happen in all Industries because it's so Cutthroat that people the oems are trying so hard to get ahead so their product sells the most doesn't mean it's right but I'm just saying it's a reality my name is Gary Sims from Android authority and I hope you enjoyed this video if you did please do give it a thumbs up don't forget to subscribe to Android Authority's YouTube channel uh please tell me in the comments below what you think about oem's cheating on benchmarks download the Android authority app and don't forget to go over to Android authority.com because we are your source for all things Androidhello there my name is Gary Sims from Andro Authority now uh Benchmark cheating has come up again in the news this time it was one plus that have been accused of cheating the benchmark scores on the OnePlus 3T they're not the first company to ever be accused of Benchmark cheating so the question is why do oems try to cheat on the benchmarks and how do they do it well let me explain so first a bit of about the types of people that buy smartphones I suppose broadly speaking there are three groups of people that buy smartphones the first group of person doesn't really care much about the specifications or the speed they'll go into the shop to buy the phone they'll ask the sales assistant they'll say yeah it's a good phone they like the price they'll buy it that's the end of the transaction dumb deal the second category of person are those that have a bit of Technology uh understanding they might understand what the specs mean they'll understand the difference between 4 GB of RAM and 2 GB of RAM 16 GB of internal storage 32 GB of internal storage they might understand screen resolutions that kind of thing and they'll make an educated uh decision based on what they understand about each of the phones they may also have read some stuff on on the internet but it isn't their era of expertise but they know enough about technology to make an educated purchase and then I suppose the third category of person is the one that is basically the geek the nerd that knows everything about smartphones reads all the reviews watches all the videos uh knows what's coming down the pipeline knows what was the best phones and things like they're the kind of person that you yes hi oh hi yeah I I'm making a making a video okay be quick okay yeah yeah it's a good phone it's a good phone what's the price oh excellent price yes great off contract brilliant okay by I've got to go okay thank bye bye bye bye bye bye sorry about that so they're the kind of person that you ring when you want to know what is a good phone to buy so the question is how do oems actually cheat well what actually happens is when a uh device is made there is a balance has to be struck between performance and power usage now really these two are enemies they're log ahads with each other let me give an example on your desktop you've probably got a an Intel or an AMD CPU it's maybe giving out between 50 100 maybe more watts of thermal uh heat it is got big heat sink on it it's got cooling fans of course it's running from the mains electricity now of course smartphones are at the other end of that scale they are run by batteries they mustn't get too hot they've got no uh fans on them and basically the more performance that you want to get from a mobile device the more heat it produces and the more battery life it uses so there's a balance between getting good battery life and getting um a good performance now when the device makers make a device they make various decisions in the hardware like the processor and the clock speed of the processor and the clock speed of the GPU and and other things like that and they also make uh some decisions in the software now the software can of course be tweaked dynamically while you're using the phone the phone is actually always changing the speed of the CPU the speed of the GPU it's throttling it's increasing there's a whole technology here which I've covered in some of my other videos now every app that runs has a unique ID and those unique IDs are part of the uh information that's put on the Google Play Store and so the uid the unique ID of every app Remains the Same so and22 has the same uid today and tomorrow and it had it last year and basically it's not very hard for an oem to put into its firmware something that detects when a particular app is running and if it is running it can tweak the software to say hey let's boost the CPU artificially during these few next few minutes let's boost the GPU now it's not the normal circumstances that the phone would use because if it ran at that speed the whole time the battery would go flat very quickly the phone would heat up very quickly but for a 90-second test for a 2-minute test it's not going to make any long-term difference and so the way they're cheating is basically when they see and22 running or geekbench running or some of the other popular ones basically the firm where says hey let's push push out the CPU let's push up the GPU and then it gives out different results than it would be if it was using its normal settings it normal performance settings for when it's running uh normal programs now of course there is this question I'll throw it out there I'll say from the beginning I think it is cheating but let me ask you is it really cheating because actually it's not like the firmware has somehow managed to send uh messages or inject data into the Benchmark and actually get it to display a full score that was the score that that device produced that is what that device is capable of doing but it's not what the device does dayto day because to reach that capability it actually has to um run the CPU much harder than it would under normal circumstances but let me look at it this way what about the speed things for cars people it will do not to 60 in you know 4 seconds or something like that well how many people actually pull away from traffic lights to go North to 60 in 4 seconds well you don't it's just too dangerous it's it's not what you do but actually people will say wow look that's the speed of that car so you know there are artificial statistics that exist in all Industries and maybe the benchmarks are an artificial statistic in this way in fact they are artificial in the sense they don't really reflect the workload than a normal app does I mean even playing quite complicated games doesn't actually reflect what you get in in the benchmarks I've done some other articles about this that shows you how the benchmarks use the CPU and GPU the graphs that are produced and they're very different to what you get under normal game so benchmarks are artificial to start with so if the smartphone is using them artificially well is that really a big issue I guess it is I I say it is cheating because people expect the phone to run that fast in normal usage and it doesn't but it's an interesting question I'd like to hear your thoughts in the comments below and so there you have it the oems are cheating on the benchmarks now this is not the only industry this is happening it's happening in the car industry we've seen that with emission we've seen that with fuel consumption figures this is going to happen in all Industries because it's so Cutthroat that people the oems are trying so hard to get ahead so their product sells the most doesn't mean it's right but I'm just saying it's a reality my name is Gary Sims from Android authority and I hope you enjoyed this video if you did please do give it a thumbs up don't forget to subscribe to Android Authority's YouTube channel uh please tell me in the comments below what you think about oem's cheating on benchmarks download the Android authority app and don't forget to go over to Android authority.com because we are your source for all things Android\n"
