We recently published our in-depth Xiaomi Mi 8 review, complete with a full set of tests: display, audio quality, performance, you know the drill. For the most part, it was business as usual, except for certain performance aspects.
Even though the phone comes with the latest Snapdragon 845 chipset, in most synthetic benchmarks it only seemed to perform at the level of a Snapdragon 835 instead. A lot of you were confused by the performance numbers we published. Some even thought our review device or its software was to blame.
So we went ahead and re-tested the handset under more favorable conditions - by applying extra cooling to its back. We reported our findings in the Performance section of the review driving the point that the software is not to blame, it's the device's thermal management.
Even so, we felt it would be beneficial to go into a bit more detail about what happened with the Mi 8 and how we went about investigating it. Also, a few words about what can be done and what this means for the end-user. Long story short:
After seeing the Snapdragon 845-equipped Xiaomi Mi 8 get benchmark scores in the Snapdragon 835 ballpark, it was pretty clear that the handset was capping its performance, to adhere to some thermal constraints. At least, that was the first theory, since there were no obvious signs of any software issues, bugs or benchmark incompatibilities. Plus, MIUI has a pretty solid track record, when it comes to proper hardware optimization.
So, we broke out our home-made active cooling solution. It is comprised of a Peltier element, sandwiched between two beefy CPU coolers. This allows us to rest a phone on a flat actively cooled surface while using it. You could always run these tests inside a fridge to the same effect but that's nowhere as convenient, isn't it?
Higher is better
Just to be clear, we sometimes use the cooler to chill phones in between benchmark runs, not during, so that we don't have to wait as long between runs. If you leave the phone on the cooler, the results are understandably a bit skewed and could only apply if your natural every-day environment is a meat freezer. Still, it helped us illustrate a point, namely, with a cooler applied, the Mi 8 performed exactly as expected of a Snapdragon 845.
AnTuTu seems to show the variance between the cooled and un-cooled runs best, but GeekBench was also affected. Especially the multi-threaded workload segment.
Higher is better
GPU performance was naturally also affected by the thermal-throttling issues, but not quite so consistently. When rendering graphics in the off-screen test, the Adreno 630 gets pushed to the brim of its performance. Naturally, since the GPU is part of the same chipset as the CPU cores, thermal limitations kick in right away, and you get throttling.
Higher is better
On-screen, however, the situation is a bit different, since some other performance limitations come into play. The Mi 8 is obviously hitting other limitations, way before the thermal threshold of the SoC, which cap the displayed frame-rate at 50 fps, in the particular test, regardless of cooling.
Higher is better
This is mostly good news for potential owners since this is the test more closely representative of real-world usage. It means that you probably won't be losing any frames in your games on the Mi 8. At least not until you play a pretty extended session. And practically every passively cooled mobile device out there is subject to this kind of performance loss.
The CPU performance drop could potentially be felt easier in real life, but only with a very specific subset of tasks. For instance, transcoding video, archiving or de-archiving large pieces of data - long-lasting, CPU intensive loads. Since most users don't really do those a lot, it's fair to assume that the Mi 8 will perform comparably to another Snapdragon 845 flagship in a real-world scenario.
Some of you might be happy with this conclusion. But if you really want to know what's happening behind the scenes (we certainly did), read on, since we did a few other tests after we published our review.
For this next portion of the testing, we grabbed a couple of really handy apps off the Google Play Store - CPU Throttling Test and CPU Float.
One conducts a prolonged stress test on the CPU while monitoring core frequencies and temperature. Then spits out an average, min and max estimated performance metric and even calculates how much performance the device lost over time, due to throttling. The second app is a simple overlay to see the real-time CPU core frequencies and a second thermal reading. We just wanted to double up the readings, just to make sure we are not missing anything.
So, first, here is the Xiaomi Mi 8, running the stress test for five minutes at room temperature, with no extra cooling. The screenshots are taken at about 30 second intervals, so you can see how things progress in time.
We can clearly see that as early as two minutes into the test, things start to taper off quickly. Performance drops, and it doesn't really recover until the very end of the run. We can note, from the breakdown at the end, that the Mi 8 ended up with about 76% of its maximum performance in the short 5 minute period.
That's all fine and dandy, but these numbers mean little without a reference point. So, we grabbed the OnePlus 6 and ran the same software. OnePlus is known for squeezing the maximum performance possible out of their chips, and the OnePlus 6 is a frequent chart-topper among Snapdragon 845 devices. We also have pretty solid empirical evidence that it doesn't really have thermal throttling issues.
Here is its extended 6-minute run.
We can clearly see the effects on longer workloads on a chipset. Or rather, pure heat. But both the average values and the percentage of maintained performance, in the end, are better. That is roughly the performance graph we can expect from a Snapdragon 845 chipset.
So what's happening? Well, you might notice that during the tests, the power-efficient quartet, as well as the more powerful four CPU cores of the OnePlus are maintaining a significantly higher temperature than those of the Mi 8. What that essentially means is that OnePlus raised the thermal limit of the Snapdragon 845 a bit higher. Or if you look at it from another angle, Xiaomi lowered the one on the Mi 8.
Some explanation might be necessary here. A typical CPU basically adheres to at least a couple of thermal constraints automatically. One constitutes a maximum safe temperature for operation. If exceeded, the chip might suffer some damage - physical and often irreversible. This is generally set by the chip vendor, and when reached, the hardware usually takes drastic measure to ensure its well-being, like a forced shutdown or a restart.
Then there are one or more thermal limits that set a temperature, that when reached, trigger an automatic measure to lower the performance of the chip in an effort to prevent it from getting warmer or at least reduce the pace at which it is accumulating heat. This is the threshold we are interested in and the ones phone manufacturers typically have control over.
Now, there might be numerous reasons for a manufacturer to decide to go easier on the chip and set a lower thermal threshold. Longevity of components comes to mind. Also, with a device as compact as a smartphone, you have to think about the overall package heat and other close-by components that might raise the overall internal heat level, while active alongside the chipset. For instance, the Mi 8 has a fancy new dual frequency GPS chip. It could potentially be the source of some extra heat as well.
Before wrapping things up, we will show you the effects of active cooling on the Mi 8. The eagle-eyed among you might have noticed a small detail, we ourselves were initially a bit perplexed about. Despite the obvious manifestation of thermal throttling, the tests seemed to always show all eight cores maintaining their maximum frequency.
This doesn't really add up since the way throttling typically works is by lowering the current operating frequency of the core. Without getting overly technical and opening up a can of worms, known as the Android CPU governor, let's just say that the reported values within the app are effectively maximums. Hence, these are not really representative of the current CPU frequencies, which are constantly changing. We just didn't have the means to monitor them.
This is the reason we used the CPU Float app. It is the small overlay window, covering part of the graph in the following screenshots. Unfortunately, this didn’t help much, as it appears that the precise CPU frequency values aren't really available without root access.
This time we let the Mi 8 run for a full 10 minutes. We had initially thought that 5 minutes would be enough to showcase the full extent of the performance loss in time. It turns out we were wrong. Letting the torture go for 10 minutes left us with as little as 67% of the maximum performance.
Since we felt pretty good about the temperatures the chipset was maintaining through, we let the test run for another 10 minutes and are happy to report that this is about the minimum performance level we observed.
Last, but not least, just to verify the extent the thermal management policy affects the Mi 8 performance, we also did a 10 minute, actively-cooled run. Same parameters, otherwise. See for yourselves:
All that said, we feel like there should be an interesting conclusion to make here. Well, this was an exciting, even if time-consuming way to illustrate a different approach to thermal and power management.
If there is one real takeaway from this, it has to be the same one we typically mention in the Performance sections of our reviews - benchmark numbers are a cool tool for comparison, but don't always translate well to real-life scenarios. In reality, the Mi 8 performs as smooth as the next 2018 flagship.
You might not be getting the full raw performance potential of the Snapdragon 845, but there are other benefits to having the CPU run cooler. Plus, manufacturers often switch-up frequencies, thresholds and performance parameters on their devices with software updates so this pre-programmed behavior may change over time. Most of the time, it's for the benefit of the average user, one way or another. It's just how the industry works.
Special thanks to HonorBuy for providing the review unit.
The Mi 8 is still getting a higher score in Geekbench and higher FPS in GFX than a Snapdragon 835 even without the extra cooling... And how come the Xperia XZ2 getting almost the same score in Geekbench and GFX as the Mi 8 without extra cooling? Wa...
Really sad how nobody actually understood the point of the article.The main point is that as soon as the device hits a certain temperature point(which is very low compared to the OP6,Vivo Nex etc.),it starts throttling,which means it won't hit any hi...
That's what i thought,99% of Mi 8 reviews show that it hits between 260k and 270k...Definitely gonna run all these tests to check if i have a faulty unit so that i can replace it asap.