Nexus 6 Performance With Android 5.1

  1. Introduction
  2. CPU And System Performance
  3. Web Browsing Benchmarks
  4. Storage Performance
  5. Battery Life
  6. GPU And Gaming Performance
  7. Conclusion

Does the Android 5.1 update bring performance and battery life improvements to the Nexus 6?

In our Nexus 6 review, we saw its Snapdragon 805 SoC deliver strong CPU and GPU performance while running the initial Android 5.0 release. Internal storage performance however, was a point of contention.

Google Nexus 6

Read the Review

With Android Lollipop, Google announced that full disk encryption (FDE), an optional feature available since Android Honeycomb, would be enabled by default. This requirement was later revoked due to performance issues on certain classes of hardware, the Nexus 6, having shipped with FDE enabled, being a prime example.

At the conclusion of our initial performance analysis, Google released the Android 5.1 update, which includes some patches meant to address the performance impact of using FDE on the Nexus 6, in addition to changes that affect overall system performance and battery life.

In light of these changes, it’s worth taking a second look at the Nexus 6, comparing performance after installing the Android 5.1 update to the initial 5.0 release.

CPU And System Performance

Google has made a couple of changes in the 5.1 update specific to the Nexus 6 that will impact system performance. For starters, Qualcomm’s thread migration boost feature has been disabled. Prior to the update, the task scheduler would notify the Qualcomm driver whenever a thread migrated from one CPU core to another, and in an effort to increase system responsiveness, the destination core’s frequency would be boosted to the lesser of the origin core’s frequency or 1.7GHz. The boosted core would remain at the higher frequency for at least 20ms. Given the frequency with which threads migrate across cores—sometimes dozens of times per second—this feature has a nontrivial influence on performance and battery life. The similar but different feature that boosts CPU frequency briefly when a touch event is detected is still active, however.

Some of the performance loss (and battery life gain) from disabling the thread migration boost may be offset by the second big change. The Nexus 6 running Android 5.0, like all of the other devices with four or more cores we’ve tested, only kept two cores online, shutting down the other two to reduce static power drain. An updated Nexus 6 will now keep all four cores available, which eliminates the latency involved when activating cores. It should also help the OS spread system load more evenly and possibly improve responsiveness since running processes are less likely to interrupt each other.

System Performance Benchmarks

Benchmark Nexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
AndEBench Pro Device Score 8287 8160 -1.54%
CoreMark-HPC (Base) 3335 3282 -1.59%
Memory Bandwidth 7843 MB/s 7809 MB/s -0.44%
Memory Latency 4633 KOps/s 4915 KOps/s 6.09%
Platform 200.9 185.9 -7.47%
3D 28.2 fps 28.7 fps 1.79%
AnTuTu Total Score 52094 50550 -2.96%
Basemark OS II Full Overall 1368 1395 1.94%
System 2851 2614 -8.31%
Memory 631 764 21.00%
Graphics 2466 2601 5.50%
Web 790 729 -7.66%
Geekbench 3 Single-Core Geekbench Score 1051 1057 0.57%
Integer 1061 1085 2.26%
Floating Point 882 888 0.68%
Memory 1374 1342 -2.29%
Geekbench 3 Multi-Core Geekbench Score 3250 3198 -1.62%
Integer 3876 3726 -3.87%
Floating Point 3269 3297 0.86%
Memory 1964 1944 -0.99%
PCMark Work Performance 4241 4234 -0.17%
Web Browsing 4064 4040 -0.58%
Video Playback 3566 3930 10.19%
Writing 4777 4243 -11.19%
Photo Editing 4672 4797 2.68%

Looking at the results in the table above, we see a mixture of plusses and minuses. Most of the deltas are small, less than plus or minus 3% and within the tolerance band for these tests; Since many of these synthetic tests keep all available cores fully active, they are not affected by the changes included in the 5.1 update. Where we do see a small loss in performance are less intense CPU-centric workloads such as AndEBench Platform, Basemark OS II System, and PCMark Writing. These tests do not keep all of the cores fully tasked, and subsequently are more sensitive to the thread migration boost. Also, with all four cores active in 5.1—with one or two cores focused on the foreground app and the remaining cores handling storage I/O or processing a background task for example—there’s less thermal/power envelope to set individual cores to their max frequency. This is what happens in the PCMark Writing test, where two or more cores are generally active (there’s several small, interspersed storage reads/writes) but no single core goes above 1497MHz. Running Android 5.0, the Nexus 6 holds two cores at max frequency (2649MHz) for the duration of the test.

The Basemark OS II System test provides another example of the differences in frequency scaling. Both the Math and Multi-Core tests load all available cores at 100%, so we don’t see any difference in behavior or performance between the two different OS versions. Performance in the Single-Core test varies quite a bit when running the 5.1 update, sometimes scoring a little better and other times a little worse than the initial release. It’s the XML Parsing test where we see a significant performance drop-off when running the 5.1 update. With Android 5.0, two cores run at max frequency for the duration the test. After the update, the max frequency is achieved only sporadically, with both cores at max frequency for just a couple short bursts. The end result is less average processing power available in this particular scenario due to the changes made in 5.1.

Web Browsing Benchmarks

Benchmark Nexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
Browsermark 1259 1258 -0.08%
JSBench 340.2 ms 344.0 ms -1.11%
Peacekeeper 953 908 -4.72%
Google Octane 4322 4254 -1.58%

The update does not seem to impact JavaScript performance in our browser benchmarks.

MobileXPRT 2013

Test Nexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
Overall Performance 238 209 -12.18%
Apply Photo Effects 24.62 s 29.91 s -21.49%
Create Photo Collages 15.11 s 18.57 s -22.94%
Create Slideshow 26.50 s 29.50 s -11.32%
Encrypt Personal Content 57.94 s 67.55 s -16.59%
Detect Faces 6.64 s 6.57 s 0.99%
Overall User Experience 100 97 -3.00%
List Scroll 60 fps 60 fps 0.00%
Grid Scroll 60 fps 60 fps 0.00%
Gallery Scroll 50 fps 50 fps 0.00%
Browser Scroll 59 fps 50 fps -15.25%
Zoom and Pinch 57 fps 57 fps 0.00%

MobileXPRT 2013 shows a worst case scenario for the updated Nexus 6. All of the content related tests read and write small amounts of data to disk and don’t fully task the CPU cores. The lower scores here likely reflect the loss of thread migration and a small reduction in storage performance for certain workloads (more on this in the next section).

The user experience tests are unchanged except for Browser Scroll, which sees a significant reduction. Investigating further reveals that the Nexus 6, running either Android 5.0 or 5.1, sets the CPU frequency to a fixed 1497MHz during UI interactions such as the List Scroll, Grid Scroll, Gallery Scroll, and Zoom and Pinch tests. When running 5.0, the Nexus 6 sets the CPU frequency to a minimum of 1497MHz during the Browser Scroll test but frequently spikes to its max frequency of 2649MHz. The 5.1 update changes this behavior, with the CPU governor frequently letting the frequency drop to 300MHz during the Browser Scroll test, which accounts for the lower score.

To see if these results are valid when using a real web browser, I fired up Opera and loaded a simple page without ads (Wikipedia’s home page). I then continuously scrolled up and down while monitoring the CPU usage and frequency. The Nexus 6 running the 5.0 release locks the CPU frequency at 1728MHz with two cores active. Running 5.1, the CPU frequency is locked at 1497MHz, which is the same frequency as the Browser Scroll test and equal to the touch input boost frequency.

For comparison, the Samsung Galaxy Note 4 running Android 5.0.1, which also uses a Snapdragon 805 SoC, behaves quite differently during these tests. The CPU governor allows the frequency to bounce around quite a bit during all of the user experience tests except for Browser Scroll, where the frequency is set to 1190MHz. With a lower average CPU frequency, the Note 4 performs worse than the Nexus 6 in these tests. I also performed the same basic browser scrolling test in Opera on the Note 4, and it also set the CPU frequency to 1190MHz.

These results suggest that UI responsiveness remains essentially unchanged with the OS update, with the Nexus 6 maintaining its lead over the Note 4 in this critical area. However, these benchmarks fail to capture the effect of having all four CPU cores active all the time. In the real-world, there will be several background tasks running, performing tasks like checking for new email, social media news, and app updates. It’s these uncontrolled scenarios where having the additional cores available will help balance the load and help keep these other processes from interrupting UI update threads. So while peak UI responsiveness is unchanged, or even slightly reduced, the effective responsiveness seen in normal use may be better with the 5.1 update.

Storage Performance

The specific issue vexing the latest Nexus is that the hardware encryption engine provided by its Snapdragon 805 SoC is disabled, forcing all file system encryption/decryption to be executed in software on the CPU.

In November and early December of last year, during the development of the 5.1 update, there was a series of software commits enabling and disabling the hardware based encryption engine. Ultimately, hardware encryption was disabled again for the Nexus 6 5.1 update. We asked Google about this decision, but they would not provide an answer.

Even though file encryption is still funneled through the CPU, the update should still improve storage performance. The encryption engine can now utilize NEON instructions and has additional threads at its disposal. Previously, with only two CPU cores online, the encryption driver was allocated only two threads to handle file read and write requests. The 5.1 update however, keeps all four cores active all the time, which means the encryption driver now has up to four threads to perform its work.

AndEBench Pro Storage Test

  512B
SW
512B
RW
4KB
SR
4KB
SW
4KB
RR
4KB
RW
16KB
RR
256KB
SR
256KB
RR
Galaxy Note 4 (5.0.1) 449 1025 20877 3340 20425 7992 56653 125891 127268
Nexus 6 (5.0) 209 723 9084 1322 7449 5951 12969 23107 24857
Nexus 6 (5.1) 169 636 8714 982 6116 5676 15060 30932 32646
Nexus 6
% Diff
-18.9% -12.0% -4.1% -25.7% -17.9% -4.6% 16.1% 33.9% 31.3%
First letter: S=sequential, R=random
Second letter: R=read, W=write
Values in KB/s – Higher is better
File Size: 5, #Folders: 3, #Files/Folder: 1

The results from the AndEBench Pro storage test show nice gains reading larger blocks of data, just over 30%. Unfortunately, reading and writing smaller blocks of data generally show a performance regression. Increasing the file size parameter does improve the numbers, causing a few negatives to flip positive, but the overall trend remains. Based on these results, it appears that disabling Qualcomm’s thread migration boost feature has a greater impact on file encryption performance than adding more threads or using NEON instructions.

What this test fails to capture, however, is the storage performance in real-world scenarios. Reading and writing to storage does not happen in isolation. Instead, the encryption driver must fight threads from the active program, background tasks, operating system, user interface, etc. for CPU time. Having all four cores active all the time means more resources are available to the system and eliminates the latency involved in bringing the other cores online. In theory at least, storage performance in these noisy scenarios should at least be more consistent with the 5.1 update.

Even with the uplift in some scenarios, the Nexus 6 running Android 5.1 still trails the Galaxy Note 4 by a significant margin. If FDE is going to be a viable option, which it needs to be, then Google needs to resolve the issues surrounding full hardware encryption.

Battery Life

The changes discussed above will also affect battery life. Disabling the thread migration boost feature should provide a net power savings. Having all four cores active means less CPU pressure, so the governor can keep individual core frequencies lower on average, again reducing dynamic CPU power. Of course keeping those extra two cores online will increase static power draw due to leakage, offsetting some of the battery life gains.

Battery Benchmarks

Benchmark Nexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
GFXBench 3.0 Performance 19.5 fps 21.0 fps 7.40%
Lifetime 181 min 175.5 min -3.04%
PCMark Work Battery Life 329 min 354 min 7.45%

Battery life is generally difficult to quantify due to the huge variation in potential workloads, and becomes even more difficult when trying to assess the impact of the specific changes included in the 5.1 update. The best test we have for this is PCMark, which performs a few common tasks instead of purely synthetic loops. Here we see more than a 7% gain in battery life, which equates to a noticeable 25 minutes of runtime.

GPU And Gaming Performance

The 5.1 update brings support for OpenGL ES 3.1 to the Nexus 6, which was missing from the initial 5.0 release. Beyond this, I could not find any other changes that should affect GPU performance.

GPU Benchmarks

Benchmark Nexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
3DMark: Ice Storm Unlimited Score 23618 21253 -10.01%
Graphics 25651 22294 -13.09%
Physics 18489 18267 -1.20%
Basemark X: Medium Quality Performance 30315 26339 -13.12%
Dunes: Offscreen 32.82 fps 28.03 fps -14.61%
Dunes: Onscreen 23.65 fps 23.03 fps -2.64%
Hangar: Offscreen 44.45 fps 38.98 fps -12.31%
Hangar: Onscreen 34.24 fps 31.01 fps -9.43%
Basemark X: High Quality Performance 20825 19178 -7.91%
Dunes: Offscreen 29.04 fps 26.84 fps -7.59%
Dunes: Onscreen 20.56 fps 20.26 fps -1.46%
Hangar: Offscreen 25.76 fps 23.65 fps -8.17%
Hangar: Onscreen 17.38 fps 16.93 fps -2.59%
GFXBench 3.0 Manhattan Offscreen 17.0 fps 18.4 fps 8.89%
Manhattan Onscreen 12.0 fps 12.1 fps 0.75%
T-Rex Offscreen 37.0 fps 39.1 fps 5.47%
T-Rex Onscreen 26.5 fps 27.5 fps 3.88%
Alpha Blending Offscreen 10694 MB/s 11344 MB/s 6.08%
Alpha Blending Onscreen 9281 MB/s 9755 MB/s 5.10%
ALU Offscreen 140.5 fps 141.5 fps 0.75%
ALU Onscreen 59.5 fps 59.0 fps -0.06%
Driver Overhead Offscreen 25.0 fps 24 fps -3.28%
Driver Overhead Onscreen 22.0 fps 19.5 fps -11.01%
Fill Offscreen 7334 MTexels/s 7465 MTexels/s 1.79%
Fill Onscreen 8474 MTexels/s 8490 MTexels/s 0.19%
Render Quality: Standard 2503 mB PSNR 2503 mB PSNR 0.00%
Render Quality: High Precision 3633 mB PSNR 3628 mB PSNR -0.14%

The 5.1 update on the Nexus 6 reduces 3DMark and Basemark X performance by up to 15%. Even though these tests primarily stress the GPU, the CPU still plays a supporting role. To understand this better, we logged CPU and GPU frequency while running Basemark X.

As expected, the GPU runs at its max frequency of 600MHz during each test segment regardless of OS version. With the initial Android 5.0 build, the Nexus 6 keeps two cores offline and the other two above the 1497MHz threshold most of the time and at least one core at max frequency fairly often. CPU behavior with the 5.1 update looks very different. No single core runs at max frequency for very long, with cores taking turns jumping from 300MHz to 2649MHz and back down again. There’s also several gaps where all four cores are sitting at 300MHz. Looking at the raw data also shows that only one or two CPU cores are above idle at any given time, so having all four cores online does not provide any performance advantage in this particular benchmark.

Things aren’t all bad, however, as we see small performance improvements in the GFXBench Manhattan, T-Rex, and Alpha Blending tests, indicating that some games will see an uptick in performance.

Conclusion

The Android 5.1 update brings several improvements to the Nexus 6, including a number of bug fixes and UI refinements. It also enables OpenGL ES 3.1 support and the Qualcomm VP8 hardware video decoder. Unfortunately, it does not enable hardware based storage encryption or fix optical image stabilization for the rear camera. There’s also no improvement in display calibration.

The other Nexus 6 specific changes are a mixed bag. Utilizing NEON instructions for the encryption engine does speed up data reads for larger block sizes, but seems to hurt performance when reading and writing smaller data blocks. Disabling Qualcomm’s thread migration boost feature reduces performance by up to 20% in specific workloads where the CPU cores aren’t fully utilized. However, this did contribute to a noticeable gain in the PCMark battery life test.

Google Nexus 6

Read the Review

The final big change in the 5.1 update for the Nexus 6, keeping all four cores online all the time, is harder to evaluate. Typically, our phones perform a number of tasks in the background keeping us updated on events happening in the world. With only two cores generally available, which was the case when running the initial 5.0 build, these tasks could interrupt the foreground app or UI interaction, producing a noticeable lag and hurting the user experience. Having more cores available to service these background tasks should improve responsiveness. The downside is that static power drain is increased, reducing the battery life gained from disabling the thread migration boost, and the thermal/power envelope is decreased, which can keep cores from reaching max frequency and reducing performance.

In the end, the performance penalty seen in specific cases is not enough to outweigh the benefits from the 5.1 update. It’s important to have realistic expectations though. I don’t expect these changes to be permanent, but rather stepping stones on a path to a more efficient CPU governor.