“Shamsung” Strikes Again (or: The Galaxy Note 3 is Programmed to “Game” Benchmarking Apps)

When the reviewers at Ars Technica noticed that Samsung’s Galaxy Note 3 “phablet” scored strikingly better than LG’s G2, despite having the exact same processor — a 2.3 GHz Snapdragon 800 processor — they decided to dig a little deeper. Their conclusion:

After a good bit of sleuthing, we can confidently say that Samsung appears to be artificially boosting the US Note 3’s benchmark scores with a special, high-power CPU mode that kicks in when the device runs a large number of popular benchmarking apps.

This isn’t the first time that Samsung has been caught artificially boosting benchmark results, but Ars Technica reports that this is the first time it’s been observed in a Samsung device shipped to the U.S.. Samsung defends the practice by saying that they boost performance for other apps as well.

In their investigation, Ars Technica did a fair bit of software disassembly and found the code that was actually boosting the speed, shown below. Remember that names and labels often get mangled in the compilation process, and that in the decompilation process, one often has to specify new names and labels, so while the logic of the code will be the same as the original source, the variable and constant names might not be:

{
BOARD_PLATFORM = SystemProperties.get("ro.board.platform");
mToken = 0;
PACKAGES_FOR_LCD_FRAME_RATE_ADJUSTMENT = new PackageInfo[0];
isEngBinary = "eng".equals(Build.TYPE);
PackageInfo[] arrayOfPackageInfo = new PackageInfo[26];
arrayOfPackageInfo[0] = new PackageInfo("com.aurorasoftworks.quadrant.ui.standard", false);
arrayOfPackageInfo[1] = new PackageInfo("com.aurorasoftworks.quadrant.ui.advanced", false);
arrayOfPackageInfo[2] = new PackageInfo("com.aurorasoftworks.quadrant.ui.professional", false);
arrayOfPackageInfo[3] = new PackageInfo("com.redlicense.benchmark.sqlite", false);
arrayOfPackageInfo[4] = new PackageInfo("com.antutu.ABenchMark", false);
arrayOfPackageInfo[5] = new PackageInfo("com.greenecomputing.linpack", false);
arrayOfPackageInfo[6] = new PackageInfo("com.greenecomputing.linpackpro", false);
arrayOfPackageInfo[7] = new PackageInfo("com.glbenchmark.glbenchmark27", false);
arrayOfPackageInfo[8] = new PackageInfo("com.glbenchmark.glbenchmark25", false);
arrayOfPackageInfo[9] = new PackageInfo("com.glbenchmark.glbenchmark21", false);
arrayOfPackageInfo[10] = new PackageInfo("ca.primatelabs.geekbench2", false);
arrayOfPackageInfo[11] = new PackageInfo("com.eembc.coremark", false);
arrayOfPackageInfo[12] = new PackageInfo("com.flexycore.caffeinemark", false);
arrayOfPackageInfo[13] = new PackageInfo("eu.chainfire.cfbench", false);
arrayOfPackageInfo[14] = new PackageInfo("gr.androiddev.BenchmarkPi", false);
arrayOfPackageInfo[15] = new PackageInfo("com.smartbench.twelve", false);
arrayOfPackageInfo[16] = new PackageInfo("com.passmark.pt_mobile", false);
arrayOfPackageInfo[17] = new PackageInfo("se.nena.nenamark2", false);
arrayOfPackageInfo[18] = new PackageInfo("com.samsung.benchmarks", false);
arrayOfPackageInfo[19] = new PackageInfo("com.samsung.benchmarks:db", false);
arrayOfPackageInfo[20] = new PackageInfo("com.samsung.benchmarks:es1", false);
arrayOfPackageInfo[21] = new PackageInfo("com.samsung.benchmarks:es2", false);
arrayOfPackageInfo[22] = new PackageInfo("com.samsung.benchmarks:g2d", false);
arrayOfPackageInfo[23] = new PackageInfo("com.samsung.benchmarks:fs", false);
arrayOfPackageInfo[24] = new PackageInfo("com.samsung.benchmarks:ks", false);
arrayOfPackageInfo[25] = new PackageInfo("com.samsung.benchmarks:cpu", false);
PACKAGES_FOR_BOOST_ALL_ADJUSTMENT = arrayOfPackageInfo;
mCameraCPUBooster = null;
mCameraCPUCoreNumBooster = null;
mCPUFrequencyTable = null;
mCPUCoreTable = null;
mRotationCPUCoreNumBooster = null;
mRotationGPUBooster = null;
}

The contents of the array arrayOfPackageInfo are the identifiers of well-known and often-used apps for benchmarking the performance of Android devices. The end result of this check is that it’s used by other code to alter CPU behaviour so that:

• If an app that has any one of these identifiers is running, the Note 3 stays in a mode with all 4 of its CPU cores active and running at full-speed 2.3 GHz mode. This special behaviour yields excellent benchmark results.
• If no such app is running, the Note 3 follows its default behaviour, which is to switch to a one-core-only energy-saving mode, with that single core running at 300 MHz. This normal behaviour extends battery life to something reasonable.

With such artificial boosting, the Note 3 gets impressive results. But by compiling Geekbench under a different name and thereby bypassing identification by the code above, Ars Technica were able to get more honest results:

Ars Technica astutely note:

The ironic thing is that even with the benchmark booster disabled, the Note 3 still comes out faster than the G2 in this test. If the intent behind the boosting was simply to ensure that the Note 3 came out ahead in the benchmark race, it doesn’t appear to have been necessary in the first place.

It’s a little crass, it’s a little desperate, and it’s…well, so, so Samgsung-y. That’s a real shame, because they do produce good stuff (I myself count a Galaxy S3 and a number of Samsung monitors among my devices).