Benchmark Rules and Guidelines

People rely on UL Solutions' benchmarks to produce accurate and unbiased results. We safeguard that trust by having clear rules for hardware manufacturers and software developers that specify how a platform must interact with our benchmark software. In simple terms:

A platform must run the benchmark without modification as if it were any other application.

Optimization policy

Benchmark specific optimizations are not allowed. Additionally, all generic optimizations that change the work specified by the benchmark are prohibited.

  • The platform may not change the quality level of the work.
  • The platform may not use an alternative technique to that requested by the workload.
  • The platform may not replace or remove any portion of the requested work even if the change would result in the same output.
  • Optimizations based on empirical data of benchmark workloads are not allowed.
  • Optimizations that change the output of the work are not allowed.

Application detection

The platform may not detect the launch of the benchmark executable. The platform must not alter, replace or override any parameters or parts of the test, nor modify the usual functioning of the platform based on the detection of the benchmark.

Exceptions
  • The platform may detect the benchmark application if it is the general approach for enabling specific platform features, such as multi-GPU use.
  • The platform may detect the benchmark application if there is a known conflict between the benchmark application and the platform that would otherwise prevent the benchmark from functioning correctly. This must be pre-approved with UL Solutions.

Delisted devices

UL Solutions' Benchmarks website helps you compare CPUs, GPUs, smartphones, and tablets by ranking models by performance. When a manufacturer breaks the rules to boost the ranking of a device it is unfair and misleading.

We delist devices that do not comply with our rules. We do this to encourage all manufactures and operators to be fair and honest with their customers.

Delisted models appear unranked, and without scores, at the bottom of our hardware rankings. Scores from delisted devices should not be used to compare devices.

Report a device

Press reviewers can report models with suspect scores by emailing us.Please include evidence to back up your claim.

Right to appeal

Manufacturers of delisted devices are invited to contact us if they believe a model has been unfairly delisted, or if they have updated their device software to comply with our benchmark rules.

  • Which models have been delisted?

    Delisted devices appear without scores at the bottom of our best smartphones and best tablets lists.

  • Why have you delisted these models?

    When a device modifies its behavior as a result of detecting 3DMark, its benchmark scores are no longer accurate and cannot be used to make fair comparisons between devices. Delisting is a last resort when we have evidence that a manufacturer is breaking our rules.

  • But Company X says it is not cheating?

    Whatever their intentions, manufacturers must follow our rules to get a valid benchmark score. Break the rules, no score. It's as simple as that. Manufacturers can contact us if they believe a device has been unfairly delisted.

  • Isn't the point of benchmarking to measure maximum performance?

    Outside of industry, there is little value in measuring theoretical maximum performance. Wouldn't you rather know how a device performs while running apps and games? 3DMark measures real-world performance, which includes the effects of CPU scaling, thermal throttling, and how the device chooses to balance power and battery life.

  • I'm not cheating, why delist my score?

    Delisting does not mean that you have done anything wrong. The fault usually lies with the operator or manufacturer of your device.

  • My phone is stock, why is it delisted?

    We are focused on identifying and delisting stock device models with factory settings that break our rules. All results from those specific models, modded or stock, are affected by the delisting.

  • So overclocking isn't allowed now?

    Overclocking by manufacturers is allowed provided that it applies equally to all apps, all of the time. Overclocking optimizations that are selectively applied to our benchmarks are forbidden.

  • If I root my phone, will it be delisted?

    No. Rooting, modifying, and tweaking your device is part of the fun of benchmarking. We have no plans to delist devices based on results from rooted, modded or jail-broken devices.

  • How can I get a valid score?

    Delisted devices will require updates from the manufacturer or operator before they can get valid scores. You should contact the manufacturer and operator of your device and show them this page.

  • Why are some devices with MediaTek processors delisted?

    These devices automatically use a hidden "Sports Mode" and other app-specific optimizations when running PCMark for Android. They do not do this with a renamed, but otherwise identical version of the app, which means PCMark cannot measure their real-world performance.

  • Why are Huawei P20 Pro, Huawei P20, Huawei Nova 3, and Huawei Honor Play delisted?

    These devices automatically use a hidden "Performance Mode" that overrides the devices' usual power profile. They do not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure their real-world performance.

  • Why are the Oppo Find X and Oppo F7 delisted?

    These models automatically run at full speed when running the 3DMark app. They do not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure their real-world performance.

  • Why are HTC One (M7), HTC One Mini, Samsung Galaxy Note 10.1, and HP Slatebook 10 x2 delisted?

    These devices detect 3DMark, then set their CPU clocks to max, even when 3DMark is idle in the UI. They do not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure their real-world performance.

  • Why is Huawei Ascend P7 delisted?

    This model boosts its performance when running 3DMark, but not when running a renamed, but otherwise identical version of the app.

  • Why is HTC One (M8) delisted?

    This device automatically enables a special high performance mode when it detects 3DMark. It does not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure its real-world performance.

  • Why are some versions of Samsung's Galaxy S IV and Galaxy Note III delisted?

    With Android 4.3 these devices detect 3DMark, then set their CPU clocks to max. They do not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure their real-world performance. Use the official Android 4.4.2 update or later to get a valid 3DMark score for these devices.

Privacy Policy  |  Cookie Policy  |  EULA