Machine Learning Inference Performance

AIMark 3

AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

Unfortunately for the ROG Phone II, it’s another device that’s lacking the proper Qualcomm proprietary libraries that AI Mark makes use of, ending up not being in the charts anywhere as the application couldn’t start.

AIBenchmark 3

AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.

We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.

AIBenchmark 3 - NNAPI CPU

We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.

AIBenchmark 3 - 1 - The Life - CPU/FP AIBenchmark 3 - 2 - Zoo - CPU/FP AIBenchmark 3 - 3 - Pioneers - CPU/INT AIBenchmark 3 - 4 - Let's Play - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/INT AIBenchmark 3 - 8 - Blur iT! - CPU/FP

We performed the AI Benchmark tests in the default mode of the phone, and here the ROG Phone II lands relatively in average to better than average amongst S855 devices.

AIBenchmark 3 - NNAPI INT8

AIBenchmark 3 - 1 - The Life - INT8 AIBenchmark 3 - 2 - Zoo - Int8 AIBenchmark 3 - 3 - Pioneers - INT8 AIBenchmark 3 - 5 - Masterpiece - INT8 AIBenchmark 3 - 6 - Cartoons - INT8

AIBenchmark 3 - NNAPI FP16

AIBenchmark 3 - 1 - The Life - FP16 AIBenchmark 3 - 2 - Zoo - FP16 AIBenchmark 3 - 3 - Pioneers - FP16 AIBenchmark 3 - 5 - Masterpiece - FP16 AIBenchmark 3 - 6 - Cartoons - FP16 AIBenchmark 3 - 9 - Berlin Driving - FP16 AIBenchmark 3 - 10 - WESPE-dn - FP16

AIBenchmark 3 - NNAPI FP32

AIBenchmark 3 - 10 - WESPE-dn - FP32

The same conclusion can be had of the INT8, FP16 and FP32 scores. It’s odd that the FP32 scores aren’t better than that of the competition as the workload is accelerated by the GPU, and we had expected the S855+’s higher GPU frequency to have a larger impact on this score.

We also have to note that if it weren’t for a very recent firmware update from ASUS we wouldn’t be able to publish any AI benchmarks at all, as it previously lacked the proper NNAPI drivers. As such, it seems that this aspect was pretty much just an afterthought for the phone rather than something that was on a critical to-do list.

System Performance GPU Performance - Top Performance
Comments Locked

75 Comments

View All Comments

  • plewis00 - Monday, September 30, 2019 - link

    I don’t know what games you play but the ones I do can’t cope with multiple connections - if they detect a network or IP change they usually treat that as a disconnection. So you’d need the game companies to get onboard too. I suspect the IP change thing is to do with your session being intercepted and that’s why it disconnects you.
  • brucethemoose - Monday, September 30, 2019 - link

    Asus could theoretically host a VPN (or contract someone else) to keep your visible IP the same across all connections.

    That would also be MUCH less work than mucking around with Android's network handoff code.
  • Lochtror - Tuesday, October 1, 2019 - link

    good idea, i think just any vpn usage would have this effect, so i will try that. but still needs some setting or software on the phone that switches the connection faster than what i experienced so far.
  • Lochtror - Tuesday, October 1, 2019 - link

    interesting. thanks. its mushroom wars 2 and hearthstone for me. hearthstone is forgiving regarding connection issues. can even reconnect for a while. turn based game.

    mw2, real time, more disconnects.
  • Wardrive86 - Monday, September 30, 2019 - link

    I buy a smartphone with gaming specifically in mind. There now you know one person who does that 😏
  • AdhesiveTeflon - Monday, September 30, 2019 - link

    I don't get it, is somebody planning to load up battlefield V on this thing and play it?
  • PeachNCream - Tuesday, October 1, 2019 - link

    I think we're heading in that direction and these early entries into adding external case plastics and lighting that might appeal to the target audience are indications that some PC manufacturers are seeing the need to enter the phone reseller and/or production market in order to establish corporate identities ahead of other potential competitors. When it comes to playing games, the end objective is amusement and as we move forward, the underlying device that provides said amusement is certainly subject to change. Look at how we have transitioned with other elements of entertainment and information consumption for examples. Families used to gather around a single, large radio console to listen to evening programming together. They eventually shifted to doing the same thing with television consoles. Television became more commonplace and more personal so more of them started popping up in individualized spaces as cost of ownership declined so they ended up in bedrooms in addition to centralized areas. Computers went the same way, from single PC per household to personalized devices and now that same data processing is moving from larger to smaller, even more portable systems via phones that will naturally supply a certain amount of entertainment value via video games. It's just the way of things that change is constantly afoot.
  • johansyren - Tuesday, November 10, 2020 - link

    I've been using this phone for a week now and as a daily driver, it is great. In terms of photography, it is not that great but when using it with GCAM, it improves your shots. It is a gaming phone so the camera is good overall. As for the games, almost all games can run on ultra high settings and still run in a smooth manner. Don't worry about the temps going high 'cause if you bought the strix edition one, it comes with aero cooler v2 and aero case. I found a good article about this phone https://casinovalley.ca/best-picks/best-gambling-p...
  • Tams80 - Wednesday, October 2, 2019 - link

    Not Battlefield V or it's like, but numerous people have made careers out of mobile gaming (and not just game developers).
  • Lord of the Bored - Monday, September 30, 2019 - link

    "Game Genie"

    How nostalgic.

Log in

Don't have an account? Sign up now