AnTuTu releases an AI performance benchmark for smartphones

AI was the buzzword of 2018 with smartphone manufacturers boasting about their phones' AI capabilities making it really hard to determine if there are any major differences between different handsets. So AnTuTu decided to release a benchmarking tool, which will help weed out the "fake AI" phones and also let phones flex their AI muscles and compete.

However, in its blog post, AnTuTu stresses the fact that AI is a more complex matter than the current system performance benchmarks. TheAI Review benchmark tool has two sub-categories - Classification and Object Recognition.

The first test contains 200 images based on the Inception v3 neural network while the second test reviews 600-frame video based on the MobileNet SSD neural network. These images are then fueled into the SDK provided by the smartphone vendor and if the chip doesn't support AI algorithms, the benchmark resorts to the TFLite for benchmarking. However, the second scenario is vastly unreliable at the time.

The final score is formed by the speed and accuracy of the test, but if accuracy is traded off for speed, the benchmark will give a lower score. The logic behind is that this prevents cheating where the phone would read the images really fast without any right answers.

AnTuTu also points out that the AI score across different platforms won't differentiate that much because the tool focuses solely on AI performance and not overall system performance. Also, Samsung hasn't released its AI SDK while HiSilicon still uses TFLite for some functions, so those phones will score lower than they should for now. And lastly, the Android version might skew the results as Google is working on improving AI functions on a system level with each Android release.

The benchmark test is still early in its development phase, but you can always test it out for yourself. Keep in mind that real AI is kind of a new thing and the complexity of hardware and software solutions working together shouldn't be overlooked.

Source