MediaTek Brings Neural Networks to Devices

In just the past two years, the industry has made great strides in artificial intelligence (AI) using artificial neural networks, better known as deep learning. With massive processing resources, massive amounts of data, and a software framework, a network of filters is created and optimized to perform select functions like image recognition. As the neural network learns, it develops models that can then be used by computing solutions with much less processing resources to perform the desired function on similar data. This is typically referred to as an inference engine or solution, which can be common processing elements like Central Processing Units (CPUs) and Graphics Processing Units (GPUs) or custom processing solutions.

Most artificial intelligent applications, like Amazon Echo using Alexa, perform all or part of the processing in the cloud. While this is satisfactory for solutions that are always connected to power and a network, there are many other applications, particularly consumer applications, that require local processing. Local processing also offers addition security of the data because it not be transferred over a network and potentially accessible to other parties. Like many others in the industry, MediaTek is looking to make local neural net data processing a reality.

First announced with the Helio X20, MediaTek developed what it calls the Deep Learning SDK for developers. The SDK is optimized to work with the TensorFlow and Caffe deep learning frameworks. Thus far, MediaTek has worked with undisclosed partners on developing solutions for image and voice recognition that leverage the heterogenous computing capabilities of the Helio X family of mobile SoCs. The Helio X family features ten CPU Cores, combined with GPU cores, Image Signal Processing (ISP) cores, and Digital Signal Processing (DSP) cores. Working in conjunction with MediaTek’s CorePilot technology, the various compute resources are used efficiently to perform the required tasks. Ideally, the local solution should be able to continue learning and modifying the models being used, but it does not appear that the MediaTek platform, like most other device platforms, allow for continued learning on the device without the aid of cloud-based resource. However, with so many compute resources on a single chip, on-device learning may not be far away.

MediaTek

Deep Learning SDK (Source: MediaTek)

In using the MediaTek Deep Learning SDK in conjunction with a Helio X20 or Helio X30 SoC, developers can enable local AI functions on a variety of platforms. Right now, the target products are high-end smartphones, but the platform could easily be leveraged for other applications, such as home entertainment or automotive infotainment. Additionally, with the SDK, developers can enable many AI applications using cloud-based deep learning resources to develop the necessary models. These include natural language processing, image and video processing, security, digital assistants, and many other applications. While MediaTek is supporting continued enhancements to the SDK, it is not providing the deep learning data center resources for development. As a result, this may be an opportunity for other data center service providers

While the MediaTek AI platform consisting of the Helio X family mobile SoCs and the Deep Learning SDK is not the ultimate on-chip AI solution, it is an important step toward device-level intelligence and will enable many consumers to experience new levels of AI in the palm of their hand.