Recently, Ali officially opened up the lightweight in-depth learning end-to-side reasoning engine
MNN is aLightweight Deep Learning End-to-side Reasoning EngineThe core of this paper is to solve the end-to-side reasoning problem of the deep neural network model, which covers the optimization, transformation and reasoning of the deep neural network model.
Officials hope that MNN can smooth out the differences between Android and iOS, debris devices and different training frameworks, achieve rapid end-to-side deployment and operation, and be able to add OP flexibly according to business models and optimize the performance of heterogeneous devices such as CPU/GPU.
Delta MNN architecture design, divided into Converter and Interpreter two parts
Specifically, MNN is responsible for loading the network model, reasoning and predicting the return of relevant results. The whole reasoning process can be divided into loading and parsing of models, scheduling of computational graphs, and efficient operation on heterogeneous backends. MNN has four characteristics:
Generality:Support Tensorflow, Caffe, ONNX and other mainstream model file formats, support CNN, RNN, GAN and other common networks;
Lightweight:Depth customization and tailoring can be easily deployed to mobile devices and embedded devices without any dependence.
High performance:It does not depend on any third-party computing library, and relies on a large number of handwritten assembly to realize core operations, giving full play to the computing power of ARM CPU.
Ease of use:With perfect documents and examples, and efficient image processing module, covering common deformation, transformation and other needs. Supporting callback mechanism to facilitate data extraction or control the direction of operation.
As for the advantages of MNN, Jia Yangqing, an AI scientist who has joined Ali, said:
Compared with Tensorflow, Caffe2 and other general frameworks covering both training and reasoning, MNN pays more attention to the acceleration and optimization of reasoning, and solves the efficiency problem in the phase of model deployment, so as to realize the business behind the model more efficiently on the mobile side.
This means that MNN and reasoning engines such as server-side TensorRT are in line. In large-scale machine learning applications, considering the large-scale model deployment, the computational load of reasoning side of machine learning is often more than ten times that of training side, so the optimization of reasoning side is particularly important.
Comparing the performance of frameworks such as MNN and Cafe2, there are more than 20% advantages.
At present, MNN has been used in more than 20 Apps, such as Hand-Tao, Handcat, Youku, Cost-effective, UC, etc. It covers live broadcasting, short video, search recommendation, commodity image search and other scenarios, and runs steadily hundreds of millions of times a day. In addition, there are several applications in scenarios such as IoT.The official plan is to release a stable version every two months.
Reference resources:MNN Project IntroductionAli Technology