Oral
MNN: A Universal and Efficient Inference Engine
Xiaotang Jiang · Huan Wang · Yiliu Chen · Ziqi Wu · Lichuan Wang · Bin Zou · Yafeng Yang · Zongyang Cui · Yu Cai · Tianhang Yu · Chengfei Lyu · Zhihua Wu

Mon Mar 2nd 03:20 -- 03:45 PM @ Ballroom A

Deploying deep learning (DL) models on mobile devices draws more and more attention recently. However, designing an efficient inference engine on devices is under the great challenges of model compatibility, device diversity, and resource limitation. To deal with these challenges, we propose Mobile Neural Network (MNN), a universal and efficient inference engine tailored to mobile applications. In this paper, the contributions of MNN include: (1) presenting a mechanism called pre-inference that manages to conduct runtime optimization; (2) delivering thorough kernel optimization on operators to achieve optimal computation performance; (3) introducing backend abstraction module which enables hybrid scheduling and keeps the engine lightweight. Extensive benchmark experiments demonstrate that MNN performs favorably against other popular lightweight DL frameworks.

Author Information

Xiaotang Jiang (Alibaba)
Huan Wang (Northeastern University)
Yiliu Chen (Alibaba Group)
Ziqi Wu (Alibaba Group)
Lichuan Wang (Alibaba Group)
Bin Zou (alibaba)
Yafeng Yang (Alibaba Group)
Zongyang Cui (Alibaba Group)
Yu Cai (Alibaba Group)
Tianhang Yu (Alibaba Group)
Chengfei Lyu (Alibaba Group)
Zhihua Wu (Alibaba)

Related Events (a corresponding poster, oral, or spotlight)