On-Device NLP at Facebook (Ahmed Aly and Kshitiz Malik, Facebook)

Deep-learning based models have revolutionized many NLP tasks (e.g. Translation, Conversational AI, Language Modeling). There is a growing need to perform these tasks on low-resource electronic devices (e.g. mobile phones, tablets, wearables) for privacy and latency reasons. However, the large computational and memory demands of deep neural networks make it difficult to deploy them on-device as-is. They usually require significant optimizations and sometimes major model architecture changes to fit under tight memory and compute budgets.

In this talk we will share the work that Facebook is doing to bring these NLP models to user devices. We will talk about efficient building blocks and model architectures that find the right balance between model quality and compute/memory requirements on multiple NLP tasks. Finally, we will outline the biggest challenges and open problems in shipping on-device NLP models at Facebook scale.

Biography Ahmed Aly is an Engineering Manager on the AI Assistant team in Facebook Reality Labs. He leads the Language understanding team, building efficient intent understanding and semantic parsing models that power Facebook’s Conversational AI systems. Prior to this, he was the founder and tech-lead of the PyText platform. Ahmed has a Masters degree in Computational Linguistics from University of Washington and a B.E. in Computer Engineering from Cairo University.

Kshitiz Malik is a Software Engineer on the AI Assistant team in Facebook Reality Labs. He works on Privacy Preserving Machine Learning, Natural Language Understanding and Natural Language Generation. Kshitiz has a PhD in Electrical and Computer Engineering from University of Illinois at Urbana-Champaign, and a B.E in Computer Engineering from University of Delhi