Timezone: »

 
DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference
Udit Gupta

Fri Apr 09 08:45 AM -- 09:00 AM (PDT) @
Event URL: http://vlsiarch.eecs.harvard.edu/wp-content/uploads/2020/05/DeepRecSys_Gupta_ISCA2020.pdf »

Neural personalized recommendation is the corner-stone of a wide collection of cloud services and products, constituting significant compute demand of the cloud infrastructure. Thus, improving the execution efficiency of neural recommendation directly translates into infrastructure capacity saving. In this paper, we devise a novel end-to-end modeling infrastructure, DeepRecInfra, that adopts an algorithm and system co-design methodology to custom-design systems for recommendation use cases. Leveraging the insights from the recommendation characterization, a new dynamic scheduler, DeepRecSched, is proposed to maximize latency-bounded throughput by taking into account characteristics of inference query size and arrival patterns, recommendation model architectures, and underlying hardware systems. By doing so, system throughput is doubled across the eight industry-representative recommendation models. Finally, design, deployment, and evaluation in at-scale production datacenter shows over 30% latency reduction across a wide variety of recommendation models running on hundreds of machines.

Author Information

Udit Gupta (Harvard University)

More from the Same Authors

  • 2021 : Closing session »
    Udit Gupta · Carole-Jean Wu
  • 2021 : Closing remarks »
    Udit Gupta · Lillian Pentecost · Abhishek Gupta · Mayoore Jaiswal · Shagun Sodhani
  • 2021 : Industry/Academia Panel »
    Zachary C Lipton · Udit Gupta · Lillian Pentecost · Shagun Sodhani · Abhishek Gupta · Mayoore Jaiswal · Michael Carbin · Devi Parikh · Gennady Pekhimenko
  • 2021 Workshop: Journal of Opportunities, Unexpected limitations, Retrospectives, Negative results, and Experiences »
    Abhishek Gupta · Udit Gupta · Mayoore Jaiswal · Lillian Pentecost · Shagun Sodhani · David Brooks · Joelle Pineau
  • 2021 : Welcome to JOURNE »
    Udit Gupta · Lillian Pentecost · Mayoore Jaiswal · Abhishek Gupta · Shagun Sodhani
  • 2021 Workshop: Personalized Recommendation Systems and Algorithms »
    Udit Gupta · Carole-Jean Wu · Gu-Yeon Wei · David Brooks
  • 2021 : Welcome to the 3rd PeRSonAl workshop »
    Udit Gupta · Carole-Jean Wu
  • 2020 Oral: MLPerf Training Benchmark »
    Peter Mattson · Christine Cheng · Gregory Diamos · Cody Coleman · Paulius Micikevicius · David Patterson · Hanlin Tang · Gu-Yeon Wei · Peter Bailis · Victor Bittorf · David Brooks · Dehao Chen · Debo Dutta · Udit Gupta · Kim Hazelwood · Andy Hock · Xinyuan Huang · Daniel Kang · David Kanter · Naveen Kumar · Jeffery Liao · Deepak Narayanan · Tayo Oguntebi · Gennady Pekhimenko · Lillian Pentecost · Vijay Janapa Reddi · Taylor Robie · Tom St John · Carole-Jean Wu · Lingjie Xu · Cliff Young · Matei Zaharia
  • 2020 Poster: MLPerf Training Benchmark »
    Peter Mattson · Christine Cheng · Gregory Diamos · Cody Coleman · Paulius Micikevicius · David Patterson · Hanlin Tang · Gu-Yeon Wei · Peter Bailis · Victor Bittorf · David Brooks · Dehao Chen · Debo Dutta · Udit Gupta · Kim Hazelwood · Andy Hock · Xinyuan Huang · Daniel Kang · David Kanter · Naveen Kumar · Jeffery Liao · Deepak Narayanan · Tayo Oguntebi · Gennady Pekhimenko · Lillian Pentecost · Vijay Janapa Reddi · Taylor Robie · Tom St John · Carole-Jean Wu · Lingjie Xu · Cliff Young · Matei Zaharia