Timezone: »

 
Workshop
Benchmarking Machine Learning Workloads on Emerging Hardware
Tom St John · Murali Emani

Wed Mar 04 07:00 AM -- 03:30 PM (PST) @ Level 3 Room 6
Event URL: https://memani1.github.io/challenge20/ »

With evolving system architectures, hardware and software stacks, diverse machine learning (ML) workloads, and data, it is important to understand how these components interact with each other. Well-defined benchmarking procedures help evaluate and reason the performance gains with ML workload-to-system mappings. We welcome all novel submissions in benchmarking machine learning workloads from all disciplines, such as image and speech recognition, language processing, drug discovery, simulations, and scientific applications. Key problems that we seek to address are: (i) which representative ML benchmarks cater to workloads seen in industry, national labs, and interdisciplinary sciences; (ii) how to characterize the ML workloads based on their interaction with hardware; (iii) which novel aspects of hardware, such as heterogeneity in compute, memory, and networking, will drive their adoption; (iv) performance modeling and projections to next-generation hardware. Along with selected publications, the workshop program will also have experts in these research areas presenting their recent work and potential directions to pursue.

Call for Papers can be found here:
https://memani1.github.io/challenge20/

Paper Submission Deadline: January 15, 2020
Author Notification: January 27, 2020
Camera-Ready Papers Due: February 21, 2020

Wed 7:00 a.m. - 7:10 a.m. [iCal]
Introduction - Tom St. John (Tesla Inc.) (Presentation)
Wed 7:10 a.m. - 7:50 a.m. [iCal]
MLPerf Inference Deep Dive - Vijay Janapa Reddi (Harvard University) (Keynote Address)
Wed 8:00 a.m. - 8:30 a.m. [iCal]
Morning Break (Break)
Wed 8:30 a.m. - 9:20 a.m. [iCal]

Precious: Resource-Demand Estimation for Embedded Neural Network Accelerators - Stefan Raif (FAU Erlangen-Nürnberg), Benedict Herzog (FAU Erlangen-Nürnberg), Judith Hemp (FAU Erlangen-Nürnberg), Timo Hönig (FAU Erlangen-Nürnberg), Wolfgang Schröder-Preikschat (FAU Erlangen-Nürnberg)

Benchmarking Machine Learning Workloads in Structural Bioinformatics Applications - Heng Ma (Argonne National Laboratory), Austin Clyde (Argonne National Laboratory), Venkatram Vishwanath (Argonne National Laboratory), Debsindhu Bhowmik (Oak Ridge National Laboratory), Arvind Ramanathan (Argonne National Laboratory), Shantenu Jha (Rutgers University, Brookhaven National Laboratory)

Benchmarking Alibaba Deep Learning Applications Using AI Matrix - Wei Zhang (Alibaba Group), Wei Wei (Alibaba Group), Lingjie Xu (Alibaba Group), Lingling Jin (Alibaba Group)

Wed 9:20 a.m. - 10:00 a.m. [iCal]
Formula One vs. Family Car, or the Need for Broader, Generalizable Benchmarks - Natalia Vassilieva (Cerebras Systems) (Invited Talk)
Wed 10:00 a.m. - 12:00 p.m. [iCal]
Lunch (Break)
Wed 12:00 p.m. - 12:40 p.m. [iCal]
Benchmarking Science: Datasets and Exascale Infrastructure - Geoffrey Fox (Indiana University) (Invited Talk)
Wed 12:45 p.m. - 1:30 p.m. [iCal]

Deep Learning Workload Performance Auto-Optimizer - Connie Yingyu Miao (Intel Corporation), Andrew Yang (Intel Corporation), Michael Anderson (Intel Corporation)

Challenges with Evaluating ML Solutions in Data Centers - Shobhit Kanaujia (Facebook), Wenyin Fu (Facebook), Abhishek Dhanotia (Facebook)

Benchmarking TinyML Systems: Challenges and Direction - Colby Banbury (Harvard University), Vijay Janapa Reddi (Harvard University), Will Fu (Harvard University), Max Lam (Harvard University), Amin Fazel (Samsung Semiconductor Inc.), Jeremy Holleman (Syntiant, University of North Carolina Charlotte), Xinyuan Huang (Cisco Systems), Robert Hurtado (HurtadoTechnology Inc.), David Kanter (Real World Insights), Anton Lokhmotov (dividiti), David Patterson (University of California Berkeley, Google), Danilo Pau (STMicroelectronics), Jeff Sieracki (Reality AI), Jae-Sun Seo (Arizona State University), Urmish Thakkar (Arm), Marian Verhelst (KU Leuven, Imec), Poonam Yadav (University of York)

Wed 1:30 p.m. - 2:00 p.m. [iCal]
Afternoon Break (Break)
Wed 2:00 p.m. - 3:20 p.m. [iCal]
Panel - Peter Mattson (Google), Shuaiwen Song (University of Sydney), Gennady Pekhimenko (University of Toronto), Carole-Jean Wu (Facebook, Arizona State University), Grigori Fursin (CodeReef), Ramesh Radhakrishnan (Dell) (Discussion Panel)
Wed 3:20 p.m. - 3:30 p.m. [iCal]
Conclusion - Murali Emani (Argonne National Laboratory) (Presentation)

Author Information

Tom St John (Tesla)
Murali Emani (Argonne National Laboratory)

Murali Emani is an Assistant Computer Scientist in the Data Science group with the Argonne Leadership Computing Facility (ALCF) at Argonne National Laboratory. His research interests are at the intersection of systems and machine learning including Parallel programming models, Hardware accelerators for ML/DL, High Performance Computing, Scalable Machine Learning, Runtime Systems, Performance optimization, Emerging HPC architectures, Online Adaptation. Prior, he was a Postdoctoral Research Staff Member at Lawrence Livermore National Laboratory, US. Murali obtained his PhD and worked as a Research Associate at the Institute for Computing Systems Architecture at the School of Informatics, University of Edinburgh, UK. His research resulted in multiple publications at top conferences such as PACT, PLDI and granted patents. Murali served as technical program committee member for conferences including ICPP'19, CCGRID'19, PACT '18, CCGRID '18, ICPP '18. He chaired the first Birds-of-feather session on Machine Learning benchmarking on HPC systems at Supercomputing 2019.

More from the Same Authors

  • 2020 Oral: MLPerf Training Benchmark »
    Peter Mattson · Christine Cheng · Gregory Diamos · Cody Coleman · Paulius Micikevicius · David Patterson · Hanlin Tang · Gu-Yeon Wei · Peter Bailis · Victor Bittorf · David Brooks · Dehao Chen · Debo Dutta · Udit Gupta · Kim Hazelwood · Andy Hock · Xinyuan Huang · Daniel Kang · David Kanter · Naveen Kumar · Jeffery Liao · Deepak Narayanan · Tayo Oguntebi · Gennady Pekhimenko · Lillian Pentecost · Vijay Janapa Reddi · Taylor Robie · Tom St John · Carole-Jean Wu · Lingjie Xu · Cliff Young · Matei Zaharia
  • 2020 Poster: MLPerf Training Benchmark »
    Peter Mattson · Christine Cheng · Gregory Diamos · Cody Coleman · Paulius Micikevicius · David Patterson · Hanlin Tang · Gu-Yeon Wei · Peter Bailis · Victor Bittorf · David Brooks · Dehao Chen · Debo Dutta · Udit Gupta · Kim Hazelwood · Andy Hock · Xinyuan Huang · Daniel Kang · David Kanter · Naveen Kumar · Jeffery Liao · Deepak Narayanan · Tayo Oguntebi · Gennady Pekhimenko · Lillian Pentecost · Vijay Janapa Reddi · Taylor Robie · Tom St John · Carole-Jean Wu · Lingjie Xu · Cliff Young · Matei Zaharia