Benchmarking Machine Learning Workloads on Emerging Hardware

Tom St John, Murali Emani


"With evolving system architectures, hardware and software stack, diverse machine learning workloads, and data, it is important to understand how these components interact with each other. Well-defined benchmarking procedures help evaluate and reason the performance gains with ML workload to system mapping.

Key problems that we seek to address are: (i) which representative ML benchmarks cater to workloads seen in industry, national labs, and interdisciplinary sciences; (ii) how to characterize the ML workloads based on their interaction with hardware; (iii) what novel aspects of hardware, such as heterogeneity in compute, memory, and bandwidth, will drive their adoption; (iv) performance modeling and projections to next-generation hardware.

The workshop will invite experts in these research areas to present recent work and potential directions to pursue. Accepted papers from a rigorous evaluation process will present state-of-the-art research efforts. A panel discussion will foster an interactive platform for discussion between speakers and the audience."

Chat is not available.

Timezone: »


Fri 8:00 a.m. - 8:10 a.m.
Introduction - Tom St. John (Cruise) (Introduction)
Tom St John
Fri 8:10 a.m. - 8:50 a.m.
"System-Level Design of Machine Learning Accelerators with the Open Source ESP Platform - Luca Carloni (Columbia) (Invited Talk)
Luca Carloni
Fri 8:50 a.m. - 9:00 a.m.
Morning Break 1 (Break)
Fri 9:00 a.m. - 9:40 a.m.
"Designing and Optimizing AI Systems for Deep Learning Recommendation and Beyond" - Carole-Jean Wu (Facebook) (Invited Talk)
Carole-Jean Wu
Fri 9:40 a.m. - 10:00 a.m.
Morning Break 2 (Break)
Fri 10:00 a.m. - 10:30 a.m.
"Being-Ahead: Benchmarking and Exploring Accelerators for Hardware-Efficient AI Deployment" - Xiaofan Zhang (UIUC) (Paper)
Xiaofan Zhang
Fri 10:30 a.m. - 10:50 a.m.
"Benchmarking Machine Learning Inference in FPGA-Based Accelerated Space Applications" - Amir Raoofy (TU Munich) (Paper)
Fri 10:50 a.m. - 11:30 a.m.
"Machine Learning Tools: Skyline and RL-Scope" - Gennady Pekhimenko and James Gleeson (University of Toronto) (Invited Talk)
Gennady Pekhimenko
Fri 11:30 a.m. - 1:30 p.m.
Lunch (Break)
Fri 1:30 p.m. - 2:10 p.m.
"Benchmarking Scientific ML on Disaggregated Cognitive Simulation HPC Platforms" - Brian Van Essen (LLNL) (Invited Talk)
Brian Van Essen
Fri 2:10 p.m. - 2:50 p.m.
"Challenges with Running DNN Workloads with Hardware Simulators" - David Kaeli (Northeastern University) and "GNNMark: a Benchmark for GNN Training" - Trinayan Baruah (Northeastern University) (Invited Talk)
Dave Kaeli
Fri 2:50 p.m. - 3:20 p.m.
Afternoon Break (Break)
Fri 3:20 p.m. - 4:50 p.m.
Panel Session - Lizy John (UT Austin), David Kaeli (Northeastern University), Tushar Krishna (Georgia Tech), Peter Mattson (Google), Brian Van Essen (LLNL), Venkatram Vishwanath (ANL), Carole-Jean Wu (Facebook) (Discussion Panel)
Tom St John, LIZY JOHn, Tushar Krishna, Peter Mattson, Venkatram Vishwanath, Carole-Jean Wu, Dave Kaeli, Brian Van Essen
Fri 4:50 p.m. - 5:00 p.m.
Conclusion - Murali Emani (ANL) (Conclusion)
Murali Emani