Workshop
Benchmarking Machine Learning Workloads on Emerging Hardware
Tom St John · Murali Emani · Wenqian Dong
Room 241
With evolving system architectures, hardware and software stack, diverse machine learning workloads,
and data, it is important to understand how these components interact with each other. Well-defined
benchmarking procedures help evaluate and reason the performance gains with ML workload to system
mapping.
Key problems that we seek to address are: (i) which representative ML benchmarks cater to workloads
seen in industry, national labs, and interdisciplinary sciences; (ii) how to characterize the ML workloads
based on their interaction with hardware; (iii) what novel aspects of hardware, such as heterogeneity in
compute, memory, and bandwidth, will drive their adoption; (iv) performance modeling and projections
to next-generation hardware.
The workshop will invite experts in these research areas to present recent work and potential directions
to pursue. Accepted papers from a rigorous evaluation process will present state-of-the-art research
efforts. A panel discussion will foster an interactive platform for discussion between speakers and the
audience.
Schedule
Thu 5:00 a.m. - 5:10 a.m.
|
Introduction
(
Opening Remarks
)
|
🔗 |
Thu 5:10 a.m. - 5:55 a.m.
|
Towards More Efficient Vision Transformers: From Novel Few-Shot Parameter-Efficient Tuning to New Linear-Angular Attention - Yingyan Lin (Georgia Tech)
(
Invited Talk
)
|
🔗 |
Thu 5:55 a.m. - 6:20 a.m.
|
Understanding Time Variations of DNN Inference in Autonomous Driving - Liangkai Liu (Wayne State University)
(
Presentation
)
|
🔗 |
Thu 6:20 a.m. - 6:45 a.m.
|
Chakra: Advancing Performance Benchmarking and Co-Design Using Standardized Execution Traces - Srinivas Sridharan (Meta)
(
Presentation
)
|
🔗 |
Thu 6:45 a.m. - 7:10 a.m.
|
Performance Analysis of Binary Neural Networks Deployed in NVM Crossbar Architectures - Ruirong Huang (Cornell)
(
Presentation
)
|
🔗 |
Thu 7:10 a.m. - 7:30 a.m.
|
Morning Break
(
Break
)
|
🔗 |
Thu 7:30 a.m. - 8:15 a.m.
|
ML Workloads in AR/VR and Their Implication to ML System Design - Hyoukjun Kwon (UC Irvine)
(
Invited Talk
)
|
🔗 |
Thu 8:15 a.m. - 9:00 a.m.
|
Benchmarks for Developing Generalist Agents: Closing the Gap in Real-World Autonomous Decision-Making - Vijay Janapa Reddi (Harvard)
(
Invited Talk
)
|
🔗 |
Thu 9:00 a.m. - 11:00 a.m.
|
Lunch
(
Break
)
|
🔗 |
Thu 11:00 a.m. - 11:45 a.m.
|
Towards AI Workflow Benchmarking with Consideration of Energy Efficiency on Leadership Computing Platforms - Wes Brewer (Oak Ridge National Laboratory)
(
Invited Talk
)
|
🔗 |
Thu 11:45 a.m. - 12:30 p.m.
|
Accelerating LLMs with Speculative Inference and Token Tree Verification - Zhihao Jia (Carnegie Mellon University)
(
Invited Talk
)
|
🔗 |
Thu 12:30 p.m. - 12:50 p.m.
|
Afternoon Break
(
Break
)
|
🔗 |
Thu 12:50 p.m. - 1:50 p.m.
|
Panel Session - Wes Brewer (ORNL), Martin Foltin (HPE), Zhihao Jia (CMU), Hyoukjun Kwon (UCI), Yingyan Lin (GA Tech)
(
Discussion Panel
)
|
🔗 |
Thu 1:50 p.m. - 2:00 p.m.
|
Conclusion
(
Closing Remarks
)
|
🔗 |