Workshop
Benchmarking Machine Learning Workloads on Emerging Hardware
Tom St John · Murali Emani · Wenqian Dong
Room 241
Thu 8 Jun, 5 a.m. PDT
With evolving system architectures, hardware and software stack, diverse machine learning workloads,
and data, it is important to understand how these components interact with each other. Well-defined
benchmarking procedures help evaluate and reason the performance gains with ML workload to system
mapping.
Key problems that we seek to address are: (i) which representative ML benchmarks cater to workloads
seen in industry, national labs, and interdisciplinary sciences; (ii) how to characterize the ML workloads
based on their interaction with hardware; (iii) what novel aspects of hardware, such as heterogeneity in
compute, memory, and bandwidth, will drive their adoption; (iv) performance modeling and projections
to next-generation hardware.
The workshop will invite experts in these research areas to present recent work and potential directions
to pursue. Accepted papers from a rigorous evaluation process will present state-of-the-art research
efforts. A panel discussion will foster an interactive platform for discussion between speakers and the
audience.
Schedule
Thu 5:00 a.m. - 5:10 a.m.
|
Introduction
(
Opening Remarks
)
>
|
🔗 |
Thu 5:10 a.m. - 5:55 a.m.
|
Towards More Efficient Vision Transformers: From Novel Few-Shot Parameter-Efficient Tuning to New Linear-Angular Attention - Yingyan Lin (Georgia Tech)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Thu 5:55 a.m. - 6:20 a.m.
|
Understanding Time Variations of DNN Inference in Autonomous Driving - Liangkai Liu (Wayne State University)
(
Presentation
)
>
SlidesLive Video |
🔗 |
Thu 6:20 a.m. - 6:45 a.m.
|
Chakra: Advancing Performance Benchmarking and Co-Design Using Standardized Execution Traces - Srinivas Sridharan (Meta)
(
Presentation
)
>
SlidesLive Video |
🔗 |
Thu 6:45 a.m. - 7:10 a.m.
|
Performance Analysis of Binary Neural Networks Deployed in NVM Crossbar Architectures - Ruirong Huang (Cornell)
(
Presentation
)
>
SlidesLive Video |
🔗 |
Thu 7:10 a.m. - 7:30 a.m.
|
Morning Break
|
🔗 |
Thu 7:30 a.m. - 8:15 a.m.
|
ML Workloads in AR/VR and Their Implication to ML System Design - Hyoukjun Kwon (UC Irvine)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Thu 8:15 a.m. - 9:00 a.m.
|
Benchmarks for Developing Generalist Agents: Closing the Gap in Real-World Autonomous Decision-Making - Vijay Janapa Reddi (Harvard)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Thu 9:00 a.m. - 11:00 a.m.
|
Lunch
|
🔗 |
Thu 11:00 a.m. - 11:45 a.m.
|
Towards AI Workflow Benchmarking with Consideration of Energy Efficiency on Leadership Computing Platforms - Wes Brewer (Oak Ridge National Laboratory)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Thu 11:45 a.m. - 12:30 p.m.
|
Accelerating LLMs with Speculative Inference and Token Tree Verification - Zhihao Jia (Carnegie Mellon University)
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Thu 12:30 p.m. - 12:50 p.m.
|
Afternoon Break
|
🔗 |
Thu 12:50 p.m. - 1:50 p.m.
|
Panel Session - Wes Brewer (ORNL), Martin Foltin (HPE), Zhihao Jia (CMU), Hyoukjun Kwon (UCI), Yingyan Lin (GA Tech)
(
Discussion Panel
)
>
SlidesLive Video |
🔗 |
Thu 1:50 p.m. - 2:00 p.m.
|
Conclusion
(
Closing Remarks
)
>
|
🔗 |