Timezone: »

 
Poster
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse
Hyoukjun Kwon · Krishnakumar Nair · Jamin Seo · Jason Yik · Debabrata Mohapatra · Dongyuan Zhan · JINOOK SONG · Peter Capak · Peizhao Zhang · Peter Vajda · Colby Banbury · Mark Mazumder · Liangzhen Lai · Ashish Sirasao · Tushar Krishna · Harshit Khaitan · Vikas Chandra · Vijay Janapa Reddi

@ Ballroom B - Position 37
in Emerging Models and Domains »

Real-time multi-model multi-task (MMMT) workloads, a new form of deep learning inference workloads, are emerging for applications areas like extended reality (XR) to support metaverse use cases. These workloads combine user interactivity with computationally complex machine learning (ML) activities. Compared to standard ML applications, these ML workloads present unique difficulties and constraints. Real-time MMMT workloads impose heterogeneity and concurrency requirements on future ML systems and devices, necessitating the development of new capabilities. This paper begins with a discussion of the various characteristics of these real-time MMMT ML workloads and presents an ontology for evaluating the performance of future ML hardware for XR systems. Next, we present XR Bench, a collection of MMMT ML tasks, models, and usage scenarios that execute these models in three representative ways: cascaded, concurrent, and cascaded-concurrency for XR use cases. Finally, we emphasize the need for new metrics that capture the requirements properly. We hope that our work will stimulate research and lead to the development of a new generation of ML systems for XR use cases.

[ Poster] 

Author Information

Hyoukjun Kwon (University of California, Irvine)
Krishnakumar Nair (Meta)
Jamin Seo (Georgia Institute of Technology)
Jason Yik (Harvard University)
Jason Yik

Jason Yik is a first-year Ph.D. student at Harvard University, advised by Prof. Vijay Janapa Reddi. His research interests are broadly in next-generation systems and applications, specifically in neuromorphic computing and extended-reality devices.

Debabrata Mohapatra (Meta)
Dongyuan Zhan (Meta Inc.)
JINOOK SONG (META)
Peter Capak (Meta XRTech)
Peizhao Zhang (Meta)
Peter Vajda (Facebook)
Colby Banbury (Harvard)
Mark Mazumder (Harvard University)
Liangzhen Lai (Facebook Inc)
Ashish Sirasao (Meta inc)
Tushar Krishna (Georgia Institute of Technology)
Harshit Khaitan (Meta)
Vikas Chandra (Meta)
Vijay Janapa Reddi (Harvard University)

More from the Same Authors