MLSys 2023 Accepted Papers List

MLSys 2023 Accepted Papers

Time zone: America/New_York

Title		Where
Uniform Sparsity in Deep Neural Networks Saurav Muralidharan		Poster Location: Ballroom B - Position 17 in Sparsity 1: Models and Algorithms.
Exploiting Hardware Utilization and Adaptive Dataflow for Efficient Sparse Convolution in 3D Point Clouds Project Page ↗ Ke Hong ⋅ Zhongming Yu ⋅ Guohao Dai ⋅ Xinhao Yang ⋅ Yaoxiu Lian ⋅ 泽浩刘 ⋅ Ningyi Xu ⋅ Yu Wang		Poster Location: Ballroom B - Position 21 in Sparsity 2: Systems.
HyperGef: A Framework Enabling Efficient Fusion for Hypergraph Neural Network on GPUs Project Page ↗ Zhongming Yu ⋅ Guohao Dai ⋅ Shang Yang ⋅ Genghan Zhang ⋅ Hengrui Zhang ⋅ Feiwen Zhu ⋅ June Yang ⋅ Jishen Zhao ⋅ Yu Wang		Poster Location: Ballroom B - Position 39 in Emerging Models and Domains.
Virtual Machine Allocation with Lifetime Predictions Hugo Barbalho ⋅ Patricia Kovaleski ⋅ Beibin Li ⋅ Luke Marshall ⋅ Marco Molinaro ⋅ Abhisek Pan ⋅ Eli Cortez ⋅ Matheus Leao ⋅ Harsh Patwari ⋅ Zuzu Tang ⋅ Larissa Rozales Gonçalves ⋅ David Dion ⋅ Thomas Moscibroda ⋅ Ishai Menache		Poster Location: Ballroom B - Position 33 in ML for Systems.
Breadth-First Pipeline Parallelism Joel Lamy-Poirier		Poster Location: Ballroom B - Position 3 in Parallel and Distributed Systems 1: Parallelism.
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure Mark Zhao ⋅ Dhruv Choudhary ⋅ Devashish Tyagi ⋅ Ajay Somani ⋅ Max Kaplan ⋅ Sung-Han Lin ⋅ Sarunya Pumma ⋅ Jongsoo Park ⋅ Aarti Basant ⋅ Niket Agarwal ⋅ Carole-Jean Wu ⋅ Christos Kozyrakis		Poster Location: Ballroom B - Position 43 in Storage, Scheduling, and Networking.
SysNoise: Exploring and Benchmarking Training-Deployment System Inconsistency Project Page ↗ Yan Wang ⋅ Yuhang Li ⋅ Ruihao Gong ⋅ Aishan Liu ⋅ yanfei wang ⋅ Jian Hu ⋅ Yongqiang Yao ⋅ Yunchen Zhang ⋅ tianzi xiaotian ⋅ Fengwei Yu ⋅ Xianglong Liu		Poster Location: Ballroom B - Position 13 in Correctness and Security.
FLINT: A Platform for Federated Learning Integration Ewen Wang ⋅ Boyi Chen ⋅ Mosharaf Chowdhury ⋅ Ajay Kannan ⋅ Franco Liang		Poster Location: Ballroom B - Position 27 in Federated Learning.
Building Verified Neural Networks for Computer Systems with Ouroboros Cheng Tan ⋅ Changliu Liu ⋅ Zhihao Jia ⋅ Tianhao Wei		Poster Location: Ballroom B - Position 15 in Correctness and Security.
Cupcake: A Compression Scheduler for Scalable Communication-Efficient Distributed Training Zhuang Wang ⋅ Xinyu Wu ⋅ Zhaozhuo Xu ⋅ T. S. Eugene Ng		Poster Location: Ballroom B - Position 4 in Parallel and Distributed Systems 2: Communication.
Cuttlefish: Low-Rank Model Training without All the Tuning Hongyi Wang ⋅ Saurabh Agarwal ⋅ Pongsakorn U-chupala ⋅ Yoshiki Tanaka ⋅ Eric Xing ⋅ Dimitris Papailiopoulos		Poster Location: Ballroom B - Position 18 in Sparsity 1: Models and Algorithms.
Tutel: Adaptive Mixture-of-Experts at Scale Project Page ↗ Changho Hwang ⋅ Wei Cui ⋅ Yifan Xiong ⋅ Ziyue Yang ⋅ Ze Liu ⋅ Han Hu ⋅ Zilong Wang ⋅ Rafael Salas ⋅ Jithin Jose ⋅ Prabhat Ram ⋅ HoYuen Chau ⋅ Peng Cheng ⋅ Fan Yang ⋅ Mao Yang ⋅ Yongqiang Xiong		Poster Location: Ballroom B - Position 2 in Parallel and Distributed Systems 1: Parallelism.
Be Careful with PyPI Packages: You May Unconsciously Spread Backdoor Model Weights Tianhang Zheng ⋅ Hao Lan ⋅ Baochun Li		Poster Location: Ballroom B - Position 14 in Correctness and Security.
GiPH: Generalizable Placement Learning for Adaptive Heterogeneous Computing Project Page ↗ Yi Hu ⋅ Chaoran Zhang ⋅ Edward Andert ⋅ Harshul Singh ⋅ Aviral Shrivastava ⋅ James Laudon ⋅ Yanqi Zhou ⋅ Bob Iannucci ⋅ Carlee Joe-Wong		Poster Location: Ballroom B - Position 31 in ML for Systems.
Communication-Efficient Graph Neural Networks with Probabilistic Neighborhood Expansion Analysis and Caching Project Page ↗ Tim Kaler ⋅ Alexandros Iliopoulos ⋅ Philip Murzynowski ⋅ Tao Schardl ⋅ Charles E. Leiserson ⋅ Jie Chen		Poster Location: Ballroom B - Position 5 in Parallel and Distributed Systems 2: Communication.
SUBGRAPH STATIONARY HARDWARE-SOFTWARE INFERENCE CO-DESIGN Payman Behnam ⋅ Alexey Tumanov ⋅ Tushar Krishna ⋅ Pranav Gadikar ⋅ Yangyu Chen ⋅ Jianming Tong ⋅ Yue Pan ⋅ Abhimanyu Rajeshkumar Bambhaniya ⋅ Alind Khare		Poster Location: Ballroom B - Position 46 in Edge.
GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning Project Page ↗ Shiqi He ⋅ Qifan Yan ⋅ Feijie Wu ⋅ Lanjun Wang ⋅ Mathias Lécuyer ⋅ Ivan Beschastnikh		Poster Location: Ballroom B - Position 29 in Federated Learning.
FedTree: A Federated Learning System For Trees Project Page ↗ Qinbin Li ⋅ Zhaomin Wu ⋅ Yanzheng Cai ⋅ yuxuan han ⋅ Ching Man Yung ⋅ Tianyuan Fu ⋅ Bingsheng He		Poster Location: Ballroom B - Position 26 in Federated Learning.
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network Project Page ↗ Vitaliy Chiley ⋅ Vithursan Thangarasa ⋅ Abhay Gupta ⋅ Anshul Samar ⋅ Joel Hestness ⋅ Dennis DeCoste		Poster Location: Ballroom B - Position 11 in Memory Optimization.
On Optimizing the Communication of Model Parallelism Yonghao Zhuang ⋅ Lianmin Zheng ⋅ Zhuohan Li ⋅ Eric Xing ⋅ Qirong Ho ⋅ Joseph Gonzalez ⋅ Ion Stoica ⋅ Hao Zhang ⋅ Hexu Zhao		Poster Location: Ballroom B - Position 7 in Parallel and Distributed Systems 2: Communication.
Validating Large Language Models with ReLM Project Page ↗ Michael Kuchnik ⋅ Virginia Smith ⋅ George Amvrosiadis		Poster Location: Ballroom B - Position 12 in Correctness and Security.
Efficient GPU Kernels for N:M-Sparse Weights in Deep Learning Bin Lin ⋅ Ningxin Zheng ⋅ Lei Wang ⋅ Shijie Cao ⋅ Lingxiao Ma ⋅ Quanlu Zhang ⋅ Yi Zhu ⋅ Ting Cao ⋅ Jilong Xue ⋅ Yuqing Yang ⋅ Fan Yang		Poster Location: Ballroom B - Position 19 in Sparsity 2: Systems.
Transcending Runtime-Memory Tradeoffs in Checkpointing by being Fusion Aware Horace He ⋅ Shangdi Yu		Poster Location: Ballroom B - Position 8 in Memory Optimization.
Efficiently Scaling Transformer Inference Project Page ↗ Reiner Pope ⋅ Sholto Douglas ⋅ Aakanksha Chowdhery ⋅ Jacob Devlin ⋅ James Bradbury ⋅ Jonathan Heek ⋅ Kefan Xiao ⋅ Shivani Agrawal ⋅ Jeff Dean		Poster Location: Ballroom B - Position 23 in Measurement and Analysis.
Hotline Profiler: Automatic Annotation and A Multi-Scale Timeline for Visualizing Time-Use in DNN Training Project Page ↗ Daniel Snider ⋅ Fanny Chevalier ⋅ Gennady Pekhimenko		Poster Location: Ballroom B - Position 24 in Measurement and Analysis.
On Noisy Evaluation in Federated Hyperparameter Tuning Kevin Kuo ⋅ Pratiksha Thaker ⋅ Mikhail Khodak ⋅ John Nguyen ⋅ Daniel Jiang ⋅ Ameet Talwalkar ⋅ Virginia Smith		Poster Location: Ballroom B - Position 28 in Federated Learning.
Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN Training Project Page ↗ Borui Wan ⋅ Juntao Zhao ⋅ Chuan Wu		Poster Location: Ballroom B - Position 6 in Parallel and Distributed Systems 2: Communication.
Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation Le Chen ⋅ Quazi Ishtiaque Mahmud ⋅ Hung Phan ⋅ Nesreen Ahmed ⋅ Ali Jannesari		Poster Location: Ballroom B - Position 32 in ML for Systems.
ApproxCaliper: A Programmable Framework for Application-aware Neural Network Optimization Project Page ↗ Yifan Zhao ⋅ Hashim Sharif ⋅ Peter Pao-Huang ⋅ Vatsin Shah ⋅ Arun Narenthiran Sivakumar ⋅ Mateus Valverde Gasparino ⋅ Abdulrahman Mahmoud ⋅ Nathan Zhao ⋅ Sarita Adve ⋅ Girish Chowdhary ⋅ Sasa Misailovic ⋅ Vikram Adve		Poster Location: Ballroom B - Position 25 in Measurement and Analysis.
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse Hyoukjun Kwon ⋅ Krishnakumar Nair ⋅ Jamin Seo ⋅ Jason Yik ⋅ Debabrata Mohapatra ⋅ Dongyuan Zhan ⋅ JINOOK SONG ⋅ Peter Capak ⋅ Peizhao Zhang ⋅ Peter Vajda ⋅ Colby Banbury ⋅ Mark Mazumder ⋅ Liangzhen Lai ⋅ Ashish Sirasao ⋅ Tushar Krishna ⋅ Harshit Khaitan ⋅ Vikas Chandra ⋅ Vijay Janapa Reddi		Poster Location: Ballroom B - Position 37 in Emerging Models and Domains.
AutoScratch: ML-Optimized Cache Management for Inference-Oriented GPUs Yaosheng Fu ⋅ Evgeny Bolotin ⋅ Aamer Jaleel ⋅ Gal Dalal ⋅ Shie Mannor ⋅ Jacob Subag ⋅ Noam Korem ⋅ Michael Behar ⋅ David Nellans		Poster Location: Ballroom B - Position 30 in ML for Systems.
Unified Convolution Framework: A compiler-based approach to support sparse convolutions Jaeyeon Won ⋅ Changwan Hong ⋅ Charith Mendis ⋅ Joel Emer ⋅ Saman Amarasinghe		Poster Location: Ballroom B - Position 20 in Sparsity 2: Systems.
Edge Impulse: An MLOps Platform for Tiny Machine Learning colby banbury ⋅ Vijay Janapa Reddi ⋅ Alexander Elium ⋅ Shawn Hymel ⋅ David Tischler ⋅ Daniel Situnayake ⋅ Carl Ward ⋅ Louis Moreau ⋅ Jenny Plunkett ⋅ Matthew Kelcey ⋅ Mathijs Baaijens ⋅ Alessandro Grande ⋅ Dmitry Maslov ⋅ Arthur Beavis ⋅ Jan Jongboom ⋅ Jessica Quaye		Poster Location: Ballroom B - Position 45 in Edge.
Safe Optimized Static Memory Allocation for Parallel Deep Learning Ioannis Lamprou ⋅ Zhen Zhang ⋅ Javier de Juan ⋅ Hang Yang ⋅ Yongqiang Lai ⋅ Etienne Filhol ⋅ Cedric Bastoul		Poster Location: Ballroom B - Position 9 in Memory Optimization.
Renee: END-TO-END TRAINING OF EXTREME CLASSIFICATION MODELS Project Page ↗ Vidit Jain ⋅ Jatin Prakash ⋅ Deepak Saini ⋅ Jian Jiao ⋅ Ramachandran Ramjee ⋅ Manik Varma		Poster Location: Ballroom B - Position 38 in Emerging Models and Domains.
Reducing Activation Recomputation in Large Transformer Models Vijay Anand Korthikanti ⋅ Jared Casper ⋅ Sangkug Lym ⋅ Lawrence McAfee ⋅ Michael Andersch ⋅ Mohammad Shoeybi ⋅ Bryan Catanzaro		Poster Location: Ballroom B - Position 10 in Memory Optimization.
Sparsity-Aware Memory Interface Architecture using Stacked XORNet Compression for Accelerating Pruned-DNN Models Younghoon Byun ⋅ Seungsik Moon ⋅ Baeseong Park ⋅ Se Jung Kwon ⋅ Dongsoo Lee ⋅ Gunho Park ⋅ Eunji Yoo ⋅ Jung Gyu Min ⋅ Youngjoo Lee		Poster Location: Ballroom B - Position 22 in Sparsity 2: Systems.
ALCOP: Automatic Load-Compute Pipelining in Deep Learning Compiler for AI-GPUs Project Page ↗ Guyue Huang ⋅ Yang Bai ⋅ Liu Liu ⋅ Yuke Wang ⋅ Bei Yu ⋅ Yufei Ding ⋅ Yuan Xie		Poster Location: Ballroom B - Position 34 in Compilers.
Practical Edge Kernels for Integer-Only Vision Transformers Under Post-training Quantization Zining Zhang ⋅ Bingsheng He ⋅ Zhenjie Zhang		Poster Location: Ballroom B - Position 44 in Edge.
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices Project Page ↗ Kazuki Osawa ⋅ Shigang Li ⋅ Torsten Hoefler		Poster Location: Ballroom B - Position 1 in Parallel and Distributed Systems 1: Parallelism.
SIRIUS: Harvesting Whole-Program Optimization Opportunities for DNNs YIJIN LI ⋅ Jiacheng Zhao ⋅ Sun Qianqi ⋅ Haohui Mai ⋅ Lei Chen ⋅ Wanlu Cao ⋅ Yanfan Chen ⋅ Li zhicheng ⋅ YING LIU ⋅ Xinyuan Zhang ⋅ Xiyu Shi ⋅ Jie Zhao ⋅ Jingling Xue ⋅ HUIMIN CUI ⋅ XiaoBing Feng		Poster Location: Ballroom B - Position 35 in Compilers.
X-RLFLOW: GRAPH REINFORCEMENT LEARNING FOR NEURAL NETWORK SUBGRAPHS TRANSFORMATION Project Page ↗ Guoliang HE ⋅ Sean Parker ⋅ Eiko Yoneki		Poster Location: Ballroom B - Position 36 in Compilers.
μ-TWO: 3× Faster Multi-Model Training with Orchestration and Memory Optimization Sanket Purandare ⋅ Abdul Wasay ⋅ Stratos Idreos ⋅ Animesh Jain		Poster Location: Ballroom B - Position 41 in Storage, Scheduling, and Networking.
PyTorch RPC: Distributed Deep Learning Built on Tensor-Optimized Remote Procedure Calls Project Page ↗ Pritam Damania ⋅ Shen Li ⋅ Alban Desmaison ⋅ Alisson Azzolini ⋅ Brian Vaughan ⋅ Edward Yang ⋅ Gregory Chanan ⋅ Guoqiang Jerry Chen ⋅ Hongyi Jia ⋅ Howard Huang ⋅ Joseph Spisak ⋅ Luca Wehrstedt ⋅ Lucas Hosseini ⋅ Manoj Krishnan ⋅ Omkar Salpekar ⋅ Pavel Belevich ⋅ Rohan Varma ⋅ Satendra Gera ⋅ Wanchao Liang ⋅ Shihao Xu ⋅ Soumith Chintala ⋅ Chaoyang He ⋅ Amir Ziashahabi ⋅ Salman Avestimehr ⋅ ⋅ Zachary DeVito		Poster Location: Ballroom B - Position 42 in Storage, Scheduling, and Networking.
Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models Project Page ↗ Daochen Zha ⋅ Louis Feng ⋅ Liang Luo ⋅ Bhargav Bhushanam ⋅ Zirui Liu ⋅ Yusuo Hu ⋅ Jade Nie ⋅ Yuzhen Huang ⋅ Yuandong Tian ⋅ Arun Kejariwal ⋅ Xia Hu		Poster Location: Ballroom B - Position 40 in Storage, Scheduling, and Networking.
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts Trevor Gale ⋅ Deepak Narayanan ⋅ Cliff Young ⋅ Matei Zaharia		Poster Location: Ballroom B - Position 16 in Sparsity 1: Models and Algorithms.