Skip to yearly menu bar Skip to main content


MLSys 2024 List of Accepted Papers

Title Where and When
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
In Gim ⋅ Guojun Chen ⋅ Seung-seob Lee ⋅ Nikhil Sarda ⋅ Anurag Khandelwal ⋅ Lin Zhong
In Room: Poster Position Number Poster Location: 25 at 9:20 AM PDT (America/Los_Angeles) in LLM 2.
Efficient Post-training Quantization with FP8 Formats
Haihao Shen ⋅ Naveen Mellempudi ⋅ Xin He ⋅ Qun Gao ⋅ Chang Wang ⋅ Mengni Wang
In Room: Poster Position Number Poster Location: 28 at 2:40 PM PDT (America/Los_Angeles) in Quantization and Compression 2.
SiDA: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models
Zhixu Du ⋅ Shiyu Li ⋅ Yuhao Wu ⋅ Xiangyu Jiang ⋅ Jingwei Sun ⋅ Qilin Zheng ⋅ Yongkai Wu ⋅ Ang Li ⋅ Hai Li ⋅ Yiran Chen
In Room: Poster Position Number Poster Location: 30 at 9:20 AM PDT (America/Los_Angeles) in Performance and Memory.
LIFL: A Lightweight, Event-driven Serverless Platform for Federated Learning
Shixiong Qi ⋅ K. K. Ramakrishnan ⋅ Myungjin Lee
In Room: Poster Position Number Poster Location: 3 at 4:10 PM PDT (America/Los_Angeles) in Federated Learning.
ACROBAT: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time
Pratik Fegade ⋅ Tianqi Chen ⋅ Phillip Gibbons ⋅ Todd Mowry
In Room: Poster Position Number Poster Location: 6 at 9:40 AM PDT (America/Los_Angeles) in Performance and Memory.
Lancet: Accelerating Mixture-of-Experts Training by Overlapping Weight Gradient Computation and All-to-All Communication
Chenyu Jiang ⋅ Ye Tian ⋅ Zhen Jia ⋅ Chuan Wu ⋅ Yida Wang ⋅ Shuai Zheng
In Room: Poster Position Number Poster Location: 19 at 4:30 PM PDT (America/Los_Angeles) in Parallel and Distributed 2.
Does Compressing Activations Help Model Parallel Training?
Song Bian ⋅ Dacheng Li ⋅ Hongyi Wang ⋅ Eric Xing ⋅ Shivaram Venkataraman
In Room: Poster Position Number Poster Location: 35 at 1:50 PM PDT (America/Los_Angeles) in Measurement and Analysis.
FlashDecoding++: Faster Large Language Model Inference with Asynchronization, Flat GEMM Optimization, and Heuristics
Ke Hong ⋅ Guohao Dai ⋅ Jiaming Xu ⋅ Qiuli Mao ⋅ Xiuhong Li ⋅ Jun Liu ⋅ kangdi chen ⋅ Yuhan Dong ⋅ Yu Wang
In Room: Poster Position Number Poster Location: 33 at 9:00 AM PDT (America/Los_Angeles) in LLM 2.
VIDUR: A LARGE-SCALE SIMULATION FRAMEWORK FOR LLM INFERENCE Project Page ↗
Amey Agrawal ⋅ Nitin Kedia ⋅ Jayashree Mohan ⋅ Ashish Panwar ⋅ Nipun Kwatra ⋅ Bhargav Gulavani ⋅ Ramachandran Ramjee ⋅ Alexey Tumanov
In Room: Poster Position Number Poster Location: 1 at 2:40 PM PDT (America/Los_Angeles) in Measurement and Analysis.
CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generation Project Page ↗
Yifei Xu ⋅ Yuning Chen ⋅ Xumiao Zhang ⋅ Xianshang Lin ⋅ Pan Hu ⋅ Yunfei Ma ⋅ Songwu Lu ⋅ Wan Du ⋅ Zhuoqing Mao ⋅ Ennan Zhai ⋅ Dennis Cai
In Room: Poster Position Number Poster Location: 2 at 1:30 PM PDT (America/Los_Angeles) in Measurement and Analysis.
FLASH: Fast Model Adaptation in ML-Centric Cloud Platforms Project Page ↗
Haoran Qiu ⋅ Weichao Mao ⋅ Archit Patke ⋅ Shengkun Cui ⋅ Chen Wang ⋅ Hubertus Franke ⋅ Zbigniew Kalbarczyk ⋅ Tamer Basar ⋅ Ravi Iyer
In Room: Poster Position Number Poster Location: 4 at 3:50 PM PDT (America/Los_Angeles) in ML for Systems.
UniDM: A Unified Framework for Data Manipulation with Large Language Models
Yichen Qian ⋅ Yongyi He ⋅ Rong Zhu ⋅ Jintao Huang ⋅ Zhijian Ma ⋅ Haibin Wang ⋅ Yaohua Wang ⋅ Xiuyu Sun ⋅ Defu Lian ⋅ Bolin Ding ⋅ Jingren Zhou
In Room: Poster Position Number Poster Location: 5 at 4:30 PM PDT (America/Los_Angeles) in ML for Systems.
FedTrans: Efficient Federated Learning via Multi-Model Transformation
Yuxuan Zhu ⋅ Jiachen Liu ⋅ Mosharaf Chowdhury ⋅ Fan Lai
In Room: Poster Position Number Poster Location: 7 at 3:30 PM PDT (America/Los_Angeles) in Federated Learning.
SLoRA: Scalable Serving of Thousands of LoRA Adapters
Ying Sheng ⋅ Shiyi Cao ⋅ Dacheng Li ⋅ Coleman Hooper ⋅ Nicholas Lee ⋅ Shuo Yang ⋅ Christopher Chou ⋅ Banghua Zhu ⋅ Lianmin Zheng ⋅ Kurt Keutzer ⋅ Joseph Gonzalez ⋅ Ion Stoica
In Room: Poster Position Number Poster Location: 9 at 2:40 PM PDT (America/Los_Angeles) in Large Language Models 1.
JIT-Q: Just-in-time Quantization with Processing-In-Memory for Efficient ML Training
Mohamed Ibrahim ⋅ Shaizeen Aga ⋅ Ada Li ⋅ Suchita Pati ⋅ Mahzabeen Islam
In Room: Poster Position Number Poster Location: 8 at 1:30 PM PDT (America/Los_Angeles) in Quantization and Compression 2.
HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning
Gyudong Kim ⋅ Mehdi Ghasemi ⋅ Soroush Heidari ⋅ Seungryong Kim ⋅ Young Geun Kim ⋅ Sarma Vrudhula ⋅ Carole-Jean Wu
In Room: Poster Position Number Poster Location: 10 at 3:50 PM PDT (America/Los_Angeles) in Federated Learning.
Fine-Tuning Language Models Using Formal Methods Feedback: A Use Case in Autonomous Systems
Yunhao Yang ⋅ Neel P. Bhatt ⋅ Tyler Ingebrand ⋅ William Ward ⋅ Steven Carr ⋅ Atlas Wang ⋅ Ufuk Topcu
In Room: Poster Position Number Poster Location: 11 at 1:50 PM PDT (America/Los_Angeles) in Large Language Models 1.
Distributed Matrix-Based Sampling for Graph Neural Network Training
Alok Tripathy ⋅ Katherine Yelick ⋅ Aydin Buluc
In Room: Poster Position Number Poster Location: 12 at 3:50 PM PDT (America/Los_Angeles) in Parallel and Distributed 1.
Atom: Low-Bit Quantization for Efficient and Accurate LLM Serving Project Page ↗
Yilong Zhao ⋅ Chien-Yu Lin ⋅ Kan Zhu ⋅ Zihao Ye ⋅ Lequn Chen ⋅ Size Zheng ⋅ Luis Ceze ⋅ Arvind Krishnamurthy ⋅ Tianqi Chen ⋅ Baris Kasikci
In Room: Poster Position Number Poster Location: 13 at 9:40 AM PDT (America/Los_Angeles) in Quantization and Compression 1.
vMCU: Coordinated Memory Management and Kernel Optimization for DNN Inference on MCUs Project Page ↗
Size Zheng ⋅ Renze Chen ⋅ Meng Li ⋅ Zihao Ye ⋅ Luis Ceze ⋅ Yun Liang
In Room: Poster Position Number Poster Location: 14 at 9:00 AM PDT (America/Los_Angeles) in Performance and Memory.
AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration
Ji Lin ⋅ Jiaming Tang ⋅ Haotian Tang ⋅ Shang Yang ⋅ Wei-Ming Chen ⋅ Wei-Chen Wang ⋅ Guangxuan Xiao ⋅ Xingyu Dang ⋅ Chuang Gan ⋅ Song Han
In Room: Poster Position Number Poster Location: 15 at 9:00 AM PDT (America/Los_Angeles) in Quantization and Compression 1.
VQPy: An Object-Oriented Approach to Modern Video Analytics Project Page ↗
Shan Yu ⋅ Zhenting Zhu ⋅ Yu Chen ⋅ Hanchen Xu ⋅ Pengzhan Zhao ⋅ Yang Wang ⋅ Arthi Padmanabhan ⋅ Hugo Latapie ⋅ Harry Xu
In Room: Poster Position Number Poster Location: 16 at 4:10 PM PDT (America/Los_Angeles) in ML for Systems.
COMET: Neural Cost Model Explanation Framework
Isha Chaudhary ⋅ Alex Renda ⋅ Charith Mendis ⋅ Gagandeep Singh
In Room: Poster Position Number Poster Location: 17 at 2:20 PM PDT (America/Los_Angeles) in Measurement and Analysis.
Schrodinger's FP Training Neural Networks with Dynamic Floating-Point Containers
Milos Nikolic ⋅ Enrique Torres Sanchez ⋅ Jiahui Wang ⋅ Ali Hadi Zadeh ⋅ Mostafa Mahmoud ⋅ Ameer Abdelhadi ⋅ Kareem Ibrahim ⋅ Andreas Moshovos
In Room: Poster Position Number Poster Location: 18 at 2:20 PM PDT (America/Los_Angeles) in Quantization and Compression 2.
Accelerating ReLU for MPC-Based Private Inference with a Communication-Efficient Sign Estimation
Kiwan Maeng ⋅ G. Edward Suh
In Room: Poster Position Number Poster Location: 20 at 4:30 PM PDT (America/Los_Angeles) in Privacy and security.
L-GreCo: Layerwise-adaptive Gradient Compression For Efficient Data-parallel Deep Learning
Ilia Markov ⋅ Kaveh Alim ⋅ Elias Frantar ⋅ Dan Alistarh
In Room: Poster Position Number Poster Location: 21 at 4:10 PM PDT (America/Los_Angeles) in Parallel and Distributed 1.
Keyformer: KV Cache reduction through key tokens selection for Efficient Generative Inference Project Page ↗
Muhammad Adnan ⋅ Akhil Arunkumar ⋅ Gaurav Jain ⋅ Prashant Nair ⋅ Ilya Soloveychik ⋅ Purushotham Kamath
In Room: Poster Position Number Poster Location: 22 at 9:40 AM PDT (America/Los_Angeles) in LLM 2.
Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design
Jian Meng ⋅ Yuan Liao ⋅ Anupreetham Anupreetham ⋅ Ahmed Hasssan ⋅ Shixing Yu ⋅ Han-sok Suh ⋅ Xiaofeng Hu ⋅ Jae-sun Seo
In Room: Poster Position Number Poster Location: 23 at 1:50 PM PDT (America/Los_Angeles) in Quantization and Compression 2.
ACCURATE LOW-DEGREE POLYNOMIAL APPROXIMATION OF NON-POLYNOMIAL OPERATORS FOR FAST PRIVATE INFERENCE IN HOMOMORPHIC ENCRYPTION
Jingtian Dang ⋅ Jianming Tong ⋅ Anupam Golder ⋅ Cong "Callie" Hao ⋅ Arijit Raychowdhury ⋅ Tushar Krishna
In Room: Poster Position Number Poster Location: 24 at 4:50 PM PDT (America/Los_Angeles) in Privacy and security.
Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache
Zhenyu Zhang ⋅ Shiwei Liu ⋅ Runjin Chen ⋅ Bhavya Kailkhura ⋅ Beidi Chen ⋅ Atlas Wang
In Room: Poster Position Number Poster Location: 26 at 1:30 PM PDT (America/Los_Angeles) in Large Language Models 1.
Proteus: Preserving Model Confidentiality during Graph Optimizations
Yubo Gao ⋅ Maryam Haghifam ⋅ Christina Giannoula ⋅ Renbo Tu ⋅ Gennady Pekhimenko ⋅ Nandita Vijaykumar
In Room: Poster Position Number Poster Location: 27 at 5:10 PM PDT (America/Los_Angeles) in Privacy and security.
On Latency Predictors for Neural Architecture Search
Yash Akhauri ⋅ Mohamed Abdelfattah
In Room: Poster Position Number Poster Location: 29 at 3:30 PM PDT (America/Los_Angeles) in ML for Systems.
QMoE: Sub-1-Bit Compression of Trillion Parameter Models
Elias Frantar ⋅ Dan Alistarh
In Room: Poster Position Number Poster Location: 31 at 9:20 AM PDT (America/Los_Angeles) in Quantization and Compression 1.
DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines
Ye Tian ⋅ Zhen Jia ⋅ Ziyue Luo ⋅ Yida Wang ⋅ Chuan Wu
In Room: Poster Position Number Poster Location: 32 at 3:30 PM PDT (America/Los_Angeles) in Parallel and Distributed 1.
Punica: Multi-Tenant LoRA Serving Project Page ↗
Lequn Chen ⋅ Zihao Ye ⋅ Yongji Wu ⋅ Danyang Zhuo ⋅ Luis Ceze ⋅ Arvind Krishnamurthy
In Room: Poster Position Number Poster Location: 34 at 2:20 PM PDT (America/Los_Angeles) in Large Language Models 1.
Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large Scale Recommendation
Liang Luo ⋅ Buyun Zhang ⋅ Michael Tsang ⋅ Yinbin Ma ⋅ Ching-Hsiang Chu ⋅ Yuxin Chen ⋅ Shen Li ⋅ Yuchen Hao ⋅ Yanli Zhao ⋅ Guna Lakshminarayanan ⋅ Ellie Wen ⋅ Jongsoo Park ⋅ Dheevatsa Mudigere ⋅ Maxim Naumov
In Room: Poster Position Number Poster Location: 36 at 4:50 PM PDT (America/Los_Angeles) in Parallel and Distributed 2.
HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
ZHAO XUANLEI ⋅ Bin Jia ⋅ Haotian Zhou ⋅ Ziming Liu ⋅ Shenggan Cheng ⋅ Yang You
In Room: Poster Position Number Poster Location: 37 at 5:10 PM PDT (America/Los_Angeles) in Parallel and Distributed 2.