MLSys 2026 Papers

Layout:

mini compact topic detail

NodeSweep: Practical Straggler Detection and Health Monitoring for Large-Scale Foundation Model Training

SchedFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling

From Tokens to Layers: Redefining Stall-Free Scheduling for LLM Serving with Layered Prefill

FlexTrain: Scalable Hybrid-Parallel Training with Elastic Resource Utilization and Consistent Accuracy

Optimizing PyTorch Inference with LLM-Based Multi-Agent Systems

FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Models

AIRS: Scaling Live Inference in Resource Constrained Environments

Ontology-Guided Long-Term Memory for Conversational RAG

FlexScale: Flexible and High-Performance FSDP at Scale

OPKV: A High-Throughput Plugin-Driven Framework for Recallable Sparsity in Paged KV Cache Systems

ZK-APEX: ZERO-KNOWLEDGE APPROXIMATE PERSONALIZED UNLEARNING WITH EXECUTABLE PROOFS

AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization

Dataflow Is All You Need

RagInfer: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval

ADS: AN AGENTIC DETECTION SYSTEM FOR ENTERPRISE AGENTIC AI SECURITY

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization

BEAM: Joint Resource–Power Optimization for Energy-Efficient LLM Inference under SLO contraints

GriNNder: Breaking the Memory Capacity Wall in Full-Graph GNN Training with Storage Offloading

RAGBoost: Efficient Retrieval-Augmented Generation with Accuracy-Preserving Context Reuse

CATWILD: Compiler Autotuning for TPU workloads in the Wild

ML Fleet Efficiency: Improving TPU Systems at Scale with ML Productivity Goodput

ProfInfer: An eBPF-based Fine-Grained LLM Inference Profiler

FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Cost

Using Span Queries to Optimize Cache and Attention Locality

Agentic Operator Generation for ML ASICs

NEST: Network- and Memory-Aware Device Placement for Distributed Deep Learning

Charon: A Unified and Fine-Grained Simulator for Large-Scale LLM Training and Inference

Scaling Up Large Language Models Serving Systems for Semantic Job Search

Automated Algorithm Design for Auto-Tuning Optimizers

ProToken: Token-Level Attribution for Federated Large Language Models

Shannonic: Efficient Entropy-Optimal Compression for ML Workloads

Optimizing Deployment Configurations for LLM Inference

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing

Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models

SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models

DriftBench: Measuring and Predicting Infrastructure Drift in LLM Serving Systems

CRAFT: Cost-aware Expert Replica Allocation with Fine-Grained Layerwise Estimations

Privatar: Scalable Privacy-preserving Multi-user VR via Secure Offloading

MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces

Zero redundancy distributed learning with differential privacy

FlashAgents: Accelerating Multi-Agent LLM Systems via Streaming Prefill Overlap

ApproxMLIR : Accuracy-Aware Compiler for Compound ML System

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving

G-HEMP: FAST MULTI-GPU PRIVATE INFERENCE FOR LARGE-SCALE GCNS WITH HOMOMORPHIC ENCRYPTION

BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching

SAKURAONE: An Open Ethernet–Based AI HPC System and Its Observed Workload Dynamics in a Single-Tenant LLM Development Environment

LLMInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems

Hawkeye: Reproducing GPU-Level Non-Determinism

PRISM: PARAMETRICALLY RESTRUCTURED INFERENCE FOR SPECULATIVE SAMPLING DRAFT MODELS

ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels

AXLearn: Modular, Hardware-Agnostic Large Model Training

HipKittens: Fast and Furious AMD Kernels

TiDAR: Think in Diffusion, Talk in Autoregression

db-SP: Accelerating Sparse Attention for Visual Generative Models with Dual-Balanced Sequence Parallelism

When Machine Learning Isn’t Sure: Building Resilient ML-Based Computer Systems by Embracing Uncertainty

TriInfer: Hybrid EPD Disaggregation for Efficient Multimodal Large Language Model Inference

EarthSight: A Distributed Framework for Low-Latency Satellite Intelligence

BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding

Breaking the Ice: Analyzing Cold Start Latency in vLLM

Attribution-based Sparse Activation in Large Language Models

Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel

ExecuTorch - A Unified PyTorch Solution to Run ML Models On-Device

REPARO: LOSS-RESILIENT GENERATIVE CODEC FOR VIDEO CONFERENCING

FarSkip-Collectives: Unhobbling Blocking Communication in Mixture of Experts Models

Meeting SLOs, Slashing Hours: Automated Enterprise LLM Optimization with OptiKIT

Practical Adversarial Multi-Armed Bandits with Sublinear Runtime

MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs

XProf: An Open, Scalable, and Extensible Profiling System for the Modern ML Stack

Sparing Strategies to Minimize Reliability Impact On Large Training Jobs

StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation

DreamDDP: Accelerating Low-Bandwidth Geo-Distributed LLM Training with Layer-wise Partial Synchronization

When Enough is Enough: Rank-Aware Early Termination for Vector Search

DisAgg: Distributed Aggregators for Efficient Secure Aggregation

Beyond the Buzz: A Pragmatic Take on Inference Disaggregation

Pylo: Towards Accessible Learned Optimizers in PyTorch

Blueprint, Bootstrap, and Bridge: A Security Look at NVIDIA GPU Confidential Computing

VeriMoA: A Mixture-of-Agents Framework for Spec-to-HDL Generation

Virtual Machine NUMA Placement at Scale: Learning the Norm, Shielding the Tail

CSLE: A Reinforcement Learning Platform for Autonomous Security Management

MAC-Attention: a Match--Amend--Complete scheme for fast and accurate attention computation

SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding

IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inference

Cost-aware Duration Prediction for Software Upgrades in Datacenters

Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost

Search Your NVFP4 Scales!

Beat the long tail: Distribution-Aware Speculative Decoding for RL Training

Efficient Long-Context Language Model Training by Core Attention Disaggregation

ProTrain: Efficient LLM Training via Automatic Memory Management

WAVE: A SYMBOLIC PYTHON DSL AND COMPILER FOR HIGH PERFORMANCE MACHINE LEARNING

PLA-Serve: A Prefill-Length-Aware LLM Serving System

Flashlight: PyTorch Compiler Extensions to Accelerate Attention Variants

Rethinking DVFS for Mobile LLMs: Unified Energy-Aware Scheduling with CORE

FaaScale: Unlocking Fast LLM Scaling for Serverless Inference

HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments

MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training

Spira: Exploiting Voxel Data Structural Properties for Efficient Sparse Convolution in Point Cloud Networks

TokenBlend: Accelerating Tensor Parallelism LLM Inference Through Efficient Compute-Communication Overlap

SuperInfer: SLO-Aware Rotary Scheduling and Memory Management for LLM Inference on Superchips

Once-for-All Channel Mixers (HyperTinyPW): Generative Compression for TinyML

RDMA Point-to-Point Communication for LLM Systems

Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding

HELIOS : Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving

OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents

CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training

Hippocampus: An Efficient and Scalable Memory Module for Agentic AI

Flash3DGS: Algorithm and System Co-Optimization for Fast 3D Gaussian Splatting on GPUs

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware

FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling

Toward Principled LLM Safety Testing: Solving the Jailbreak Oracle Problem

FP8-Flow-MoE: A Casting-Free FP8 Recipe without Double Quantization Error

Locality-Aware Beam Scheduling for Efficient Test-Time Compute with a Consumer-grade GPU

PROMPTS: PeRformance Optimization via Multi-Agent Planning for LLM Training and Serving

PLayer-FL: A Principled Approach to Personalized Layer-wise Cross-Silo Federated Learning

Demystifying the Mixture of Experts Serving Tax

RaidServe: High-performance Resilient Serving

Unified LLM Model for Power, Performance, and Area Prediction from Hardware Code

SHIP: SRAM-Based Huge Inference Pipelines for Fast LLM Serving

Grolar: Efficient LLM Training on Heterogeneous Clusters

Speculative Decoding: Performance or Illusion?

MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design

Massive-Scale Out-Of-Core UMAP on the GPU

The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents

Parrot: Persuasion and Agreement Robustness Rating of Output Truth

Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP

SONAR: Benchmarking Topology and Collaboration in Decentralized Learning

Stream2LLM: Overlap Context Streaming and Prefill for Reduced Time-to-First-Token

A Lightweight High-Throughput Collective-Capable NoC for Large-Scale ML Accelerators

Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework

FlexiCache: Leveraging Temporal Stability of Attention Heads for Efficient KV Cache Management

CDLM: CONSISTENCY DIFFUSION LANGUAGE MODELS FOR FASTER SAMPLING

LEANN: A Low-Storage Overhead Vector Index

Efficient, VRAM-Constrained xLM Inference on Clients

AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents

NexSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems