Timezone: »

 
Oral
SRIFTY: Swift and Thrifty Distributed Neural Network Training on the Cloud
Liang Luo · Liang Luo · Peter West · Peter West · Pratyush Patel · Pratyush Patel · Arvind Krishnamurthy · Luis Ceze · Luis Ceze

Wed Aug 31 09:03 AM -- 09:21 AM (PDT) @ Exhibit Hall A
in Efficient Training »

Finding the best VM configuration is key to achieve lower cost and higher throughput, two primary concerns in cloud-based distributed neural network (NN) training today. Optimal VM selection that meets user constraints requires efficiently navigating a large search space while controlling for the performance variance associated with sharing cloud instances and networks.In this work, we characterize this variance in the context of distributed NN training and present results of a comprehensive throughput and cost-efficiency study we conducted across a wide array of instances to prune for the optimal VM search space. Using insights from these studies, we built Srifty, a system that combines runtime profiling with learned performance models to accurately predict training performance and find the best VM choice that satisfies user constraints, potentially leveraging both heterogeneous setups and spot instances. We integrated Srifty with PyTorch and evaluated it on Amazon EC2. We conducted a large-scale generalization study of Srifty across more than 2K training setups on EC2. Our results show that Srifty achieves an iteration latency prediction error of 8%, and its VM instance recommendations offer significant throughput gain and cost reduction while satisfying user constraints compared to existing solutions in complex, real-world scenarios.

Keywords: systems for ml

Author Information

Liang Luo (University of Washington)
Liang Luo (University of Washington)
Peter West (University of Washington)
Peter West (University of Washington)
Pratyush Patel (University of Washington)
Pratyush Patel (University of Washington)
Arvind Krishnamurthy (University of Washington)
Luis Ceze (University of Washington and OctoML)
Luis Ceze (University of Washington and OctoML)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors