Timezone: »

 
Demonstration
Skyline: Interactive In-editor Performance Visualizations and Debugging for DNN Training
Geoffrey Yu · Tovi Grossman · Gennady Pekhimenko

Mon Mar 04:00 PM -- 07:00 PM PST @ Ballroom B + C

Training a modern state-of-the-art deep neural network (DNN) is often a long running, memory-intensive, and computationally-expensive process. As a result, deep learning researchers and practitioners often spend considerable effort tuning their models for computational performance (e.g., reducing its memory footprint to increase its batch size). However, effectively performing this tuning requires intimate knowledge of the underlying software and hardware systems—something that not all deep learning users have.

To address this problem, we introduce Skyline: a new interactive in-editor DNN performance visualization and debugging tool. Skyline analyzes models live as they are developed and provides interactive real-time performance feedback with inline code highlighting. In our demonstration, through examples using ResNet-50, the Transformer, and GNMT, MLSysʼ20 attendees will learn how they can use Skyline in their daily workflows to simplify the process of reasoning about and identifying run time and memory bottlenecks in their models.

Author Information

Geoffrey Yu (University of Toronto)
Tovi Grossman (University of of Toronto)
Gennady Pekhimenko (University of Toronto)

More from the Same Authors

  • 2020 Oral: MLPerf Training Benchmark »
    Peter Mattson · Christine Cheng · Gregory Diamos · Cody Coleman · Paulius Micikevicius · David Patterson · Hanlin Tang · Gu-Yeon Wei · Peter Bailis · Victor Bittorf · David Brooks · Dehao Chen · Debo Dutta · Udit Gupta · Kim Hazelwood · Andy Hock · Xinyuan Huang · Daniel Kang · David Kanter · Naveen Kumar · Jeffery Liao · Deepak Narayanan · Tayo Oguntebi · Gennady Pekhimenko · Lillian Pentecost · Vijay Janapa Reddi · Taylor Robie · Tom St John · Carole-Jean Wu · Lingjie Xu · Cliff Young · Matei Zaharia
  • 2020 Poster: MLPerf Training Benchmark »
    Peter Mattson · Christine Cheng · Gregory Diamos · Cody Coleman · Paulius Micikevicius · David Patterson · Hanlin Tang · Gu-Yeon Wei · Peter Bailis · Victor Bittorf · David Brooks · Dehao Chen · Debo Dutta · Udit Gupta · Kim Hazelwood · Andy Hock · Xinyuan Huang · Daniel Kang · David Kanter · Naveen Kumar · Jeffery Liao · Deepak Narayanan · Tayo Oguntebi · Gennady Pekhimenko · Lillian Pentecost · Vijay Janapa Reddi · Taylor Robie · Tom St John · Carole-Jean Wu · Lingjie Xu · Cliff Young · Matei Zaharia
  • 2020 Poster: BPPSA: Scaling Back-propagation by Parallel Scan Algorithm »
    Shang Wang · Yifan Bai · Gennady Pekhimenko
  • 2020 Oral: BPPSA: Scaling Back-propagation by Parallel Scan Algorithm »
    Shang Wang · Yifan Bai · Gennady Pekhimenko