Timezone: »

 
Oral
Boveda: Building an On-Chip Deep Learning Memory Hierarchy Brick by Brick
Isak Edo Vivancos · Sayeh Sharify · Daniel Ly-Ma · Ameer Abdelhadi · Ciaran Bannon · Milos Nikolic · Mostafa Mahmoud · Alberto Delmas Lascorz · Gennady Pekhimenko · Andreas Moshovos

Thu Apr 08 09:10 AM -- 09:30 AM (PDT) @
in Session 9: Hardware »

Data access between on- and off-chip memories account for a large fraction of overall energy consumption during inference with deep learning networks. On-chip memory compression can greatly reduce this energy cost as long as it balances the simplicity and low cost of the compression/decompression implementation and its effectiveness in data size reduction. We present Boveda, a simple and effective on-chip lossless memory compression technique for fixed-point precision networks. It reduces data widths by exploiting the value distribution deep learning applications naturally exhibit. Boveda can increase the effective on-chip capacity, reduce off-chip traffic, and/or achieve a desired performance/energy target while using smaller on-chip memories. Boveda can be placed after any memory block in the on-chip memory hierarchy and can work with \textul{any} data-parallel processing units such as the vector-like or the tensorcore units of modern graphics processors, systolic arrays such as that used in the Tensor Processing Unit, and units that process sparse tensors such as those used in the SCNN accelerator. To demonstrate the potential of Boveda, we implement it over (i) SCNN, a state-of-the-art accelerator for sparse networks, (ii) a Tensorcore-like architecture, and (iii) TPU. Boveda reduces memory footprint by 34\% for SCNN and sparse models on top of zero compression. For dense models, Boveda improves compression by 47\%. We also present a prototype FPGA implementation.

Author Information

Isak Edo Vivancos (University of Toronto)
Sayeh Sharify (University of Toronto)
Daniel Ly-Ma (University of Toronto)
Ameer Abdelhadi (University of Toronto)
Ciaran Bannon (University of Toronto)
Milos Nikolic (University of Toronto)
Mostafa Mahmoud (University of Toronto)
Alberto Delmas Lascorz (University of Toronto)
Gennady Pekhimenko (University of Toronto)
Andreas Moshovos (University of Toronto)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors