Systems for ML and ML for Systems: A Virtuous Cycle

This talk is about the virtuous interplay between machine learning (ML) and systems. I will show examples of how systems optimized for ML computation can be used to train more accurate and capable ML models and how these ML models can be used to improve upon the ad-hoc heuristics used in system design and management. These improved systems can then be used to train better ML models. The latest trend in ML is the development of Foundation models. Foundation models are large pretrained models that have obtained state-of-the-art quality in natural language processing, vision, speech, and other areas. These models are challenging to train and serve because they are characterized by billions of parameters, irregular data access (sparsity) and irregular control flow. I will explain how Reconfigurable Dataflow Accelerators (RDAs) can be designed to accelerate foundation models with these characteristics. SambaNova Systems is using RDA technology to achieve record-setting performance on foundation models. I will describe how the RDAs can also be used to build Taurus, an intelligent network data plane that enables ML models to be used to manage computer networks at full line-rate bandwidths. In particular, a Taurus prototype detects two orders of magnitude more events in a security application than a state-of-the-art system based on conventional network technology.