15-884 | Schedule

Date	Plan	Readings	Deadline
02/02 Tue	Week 1: Course Introduction [ slides ]
02/04 Thu	Week 1: Machine Learning Frameworks and Abstractions [ slides ]
02/09 Tue	Week 2: Paper Discussions -- Machine Learning Frameworks [ slides ]	TensorFlow: A System for Large-Scale Machine Learning JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs PyTorch: An Imperative Style, High-Performance Deep Learning Library	Paper Review
02/11 Thu	Week 2: Automatic Differentiation [ slides ]
02/16 Tue	Week 3: Paper Discussions -- Automatic Differentiation [ slides ]	Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients Dynamic Tensor Rematerialization Sense & Sensitivities: The Path to General-Purpose Algorithmic Differentiation	Paper Review
02/18 Thu	Week 3: Graph-level Optimizations [ slides ]
02/23 Tue	Week 4: Break day, no class [ slides ]
02/25 Thu	Week 4: Paper Discussions -- Graph-Level Optimizations [ slides ]	TASO: Optimizing Deep Learning Computation with Automatic Generation of Graph Substitutions Equality Saturation for Tensor Graph Superoptimization Device Placement Optimization with Reinforcement Learning	Paper Review
03/02 Tue	Week 5: Parallelism [ slides ]
03/04 Thu	Week 5: Paper Discussions -- Parallelism [ slides ]	Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning Beyond Data and Model Parallelism for Deep Neural Networks PipeDream: Generalized Pipeline Parallelism for DNN Training	Paper Review
03/09 Tue	Week 6: Distributed Training [ slides ]
03/11 Thu	Week 6: Paper Discussions -- Distributed Training [ slides ]	A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Ray: A Distributed Framework for Emerging AI Applications	Paper Review
03/16 Tue	Week 7: Hardware Backend Specialization and Optimizations [ slides ]
03/18 Thu	Week 7: Paper Discussions -- Hardware Backend Specialization [ slides ]	Fast Algorithms for Convolutional Neural Networks In-Datacenter Performance Analysis of a Tensor Processing Unit Design Automation for Efficient Deep Learning Computing	Paper Review
03/23 Tue	Week 8: Machine Learning for Systems [ slides ]
03/25 Thu	Week 8: Paper Discussions -- Machine Learning for Systems [ slides ]	The Case for Learned Index Structures Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networks Chip Placement with Deep Reinforcement Learning	Paper Review
03/30 Tue	Week 9: Machine Learning Compilation [ slides ]
04/01 Thu	Week 9: Paper Discussions -- Machine Learning Compilation [ slides ]	TVM: An Automated End-to-End Optimizing Compiler for Deep Learning Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasks MLIR: A Compiler Infrastructure for the End of Moore's Law	Paper Review
04/06 Tue	Week 10: Automating ML Compilation [ slides ]
04/08 Thu	Week 10: Paper Discussions -- Automating ML Compilation [ slides ]	Learning to Optimize Halide with Tree Search and Random Programs Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions Ansor: Generating High-Performance Tensor Programs for Deep Learning	Paper Review
04/13 Tue	Week 11: Tiny ML [ slides ]
04/15 Thu	Week 11: Break Day [ slides ]
04/20 Tue	Week 12: Guest Lecture by Matteo Interlandi [ slides ]		Different zoom link. Start at 11:00am.
04/22 Thu	Week 12: Paper Discussion -- TinyML [ slides ]	MCUNet: Tiny Deep Learning on IoT Devices Riptide: Fast End-to-End Binarized Neural Networks TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems	Paper Review
04/27 Tue	Week 13: Guest Lecture by Zhihao Jia [ slides ]		Different zoom link.
04/29 Thu	Week 13: Federated learning [ slides ]
05/04 Tue	Week 14: Paper Discussion -- Federated learning [ slides ]	Communication-Efficient Learning of Deep Networks from Decentralized Data Towards Federated Learning at Scale: System Design Federated Multi-Task Learning	Paper Review
05/06 Thu	Week 14: Project Hacking [ slides ]
05/11 Tue 05/13 Thu	Week 15: Final Project Presentations

15-884: Machine Learning Systems

Schedule