Catalyst: Home

Catalyst is an interdisciplinary machine learning and systems research group exploring problems to automate learning systems. Our research spanning multiple layers of the machine learning and system stack. Our group is a collaboration between researchers from the Machine Learning Department, Computer Science Department and Electrical & Computer Engineering Department at the Carnegie Mellon University.

Research

TidalDecode

A sparse attention framework for large language model decoding

Machine Learning Compilation for Large Language Models

A universal solution that allows any language model to be deployed natively on a diverse set of hardware backends and native applications.

FlexFlow Serve

Low-Latency, High-Performance LLM Serving

Cortex

End-to-end compilation of ML applications with dynamic and irregular control flow and data structure accesses

Apache TVM Stack

TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

Hyperband and ASHA

Principled early-stopping approaches for hyperparameter optimization

FlexFlow

Automatically Discovering Fast Parallelization Strategies for DNN Training

TASO

The Tensor Algebra SuperOptimizer for Deep Learning

XGBoost

A Scalable Tree Boosting System

Mission

The rapid advance in ML models and ML-specific hardware makes it increasingly challenging to build efficient and scalable learning systems that can take full advantage of the performance capability of modern hardware and runtime environments. Today's ML systems heavily rely on human effort to optimize the training and deployment of ML models on specific target platforms. Unlike conventional application domains, learning systems need to address a continuously growing complexity and diversity in machine learning models, hardware backends, and runtime time environments. Our response to this unique challenge in ML systems is Catalyst (CMU automated learning systems group), a joint research group across the area of machine learning, systems, programming languages, and computer architecture. Our mission is to build ML algorithms and learning systems that automate cross stack optimizations by leveraging mathematical and statistical properties of ML computations and by co-designing systems, hardware, and ML algorithms.

Research

TidalDecode

Machine Learning Compilation for Large Language Models

FlexFlow Serve

Cortex

Apache TVM Stack

Hyperband and ASHA

FlexFlow

TASO

XGBoost

Mission

People

Faculty

PhD Students

Masters Students

Undergraduate Students

Alumni