A schedule language for large model training

CUDA Templates for Linear Algebra Subroutines

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.

The model pre- and post-processing framework

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

A scalable eigensolver for dense, symmetric (hermitian) matrices (fork of https://gitlab.mpcdf.mpg.de/elpa/elpa.git)

Continuous builder and binary build scripts for pytorch

A lightweight parameter server interface

A high performance and general PS framework for distributed training

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)

Lingvo

Python bindings for UCX

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Count the MACs / FLOPs of your PyTorch model.

gossip: Efficient Communication Primitives for Multi-GPU Systems

HugeCTR is a high efficiency GPU framework designed for Click-Through-Rate (CTR) estimating training

Distributed training framework for TensorFlow, Keras, PyTorch, and MXNet.

Open deep learning compiler stack for cpu, gpu and specialized accelerators

An Open Source Machine Learning Framework for Everyone

NLP made easy

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Go, Javascript and more

Trax — Deep Learning with Clear Code and Speed

Dive into Deep Learning Compiler

Run a conference from your backyard.

Open Fabric Interfaces