Fast inference engine for Transformer models

📚Tensor/CUDA Cores, 📖150+ CUDA Kernels, ⚡️⚡️toy-hgemm library with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS 🎉🎉).

Tuned OpenCL BLAS

BLISlab: A Sandbox for Optimizing GEMM

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

High-Performance FP32 Matrix Multiplication on CPU

Optimizing SGEMM kernel functions on NVIDIA GPUs to a close-to-cuBLAS performance.

The HPC toolbox: fused matrix multiplication, convolution, data-parallel strided tensor primitives, OpenMP facilities, SIMD, JIT Assembler, CPU detection, state-of-the-art vectorized BLAS for floats and integers

Stretching GPU performance for GEMMs and tensor contractions.

🔥🔥🔥 A collection of some awesome public CUDA, cuBLAS, TensorRT and High Performance Computing (HPC) projects.

DBCSR: Distributed Block Compressed Sparse Row matrix library

Single file libraries for C/C++

Stepwise optimizations of DGEMM on CPU, reaching performance faster than Intel MKL eventually, even under multithreading.

hipBLASLt is a library that provides general matrix-matrix operations with a flexible API and extends functionalities beyond a traditional BLAS library

Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.

FP64 equivalent GEMM via Int8 Tensor Cores using the Ozaki scheme

PyTorch half precision gemm lib w/ fused optional bias + optional relu/gelu

Serial and parallel implementations of matrix multiplication

Code for testing the native float16 matrix multiplication performance on Tesla P100 and V100 GPU based on cublasHgemm