WallFacer: Harnessing Multi-dimensional Ring Parallelism for Efficient Long Sequence Model Training
Efficient Sequence Parallelism System for Transformer model training.
Jun 30, 2024
AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
Memory-efficient long sequence inference through automated activation chunking.
May 7, 2024
ATP: Adaptive Tensor Parallelism for Foundation Models
Adaptive Tensor Parallelism for efficient foundation model training.
Jan 1, 2023
EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models
An inference system designed for handling 10-100 billion parameter transformer models efficiently.
Jan 1, 2022