WallFacer: Harnessing Multi-dimensional Ring Parallelism for Efficient Long Sequence Model Training
Efficient Sequence Parallelism System for Transformer model training.
Jun 30, 2024
HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
Efficient Heterogeneous Parallel Inference System for LLM on resource-constrained devices.
May 13, 2024
Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency
Accepted by SC '23. Efficient Pipeline Parallelism System for LLM.
Nov 11, 2023