WallFacer: Harnessing Multi-dimensional Ring Parallelism for Efficient Long Sequence Model Training
Efficient Sequence Parallelism System for Transformer model training.
Jun 30, 2024
DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
Dynamic Sequence Parallelism for multi-dimensional transformers.
Jan 1, 2024