Memory-Efficient Inference

AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference

Memory-efficient long sequence inference through automated activation chunking.

May 7, 2024