Heterogeneous Parallel Inference

HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices

Efficient Heterogeneous Parallel Inference System for LLM on resource-constrained devices.

May 13, 2024