Ziming Liu 🚀
Ziming Liu 子铭刘

Ph.D. Candidate

National University of Singapore

About Me

Hi, I am a second-year CS Ph.D. candidate at NUS, supervised by Prof. Yang You and work as a member of HPC-AI lab. Currently I am also an intern at Microsoft Research Asia System Group (Shanghai), supervised by Dr. Zhenhua Han and Dr. Yuqing Yang. I received my bachelor’s degree of computer science and engineering at Peking University in 2020, supervised by Prof. Tong Yang.

My research interest is machine learning system and high performance computing. I have been working on pipeline parallelism and sequence parallelism in deep learning training, and I am currently digging into sparse inference and training of deep learning models. I am looking forward to collaborations and research internship opportunities, so please feel free to reach out to me if you are interested in my research.

Download CV
Interests
  • Machine Learning System
  • High Performance Computing
  • Distributed Training & Inference
  • Sparse Inference & Training
Education

  • PhD Computer Science

    National University of Singapore

  • Msc Artificial Intelligence

    National University of Singapore

  • BSc Computer Science

    Peking University

Experience

  1. Research Intern

    Microsoft Research
    Working on sparse inference and training of text-to-image and text-to-video models. Supervised by Dr. Zhenhua Han and Dr. Yuqing Yang.

  2. Research Intern

    HPC-AI Tech

    Responsibilities include:

    • Developing the efficient LLM inference system EnergonAI.
    • Optimizing the implementation of ColossalAI.

  3. Machine Learning Engineer

    ByteDance
    NLP algorithm engineer at Lark, ByteDance.

Education

  1. PhD Computer Science

    National University of Singapore
    Working on Machine Learning System, supervised by Presidential Young Professor Yang You.

  2. Msc Artificial Intelligence

    National University of Singapore

  3. BSc Computer Science

    Peking University
    Bacholor’s degree of computer science and engineering. Supervised by Prof. Tong Yang.
Featured Publications
WallFacer: Harnessing Multi-dimensional Ring Parallelism for Efficient Long Sequence Model Training
Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency
Recent Publications
Ziming Liu, Shaoyu Wang, Shenggan Cheng, Zhongkai Zhao, Kai Wang, Xuanlei Zhao, Hames Demmel, Yang You (2024). WallFacer: Harnessing Multi-dimensional Ring Parallelism for Efficient Long Sequence Model Training. Arxiv Preprint.
PDF Cite
Xuanlei Zhao, Bin Jia, Haotian Zhou, Ziming Liu, Shenggan Cheng, Yang You (2024). HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices. In MLSys 2024, Proceedings of Machine Learning and Systems.
PDF Cite
Xuanlei Zhao, Shenggan Cheng, Guangyang Lu, Jiarui Fang, Haotian Zhou, Bin Jia, Ziming Liu, Yang You (2024). AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference. In ICLR 2024, International Conference on Learning Representations.
PDF Cite
Xuanlei Zhao, Shenggan Cheng, Zangwei Zheng, Zheming Yang, Ziming Liu, Yang You (2024). DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers. Arxiv Preprint.
PDF Cite Project
Ziming Liu, Shenggan Cheng, Haotian Zhou, Yang You (2023). Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency. In SC ‘23, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis.
PDF Cite
Shenggan Cheng, Ziming Liu, Jiangsu Du, Yang You (2023). ATP: Adaptive Tensor Parallelism for Foundation Models. Arxiv Preprint.
PDF Cite
Jiangsu Du, Ziming Liu, Jiarui Fang, Shenggui Li, Yongbin Li, Yutong Lu, Yang You (2022). EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models. Arxiv Preprint.
PDF Cite Project