About

I am a member of technical staff at xAI. I lead the inference team responsible for building efficient and reliable infrastructure to run Grok models. My expertise includes machine learning systems, large language models, compilers, and distributed systems.

Previously, I completed my Ph.D. at UC Berkeley, where I was advised by Ion Stoica and Joseph E. Gonzalez. I obtained my B.S. degree from ACM Honored Class, Shanghai Jiao Tong University. I was honored to receive the Meta PhD Fellowship and the a16z Open Source AI Grant (twice) in recognition of my innovative research and impactful open-source projects. I also co-founded the non-profit organization LMSYS.org to advance open research on large models. We have developed open models with millions of downloads, crowdsourced platforms with millions of users, and systems that are orders of magnitude faster.

Research

• Systems for training and serving large models
  • Projects: SGLang (lead), FastChat (lead), Alpa (lead), FlexGen, vLLM
  • First-author publications: NeurIPS 24, OSDI 22, OSDI 23, ICML 21
  • Other publications: ICML 23, SOSP 23, MLSys 23, MLSys 24
• Open large language models, benchmarks, and datasets for large language models
  • Projects: Vicuna (lead), Chatbot Arena (lead), LMSYS-Chat-1M (lead)
  • First-author publications: NeurIPS 23, ICML 24, ICLR 24, Preprint 23
• Deep learning compilers and auto-tuning
  • Projects: TVM, Ansor (lead)
  • First-author publications: OSDI 20, NeurIPS 21
  • Other publications: OSDI 18, NeurIPS 20, ASPLOS 23

*(lead) indicates that I lead or co-lead the project.

More publications