Xinyu Zhao

About Me

Hello! I am Xinyu Zhao, a first-year PhD student at UNITES @ UNC CS advised by Prof. Tianlong Chen. Prior to that, I obtained my master degree at PKU and BA at HUST. My current research interests include (A) improving scaling efficiency (via MoE, merging) in LLM and Multimodal LLM, and (B) building trustworthy LLM agent system for general and healthcare application.

News

[Sep. 2024] One paper about combining LLMs by merging and mixture is accepted to NeurIPS 2024 Dataset and Benchmark Track.
[May. 2024] One paper about MoE reliability evaluation is accepted to ICML 2024.
[Jan. 2024] One paper about SMoE for multilingual machine translation is accepted to ICLR 2024.

Publications

NeurIPS

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

Xinyu Zhao*, Guoheng Sun*, Ruisi Cai*, Yukun Zhou*, Pingzhi Li*, Peihao Wang*, Bowen Tan, Yexiao He, Li Chen, Yi Liang, Beidi Chen, Binhang Yuan, Hongyi Wang^, Ang Li^, Zhangyang Wang^, Tianlong Chen^

Conference on Neural Information Processing Systems (NeurIPS) Dataset and Benchmark Track, 2024.

PDF Code BibTex

ICML

MoE-RBench: Towards Building Reliable Language Models with Sparse Mixture-of-Experts

Guanjie Chen*, Xinyu Zhao*, Tianlong Chen, Yu Cheng

International Conference on Machine Learning (ICML), 2024.

PDF Code BibTex

ICLR

Sparse MoE with Language Guided Routing for Multilingual Machine Translation

Xinyu Zhao, Xuxi Chen, Yu Cheng, Tianlong Chen

International Conference on Learning Representations (ICLR), 2024.

PDF Code BibTex

About Me

News

Publications

Services

Conference Reviewers

Tutorials