Zeyuan Allen-Zhu, Sc.D. 
You're the th
visitor.
Last Update: December 20, 2024.
You're the th
visitor.
Last Update: December 20, 2024.
Many asked about intern/collaboration possibilities (details). Short answer: unless you're from Meta and willing to work with me in your spare time (20+ hrs/week), or an early-year grad student from UCB/NYU/CMU/UW.
If you previously had my WeChat, please add my new one by replacing the '-' symbol in my wechat ID with '_'.
Incredibly honored and humbled by the overwhelming response to my tutorial, and thank you everyone who attended in person. Truly heartwarming to hear how much you enjoyed it. Many have been asking for a recording, and I prepared one with my own subtitles https://t.co/RjTm9ZHpId https://t.co/PFi2elHnsi pic.twitter.com/hBy1aPzIFU
— Zeyuan Allen-Zhu (@ZeyuanAllenZhu) July 25, 2024
My current research focuses on investigating the physics of language models and AI in a broader sense. This involves designing experiments to elucidate the underlying universal principles governing how LLMs learn to accomplish diverse AI tasks. By probing into the neurons, one can uncover intricate (and sometimes surprising!) mechanisms behind how these AI models function. The ultimate goal is to provide theoretical guidance and practical suggestions on how we can ultimately achieve AGI. This line of work is featured at ICML2024 tutorial.
Before that, I work on the mathematics of deep learning. That involves developing rigorous theoretical proofs towards the learnability of neural networks, in ideal and theory-friendly settings, to explain certain mysterious phenomena observed in deep learning. In this area, our paper on ensemble / knowledge distillation received some award from ICLR'23; although I am most proud of our COLT'23 result that provably shows why deep learning is actually deep –– better than shallow learners such as layer-wise training, kernel methods, etc.
In my past life, I have also worked in machine learning, optimization theory, and theoretical computer science.
In algorithm competitions, I was fortunate to win a few awards in my past life, including two IOI gold medals, a USACO world champion, an ACM/ICPC world-final gold medal, a Google Codejam world runner-up, and a USA MCM Top Prize.
In research, I used to be supported by a Microsoft Young Fellow Award, a Simons Student Award and a Microsoft Azure Research Award.
For a full list, click here.