IC Spring Seminar Series with Guest Speaker Zhuang Liu

Abstract

Deep learning with neural networks has emerged as a key approach for discovering patterns and modeling relationships in complex data. AI systems powered by deep learning are used widely in applications across a broad spectrum of scales. There have been strong needs for scaling deep learning both upward and downward. Scaling up highlights the pursuit of scalability — the ability to utilize increasingly abundant computing and data resources to achieve superior capabilities, overcoming diminishing returns. Scaling down represents the demand for efficiency — there is limited data for many application domains, and deployment is often in compute-limited settings. My research focuses on scaling deep learning both up and down, to build capable models and understand their behaviors in different computational and data environments.

In this talk, we will present several studies in both directions. For scaling up, we will first explore the design of scalable neural network architectures that are widely adopted in various applications. We then discuss an intriguing observation on modern vision datasets and its implication on scaling training data. For scaling down, we introduce simple, effective, and popularly used approaches for compressing convolutional networks and large language models, alongside interesting empirical findings. Notably, a recurring theme in this talk is the careful examination of implicit assumptions in the literature, which often leads to surprising revelations that reshape community understanding. We conclude with exciting avenues for future deep learning and vision research, such as next-gen architectures and dataset modeling.

Bio

Zhuang Liu received his Ph.D. in Computer Science from UC Berkeley in 2022, advised by Trevor Darrell. He is currently a Research Scientist at Meta AI Research in New York City. He has broad research interests in Machine Learning, Deep Learning, and Computer Vision. His work focuses on scaling neural networks both up and down, to build capable models and understand their behaviors in different computational and data environments. He is a recipient of the CVPR 2017 Best Paper Award.