Departmental Colloquium
Dr. Yufeng Liu
University of Michigan
Low-rank Reinforcement Learning with Heterogeneous Human Feedback
Abstract: Modern decision-making systems, from online marketplaces to large language models (LLMs), increasingly rely on high-dimensional human feedback, where heterogeneous user preferences and massive feature spaces pose major challenges for statistical efficiency and alignment. In this talk, I will present low-rank reinforcement learning (RL) methods that exploit latent structures in human feedback to enable scalable and theoretically grounded learning. In the first part, we study the dynamic assortment problem in high-dimensional e-commerce and show how a low-rank structure in user–item interactions reduces the complexity of estimating personalized utilities and enables efficient exploration–exploitation strategies with provable regret guarantees. In the second part, we extend these ideas to reinforcement learning from human feedback (RLHF) in large-scale contextual environments, proposing a low-rank contextual framework that accommodates diverse user preferences and complex latent spaces in LLMs while providing theoretical guarantees on sample efficiency and robustness under distribution shifts.
Friday April 17, 2026 at 3:00 PM in 636 SEO