How do people navigate the exploration-exploitation (EE) trade-off when making repeated choices with unknown rewards? We study this question through the lens of multi-armed bandit problems and introduce a novel behavioral model, Quantal Choice with Adaptive Reduction of Exploration (QCARE). It generalizes Thompson Sampling, allowing for a principled way to quantify the EE trade-off and reflect human decision-making patterns. The model adaptively reduces exploration as information accumulates, with the reduction rate serving as a parameter to quantify the EE trade-off dynamics. We theoretically analyse how varying reduction rates influence decision quality, shedding light on the effects of “over-exploration” and “under-exploration.” Empirically, we validate QCARE through experiments collecting behavioural data from human participants. QCARE not only captures critical behavioural patterns in the EE trade-off but also outperforms alternative models in predictive power. Our analysis reveals a behavioural tendency toward over-exploration.

Papers/Articles

Yifan Feng is an Assistant Professor in the Department of Analytics and Operations at NUS Business School. His research develops theory and methods for learning and decision-making, especially when the data itself is shaped by human preferences, behaviour, or strategic incentives. His work spans applications in platform operations and market design, and more recently LLM alignment. Yifan’s research has appeared in Management Science, Operations Research, Mathematics of Operations Research, and leading AI/ML venues including EC, ICML, NeurIPS, and ICLR. His work has received awards from the INFORMS JFIG Paper Competition, CSAMSE Best Paper Competition, and POMS-HK Best Student Paper Competition, among others. Prior to NUS, Yifan obtained his PhD in Management Science/Operations Management from the University of Chicago Booth School of Business.

For more information about the ESD Seminar, please email esd_invite@sutd.edu.sg