Conference on Cognitive Computational Neuroscience

While the role of uncertainty in learning and inference has been well studied, its additional contribution to exploratory decision making is less well understood. Here, we investigate through behavioral and pupillometric data how uncertainty and reward jointly govern exploration in decision making in a novel two-armed bandit paradigm, in which there are occasional changes in reward contingencies (change points). In terms of expected uncertainty (EU), which is thought to reflect observation noise and inferential uncertainty, we find subjects accord EU an exploratory value that is amplified when expected reward (ER) is larger. In terms of unexpected uncertainty (UU), which is associated with detecting change points, we find that subjects readily switch choices after change points, and the magnitude and timing of the transient pupil response reflect when the behavioral switch happens. These findings reveal pupillometry measures as a valuable tool for revealing uncertainty- and reward-related factors driving learning and exploration.