Conference on Cognitive Computational Neuroscience

While neural networks approach human levels of perfor- mance in many complex tasks, they require much more training than humans. This may be because only humans can infer and apply generalizable principles from prior ex- perience (Lake, Ullman, Tenenbaum, & Gershman, 2017). However, the statistics that underlie the human learning process are poorly understood and hard to investigate in the large state spaces found in most complex tasks (van Opheusden & Ma, 2019). We thus designed a cog- nitive task whose potential solutions are few enough for subjects to densely sample policy space, but complex enough to compel intelligent search. We launched the game as a smartphone-based app (hexxed.io) to collect data from 10k human participants. We find that unlike re- inforcement learning agents (Deep-Q Networks ; DQNs), humans (1) search a highly restricted subset of the pol- icy space; (2) attempt even poor solutions many times before discarding them; (3) arrive at the optimal policy suddenly and unpredictably with a “leap of insight”. Our data suggest a “top-down” learning process by which hu- mans propose explanatory solutions which they replace only upon collecting sufficient evidence to the contrary, in contrast to the “bottom-up” learning of DQNs that as- sociates states with rewarding actions