Conference on Cognitive Computational Neuroscience

Recent advances in the field of Artificial Intelligence have paved the way for the development of novel brain encoding models. Artificial Neural networks (ANN) can be trained to replicate the properties of brain dynamics in a range of cognitive processes. Videogames provide a promising framework linking brain activity to individual behavior in a naturalistic setting. In this study, we aimed to use ANNs to model functional magnetic resonance imaging (fMRI) and behavioral gameplay data, which we collected while subjects played the Shinobi III videogame. Using imitation learning, we trained an ANN to play the game closely replicating the unique gameplay style of individual participants. We found that hidden layers of our imitation learning model successfully encode task relevant neural representations and predict individual brain dynamics with higher accuracy than various control models. In particular, the highest correlations between layer activations and brain signals were observed in somatosensory and visual cortices. Our results highlight the potential of combining imitation learning, brain imaging and videogames to uncover subject-specific relationships between brain and behavior.