Conference on Cognitive Computational Neuroscience

Working memory (WM) enables humans and other animals to hold information temporarily for various kinds of mental processing. WM has limited capacity and the maintenance of information in WM involves interactions between multiple brain regions. To account for such properties, we built multi-system models of WM, i.e., models that involve both sensory and cognitive systems, and their interactions. Our contributions are twofold, involving engineering and science. Engineering-wise, we built a framework to systematically construct such models to generate and test hypotheses in neuroscience research. Our models take sensory stimuli in their raw form, and reproduce diverse behavioral and neural findings across classical and recent WM experiments. Science-wise, our framework allows us to dissect the sensory and cognitive system’s contribution to WM capacity limitation. Our models reproduced behavioral findings in several WM tasks commonly used to assess capacity limitation. We found human-like capacity limitations arise in models with sensory systems pre-trained to recognize natural images, but not in models trained end-to-end on WM tasks. Our results suggest that WM capacity limitation is partly attributed to the sensory system when it is optimized for naturalistic objectives other than tasks artificially designed to probe WM.