Generalization Demands Task-Appropriate Modular Neural Architectures
Ruiyi Zhang, Dora Angelaki, New York University, United States; Xaq Pitkow, Baylor College of Medicine, United States
Posters 1 Poster
Pacific Ballroom H-O
Thu, 25 Aug, 19:30 - 21:30 Pacific Time (UTC -8)
Although both use reward-based learning, it is still unclear why deep reinforcement learning (RL) agents, in general, perform worse than the brain in novel out-of-distribution (OOD) tasks. Here, we propose one reason: generalization requires task-appropriate modular neural architectures like the brain; inferior generalization abilities result from using architectures without task-appropriate modules. To verify this hypothesis, we used a spatial navigation task to train RL agents with different neural architectures varying in modularity, then compared their generalization abilities in a novel task presenting novel sensorimotor mappings. We found that, although all agents could master the training task, only those with highly modular architectures separating computations of task variables learned a robust state representation, which supported generalization in the novel task. Our work exemplifies the rationale of the architectural modularity, i.e., supporting generalization.