Conference on Cognitive Computational Neuroscience

The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelli- gence. Here, we explore if these computational trends have brought concomitant improvements in explaining the visual strategies that humans use to recognize objects. We compare two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects. We find a trade-off between the categorization accuracy of 85 different DNNs and the alignment of their visual strategies with humans. DNNs have progressively become less aligned with humans as they have become more accurate at object classification. We rectify this growing issue with our novel neural har- monizer, a general-purpose training routine that aligns DNN and human visual strategies while also improving object classification accuracy. Our work represents the first systematic demonstration that the scaling laws that have guided the development of DNNs have also produced worse models of human vision.