Conference on Cognitive Computational Neuroscience

What level of information is captured in subjective judgments of similarity between natural scenes? To better understand the level of description of the representational space captured with multiple arrangements of natural scene stimuli, we tested 58 participants across three different MA tasks arranging visual images, or sentence captions describing the visual images. We observed significant correlations between visual similarity judgments and sentence similarity judgments. Additionally, these behavioural RDMs were significantly correlated with computational models capturing semantic representations via sentence embeddings. Altogether our results provide evidence that multiple arrangements capture semantic-level representations.