Conference on Cognitive Computational Neuroscience

Speech comprehension requires extracting semantic meaning from sound pressure waves. It is thought that comprehension requires a set of intermediate representations based on phonemic units, syllables, words, etc. Here we investigated the locus of cortical phonemic representations and the major phonemic representation within these phoneme-selective areas. For this purpose, we collected functional MRI data while subjects listened to narrative stories. We then compared the predictive power of voxelwise encoding models that used spectral features of sound, single phonemes, diphones, triphones and full word meaning embeddings. We identified regions in the superior temporal (STC), lateral temporal (LTC) and inferior prefrontal cortex (IPFC) that selectively responded to these three phonemic features. In all these phoneme-selective regions, diphone features produced significantly more accurate predictions of brain activity than did single phonemes or triphones. This result suggests that the most important phoneme-related representations in the brain occur at the level of diphones.