PUB - Publikationen an der Universität Bielefeld

Visual speech is an integral part of communication. Yet it remains unclear whether semantic information carried by movements of the lips or tongue is represented in the same brain regions that mediate acoustic speech representations. Behaviourally, our ability to understand acoustic speech seems independent from that to understand visual speech, but neuroimaging studies suggest that acoustic and visual speech representations largely overlap. To resolve this discrepancy, and to understand whether acoustic and lip-reading speech comprehension are mediated by the same cerebral representations, we systematically probed where the brain represents acoustically and visually conveyed word identities in a human MEG study. We designed a single-trial classification paradigm to dissociate where cerebral representations merely reflect the sensory stimulus and where they are predictive of the participant’s percept. In general, those brain regions allowing for the highest word classification were distinct from those in which cerebral representations were predictive of participant’s percept. Across the brain, word representations were largely modality-specific and auditory and visual comprehension were mediated by distinct left-lateralised ventral and dorsal fronto-temporal regions, respectively. Only within the inferior frontal gyrus and the anterior temporal lobe did auditory and visual representations converge. These results provide a neural explanation for why acoustic speech comprehension is a poor predictor of lip-reading skills and suggests that those cerebral speech representations that encode word identity may be more modality-specific than often upheld.