PUB - Publikationen an der Universität Bielefeld

Crossmodal correspondences describe our tendency to associate sensory features from different modalities with each other, such as the pitch of a sound with a specific size of a visual object. While such crossmodal correspondences (or associations) have been described in many behavioural studies their neurophysiological correlates remain debated. Under the current working model of multisensory perception both a low- and a high-level account seem plausible. That is, the neurophysiological processes shaping these associations could commence in early sensory regions, or may rather emerge only later and as a result from high-level multisensory integration processes or in semantic and object identification networks. We implemented a paradigm based on steady-state visual evoked potentials (SSVEP) to directly test the hypothesis that the associations between acoustic pitch and the visual features of size, hue or chromatic saturation are visible in brain activity from early visual cortices. Our results reveal a pitch-size congruency effect in the SSVEP over occipital electrodes and in the behavioural data, supporting a low-level account of this crossmodal association. For the associations between pitch and chromatic features we found no significant effects. We speculate that this signature of the pitch size association in early visual cortices reflects the successful pairing of congruent visual and acoustic object properties and may contribute to establishing causal relations between multisensory objects.