The neural mechanisms of reliability weighted integration of shape information from vision and touch

Helbig HB, Ernst MO, Ricciardi E, Pietrini P, Thielscher A, Mayer KM, Schultz J, Noppeney U (2011)
NeuroImage 60(2): 1063-1072.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
Helbig, Hannah B.; Ernst, Marc O.UniBi; Ricciardi, Emiliano; Pietrini, Pietro; Thielscher, Axel; Mayer, Katja M.; Schultz, Johannes; Noppeney, Uta
Abstract / Bemerkung
Behaviourally, humans have been shown to integrate multisensory information in a statistically-optimal fashion by averaging the individual unisensory estimates according to their relative reliabilities. This form of integration is optimal in that it yields the most reliable (i.e. least variable) multisensory percept. The present study investigates the neural mechanisms underlying integration of visual and tactile shape information at the macroscopic scale of the regional BOLD response. Observers discriminated the shapes of ellipses that were presented bimodally (visual-tactile) or visually alone. A 2 x 5 factorial design manipulated (i) the presence vs. absence of tactile shape information and (ii) the reliability of the visual shape information (five levels). We then investigated whether regional activations underlying tactile shape discrimination depended on the reliability of visual shape. Indeed, in primary somatosensory cortices (bilateral BA2) and the superior parietal lobe the responses to tactile shape input were increased when the reliability of visual shape information was reduced. Conversely, tactile inputs suppressed visual activations in the right posterior fusiform, when the visual signal was blurred and unreliable. Somatosensory and visual cortices may sustain integration of visual and tactile shape information either via direct connections from visual areas or top-down effects from higher order parietal areas.
Page URI


Helbig HB, Ernst MO, Ricciardi E, et al. The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage. 2011;60(2):1063-1072.
Helbig, H. B., Ernst, M. O., Ricciardi, E., Pietrini, P., Thielscher, A., Mayer, K. M., Schultz, J., et al. (2011). The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage, 60(2), 1063-1072.
Helbig, Hannah B., Ernst, Marc O., Ricciardi, Emiliano, Pietrini, Pietro, Thielscher, Axel, Mayer, Katja M., Schultz, Johannes, and Noppeney, Uta. 2011. “The neural mechanisms of reliability weighted integration of shape information from vision and touch”. NeuroImage 60 (2): 1063-1072.
Helbig, H. B., Ernst, M. O., Ricciardi, E., Pietrini, P., Thielscher, A., Mayer, K. M., Schultz, J., and Noppeney, U. (2011). The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage 60, 1063-1072.
Helbig, H.B., et al., 2011. The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage, 60(2), p 1063-1072.
H.B. Helbig, et al., “The neural mechanisms of reliability weighted integration of shape information from vision and touch”, NeuroImage, vol. 60, 2011, pp. 1063-1072.
Helbig, H.B., Ernst, M.O., Ricciardi, E., Pietrini, P., Thielscher, A., Mayer, K.M., Schultz, J., Noppeney, U.: The neural mechanisms of reliability weighted integration of shape information from vision and touch. NeuroImage. 60, 1063-1072 (2011).
Helbig, Hannah B., Ernst, Marc O., Ricciardi, Emiliano, Pietrini, Pietro, Thielscher, Axel, Mayer, Katja M., Schultz, Johannes, and Noppeney, Uta. “The neural mechanisms of reliability weighted integration of shape information from vision and touch”. NeuroImage 60.2 (2011): 1063-1072.
Alle Dateien verfügbar unter der/den folgenden Lizenz(en):
Copyright Statement:
Dieses Objekt ist durch das Urheberrecht und/oder verwandte Schutzrechte geschützt. [...]
Access Level
OA Open Access
Zuletzt Hochgeladen
MD5 Prüfsumme

24 Zitationen in Europe PMC

Daten bereitgestellt von Europe PubMed Central.

3D-printed model improves clinical assessment of surgeons on anatomy.
Zheng B, Wang X, Zheng Y, Feng J., J Robot Surg 13(1), 2019
PMID: 29693206
Seeing Your Foot Move Changes Muscle Proprioceptive Feedback.
Ackerley R, Chancel M, Aimonetti JM, Ribot-Ciscar E, Kavounoudias A., eNeuro 6(2), 2019
PMID: 30923738
Processing of haptic texture information over sequential exploration movements.
Lezkan A, Drewing K., Atten Percept Psychophys 80(1), 2018
PMID: 28975581
How prior expectations shape multisensory perception.
Gau R, Noppeney U., Neuroimage 124(pt a), 2016
PMID: 26419391
Stimulus intensity modulates multisensory temporal processing.
Krueger Fister J, Stevenson RA, Nidiffer AR, Barnett ZP, Wallace MT., Neuropsychologia 88(), 2016
PMID: 26920937
Optimal visuotactile integration for velocity discrimination of self-hand movements.
Chancel M, Blanchard C, Guerraz M, Montagnini A, Kavounoudias A., J Neurophysiol 116(3), 2016
PMID: 27385802
Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception.
Kilteni K, Maselli A, Kording KP, Slater M., Front Hum Neurosci 9(), 2015
PMID: 25852524
Whole brain mapping of visual and tactile convergence in the macaque monkey.
Guipponi O, Cléry J, Odouard S, Wardak C, Ben Hamed S., Neuroimage 117(), 2015
PMID: 25988226
Spatial orienting in complex audiovisual environments.
Nardo D, Santangelo V, Macaluso E., Hum Brain Mapp 35(4), 2014
PMID: 23616340
The cortical distribution of multisensory neurons was modulated by multisensory experience.
Xu J, Sun X, Zhou X, Zhang J, Yu L., Neuroscience 272(), 2014
PMID: 24813435
Subclinical visuospatial impairment in Parkinson's disease: the role of Basal Ganglia and limbic system.
Caproni S, Muti M, Di Renzo A, Principi M, Caputo N, Calabresi P, Tambasco N., Front Neurol 5(), 2014
PMID: 25157239
Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons.
Fetsch CR, DeAngelis GC, Angelaki DE., Nat Rev Neurosci 14(6), 2013
PMID: 23686172
A neural hierarchy for illusions of time: duration adaptation precedes multisensory integration.
Heron J, Hotchkiss J, Aaen-Stockdale C, Roach NW, Whitaker D., J Vis 13(14), 2013
PMID: 24306853
Differential representations of prior and likelihood uncertainty in the human brain.
Vilares I, Howard JD, Fernandes HL, Gottfried JA, Kording KP., Curr Biol 22(18), 2012
PMID: 22840519

84 References

Daten bereitgestellt von Europe PubMed Central.

Visuo-haptic object-related activation in the ventral visual pathway.
Amedi A, Malach R, Hendler T, Peled S, Zohary E., Nat. Neurosci. 4(3), 2001
PMID: 11224551
Convergence of visual and tactile shape processing in the human lateral occipital complex.
Amedi A, Jacobson G, Hendler T, Malach R, Zohary E., Cereb. Cortex 12(11), 2002
PMID: 12379608
Functional imaging of human crossmodal identification and object recognition.
Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ., Exp Brain Res 166(3-4), 2005
PMID: 16028028
Multisensory integration in the ventral intraparietal area of the macaque monkey.
Avillac M, Ben Hamed S, Duhamel JR., J. Neurosci. 27(8), 2007
PMID: 17314288
The functional anatomy of visual-tactile integration in man: a study using positron emission tomography.
Banati RB, Goerres GW, Tjoa C, Aggleton JP, Grasby P., Neuropsychologia 38(2), 2000
PMID: 10660224
Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions.
Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI., J Cogn Neurosci 17(3), 2005
PMID: 15813999
Unraveling multisensory integration: patchy organization within human STS multisensory cortex.
Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A., Nat. Neurosci. 7(11), 2004
PMID: 15475952
Neural substrates of reliability-weighted visual–tactile multisensory integration
Beauchamp, Front. Syst. Neurosci. 4(), 2010
Hierarchical processing of tactile shape in the human brain.
Bodegard A, Geyer S, Grefkes C, Zilles K, Roland PE., Neuron 31(2), 2001
PMID: 11502261
Neural basis of the ventriloquist illusion.
Bonath B, Noesselt T, Martinez A, Mishra J, Schwiecker K, Heinze HJ, Hillyard SA., Curr. Biol. 17(19), 2007
PMID: 17884498
Tactile attention tasks enhance activation in somatosensory regions of parietal cortex: a positron emission tomography study.
Burton H, Abend NS, MacLeod AM, Sinclair RJ, Snyder AZ, Raichle ME., Cereb. Cortex 9(7), 1999
PMID: 10554989
Object familiarity modulates effective connectivity during haptic shape perception.
Deshpande G, Hu X, Lacey S, Stilla R, Sathian K., Neuroimage 49(3), 2009
PMID: 19732841
A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data.
Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K., Neuroimage 25(4), 2005
PMID: 15850749
Merging the senses into a robust percept
Ernst, Trends Cogn. Neurosci. 8(), 2004
Anatomical mapping of functional activation in stereotactic coordinate space.
Evans AC, Marrett S, Neelin P, Collins L, Worsley K, Dai W, Milot S, Meyer E, Bub D., Neuroimage 1(1), 1992
PMID: 9343556
Statistical parametric maps in functional imaging a general linear approach
Friston, Hum. Brain Mapp. 2(), 1995
How many subjects constitute a study?
Friston KJ, Holmes AP, Worsley KJ., Neuroimage 10(1), 1999
PMID: 10385576
Integration of visual and tactile signals from the hand in the human brain: an FMRI study.
Gentile G, Petkova VI, Ehrsson HH., J. Neurophysiol. 105(2), 2010
PMID: 21148091
Is neocortex essentially multisensory?
Ghazanfar AA, Schroeder CE., Trends Cogn. Sci. (Regul. Ed.) 10(6), 2006
PMID: 16713325
Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex.
Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK., J. Neurosci. 25(20), 2005
PMID: 15901781
Differential processing of objects under various viewing conditions in the human lateral occipital complex.
Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R., Neuron 24(1), 1999
PMID: 10677037
Neural correlates of multisensory cue integration in macaque MSTd.
Gu Y, Angelaki DE, Deangelis GC., Nat. Neurosci. 11(10), 2008
PMID: 18776893
Optimal integration of shape information from vision and touch.
Helbig HB, Ernst MO., Exp Brain Res 179(4), 2007
PMID: 17225091
Knowledge about a common source can promote visual- haptic integration.
Helbig HB, Ernst MO., Perception 36(10), 2007
PMID: 18265835
Visual–haptic cue weighting is independent of modality-specific attention
Helbig, J. Vis. 8(21), 2008
Slant from texture and disparity cues: optimal cue combination.
Hillis JM, Watt SJ, Landy MS, Banks MS., J Vis 4(12), 2004
PMID: 15669906
Bilateral hand representation in the postcentral somatosensory cortex.
Iwamura Y, Iriki A, Tanaka M., Nature 369(6481), 1994
PMID: 8202155
Haptic study of three-dimensional objects activates extrastriate visual areas.
James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA., Neuropsychologia 40(10), 2002
PMID: 11992658
Do early sensory cortices integrate cross-modal information?
Kayser C, Logothetis NK., Brain Struct Funct 212(2), 2007
PMID: 17717687
Integration of touch and sound in auditory cortex.
Kayser C, Petkov CI, Augath M, Logothetis NK., Neuron 48(2), 2005
PMID: 16242415
Functional imaging reveals visual modulation of specific fields in auditory cortex.
Kayser C, Petkov CI, Augath M, Logothetis NK., J. Neurosci. 27(8), 2007
PMID: 17314280
Visual modulation of neurons in auditory cortex.
Kayser C, Petkov CI, Logothetis NK., Cereb. Cortex 18(7), 2008
PMID: 18180245
Multisensory activation of the intraparietal area when classifying grating orientation: a functional magnetic resonance imaging study.
Kitada R, Kito T, Saito DN, Kochiyama T, Matsumura M, Sadato N, Lederman SJ., J. Neurosci. 26(28), 2006
PMID: 16837597
Do humans optimally integrate stereo and texture information for judgments of surface slant?
Knill, Vis. Res. 43(), 2003
Neuronal oscillations and multisensory interaction in primary auditory cortex.
Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE., Neuron 53(2), 2007
PMID: 17224408
Tactile shape discrimination recruits human lateral occipital complex during early perceptual processing.
Lucan JN, Foxe JJ, Gomez-Ramirez M, Sathian K, Molholm S., Hum Brain Mapp 31(11), 2010
PMID: 20162607
Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex.
Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RB., Proc. Natl. Acad. Sci. U.S.A. 92(18), 1995
PMID: 7667258
Multisensory interactions within human primary cortices revealed by BOLD dynamics.
Martuzzi R, Murray MM, Michel CM, Thiran JP, Maeder PP, Clarke S, Meuli RA., Cereb. Cortex 17(7), 2006
PMID: 16968869
Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study.
Molholm S, Ritter W, Javitt DC, Foxe JJ., Cereb. Cortex 14(4), 2004
PMID: 15028649
Multisensory integration in macaque visual cortex depends on cue reliability.
Morgan ML, Deangelis GC, Angelaki DE., Neuron 59(4), 2008
PMID: 18760701
Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices.
Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J., J. Neurosci. 27(42), 2007
PMID: 17942738
Characterization of multisensory integration with fMRI: experimental design, statistical analysis and interpretation
Noppeney, 2011
Spatial attention modulates the cortical somatosensory representation of the digits in humans.
Noppeney U, Waberski TD, Gobbele R, Buchner H., Neuroreport 10(15), 1999
PMID: 10574549
The effect of prior visual information on recognition of speech and sounds.
Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ., Cereb. Cortex 18(3), 2007
PMID: 17617658
Perceptual decisions formed by accumulation of audiovisual evidence in prefrontal cortex.
Noppeney U, Ostwald D, Werner S., J. Neurosci. 30(21), 2010
PMID: 20505110
Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception.
Peltier S, Stilla R, Mariola E, LaConte S, Hu X, Sathian K., Neuropsychologia 45(3), 2006
PMID: 16616940
Beyond sensory images: Object-based representation in the human ventral pathway.
Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu WH, Cohen L, Guazzelli M, Haxby JV., Proc. Natl. Acad. Sci. U.S.A. 101(15), 2004
PMID: 15064396
Task-specific recruitment of dorsal and ventral visual areas during tactile perception.
Prather SC, Votaw JR, Sathian K., Neuropsychologia 42(8), 2004
PMID: 15093147
Tactile-visual cross-modal shape matching: a functional MRI study.
Saito DN, Okada T, Morita Y, Yonekura Y, Sadato N., Brain Res Cogn Brain Res 17(1), 2003
PMID: 12763188
Selective visuo-haptic processing of shape and texture.
Stilla R, Sathian K., Hum Brain Mapp 29(10), 2008
PMID: 17924535
Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks
Stoesz, Int. J. Psychophysiol. 50(1–2), 2003
Using fMR-adaptation to study visual–tactile integration of objects in humans
Tal, 2009
Combining information: probability summation and probability averaging in detection and discrimination
Treisman, Psychol. Methods 3(), 1998
Integration of letters and speech sounds in the human brain.
van Atteveldt N, Formisano E, Goebel R, Blomert L., Neuron 43(2), 2004
PMID: 15260962
Tactile form and location processing in the human brain.
Van Boven RW, Ingeholm JE, Beauchamp MS, Bikle PC, Ungerleider LG., Proc. Natl. Acad. Sci. U.S.A. 102(35), 2005
PMID: 16116098
Representation and integration of multiple sensory inputs in primate superior colliculus.
Wallace MT, Wilkinson LK, Stein BE., J. Neurophysiol. 76(2), 1996
PMID: 8871234
Multisensory cortical processing of object shape and its relation to mental imagery.
Zhang M, Weisser VD, Stilla R, Prather SC, Sathian K., Cogn Affect Behav Neurosci 4(2), 2004
PMID: 15460931
Tactile discrimination of grating orientation: fMRI activation patterns.
Zhang M, Mariola E, Stilla R, Stoesz M, Mao H, Hu X, Sathian K., Hum Brain Mapp 25(4), 2005
PMID: 15852384

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®

PMID: 22001262
PubMed | Europe PMC

Suchen in

Google Scholar