Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events

Bresciani J-P, Darnmeier F, Ernst MO (2008)
Brain Research Bulletin 75(6): 753-760.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Autor*in
Bresciani, Jean-Pierre; Darnmeier, Franziska; Ernst, Marc O.UniBi
Abstract / Bemerkung
We investigated the interactions between visual, tactile and auditory sensory signals for the perception of sequences of events. Sequences of flashes, taps and beeps were presented simultaneously. For each session, subjects were instructed to count the number of events presented in one modality (Target) and to ignore the stimuli presented in the other modalities (Background). The number of events presented in the background sequence could differ from the number of events in the target sequence. For each session, we quantified the Background-evoked bias by comparing subjects' responses with and without Background (Target presented alone). Nine combinations between vision, touch and audition were tested. In each session but two, the Background significantly biased the Target. Vision was the most susceptible to Background-evoked bias and the least efficient in biasing the other two modalities. By contrast, audition was the least susceptible to Background-evoked bias and the most efficient in biasing the other two modalities. These differences were strongly correlated to the relative reliability of each modality. In line with this, the evoked biases were larger when the Background consisted of two instead of only one modality. These results show that for the perception of sequences of events: (1) vision, touch and audition are automatically integrated; (2) the respective contributions of the three modalities to the integrated percept differ; (3) the relative contribution of each modality depends on its relative reliability (1/variability); (4) task-irrelevant stimuli have more weight when presented in two rather than only one modality. (c) 2008 Elsevier Inc. All rights reserved.
Stichworte
vision; multimodal integration; virtual environments; audition; touch
Erscheinungsjahr
2008
Zeitschriftentitel
Brain Research Bulletin
Band
75
Ausgabe
6
Seite(n)
753-760
ISSN
0361-9230
Page URI
https://pub.uni-bielefeld.de/record/2288008

Zitieren

Bresciani J-P, Darnmeier F, Ernst MO. Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin. 2008;75(6):753-760.
Bresciani, J. - P., Darnmeier, F., & Ernst, M. O. (2008). Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin, 75(6), 753-760. https://doi.org/10.1016/j.brainresbull.2008.01.009
Bresciani, Jean-Pierre, Darnmeier, Franziska, and Ernst, Marc O. 2008. “Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events”. Brain Research Bulletin 75 (6): 753-760.
Bresciani, J. - P., Darnmeier, F., and Ernst, M. O. (2008). Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin 75, 753-760.
Bresciani, J.-P., Darnmeier, F., & Ernst, M.O., 2008. Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin, 75(6), p 753-760.
J.-P. Bresciani, F. Darnmeier, and M.O. Ernst, “Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events”, Brain Research Bulletin, vol. 75, 2008, pp. 753-760.
Bresciani, J.-P., Darnmeier, F., Ernst, M.O.: Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin. 75, 753-760 (2008).
Bresciani, Jean-Pierre, Darnmeier, Franziska, and Ernst, Marc O. “Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events”. Brain Research Bulletin 75.6 (2008): 753-760.

21 Zitationen in Europe PMC

Daten bereitgestellt von Europe PubMed Central.

Late Bayesian inference in mental transformations.
Remington ED, Parks TV, Jazayeri M., Nat Commun 9(1), 2018
PMID: 30356049
Newly acquired audio-visual associations bias perception in binocular rivalry.
Einhäuser W, Methfessel P, Bendixen A., Vision Res 133(), 2017
PMID: 28237813
Central-peripheral differences in audiovisual and visuotactile event perception.
Chen YC, Maurer D, Lewis TL, Spence C, Shore DI., Atten Percept Psychophys 79(8), 2017
PMID: 28815431
The dynamics of multimodal integration: The averaging diffusion model.
Turner BM, Gao J, Koenig S, Palfy D, L McClelland J., Psychon Bull Rev 24(6), 2017
PMID: 28275990
Superadditive and Subadditive Neural Processing of Dynamic Auditory-Visual Objects in the Presence of Congruent Odors.
Ohla K, Höchenberger R, Freiherr J, Lundström JN., Chem Senses 43(1), 2017
PMID: 29045615
Multisensory integration of redundant trisensory stimulation.
Hagmann CE, Russo N., Atten Percept Psychophys 78(8), 2016
PMID: 27562676
Persistency and flexibility of complex brain networks underlie dual-task interference.
Alavash M, Hilgetag CC, Thiel CM, Gießing C., Hum Brain Mapp 36(9), 2015
PMID: 26095953
Integration of auditory and tactile inputs in musical meter perception.
Huang J, Gamble D, Sarnlertsophon K, Wang X, Hsiao S., Adv Exp Med Biol 787(), 2013
PMID: 23716252
Observers can reliably identify illusory flashes in the illusory flash paradigm.
van Erp JB, Philippi TG, Werkhoven P., Exp Brain Res 226(1), 2013
PMID: 23354667
Dynamic characteristics of multisensory facilitation and inhibition.
Wang WY, Hu L, Valentini E, Xie XB, Cui HY, Hu Y., Cogn Neurodyn 6(5), 2012
PMID: 24082962
Feeling music: integration of auditory and tactile inputs in musical meter perception.
Huang J, Gamble D, Sarnlertsophon K, Wang X, Hsiao S., PLoS One 7(10), 2012
PMID: 23119038
A "unity assumption" does not promote intersensory integration.
Misceo GF, Taylor NJ., Exp Psychol 58(5), 2011
PMID: 21592944
Audiotactile interactions in temporal perception.
Occelli V, Spence C, Zampini M., Psychon Bull Rev 18(3), 2011
PMID: 21400125
The time-course of auditory and visual distraction effects in a new crossmodal paradigm.
Bendixen A, Grimm S, Deouell LY, Wetzel N, Mädebach A, Schröger E., Neuropsychologia 48(7), 2010
PMID: 20385149
Crossmodal congruency effects based on stimulus identity.
Frings C, Spence C., Brain Res 1354(), 2010
PMID: 20674555
Sensory dominance in combinations of audio, visual and haptic stimuli.
Hecht D, Reiner M., Exp Brain Res 193(2), 2009
PMID: 18985327
Visual stimulus locking of EEG is modulated by temporal congruency of auditory stimuli.
Schall S, Quigley C, Onat S, König P., Exp Brain Res 198(2-3), 2009
PMID: 19526359

50 References

Daten bereitgestellt von Europe PubMed Central.

Maximum Likelihood Integration of rapid flashes and beeps.
Andersen TS, Tiippana K, Sams M., Neurosci. Lett. 380(1-2), 2005
PMID: 15854769
Integrating physically based sound models in a multimodal rendering architecture
Avanzini, Comput. Anim. Vir. Worlds 17(), 2006
Effects of an auditory signal on visual reaction time.
Bernstein IH, Clark MH, Edelstein BA., J Exp Psychol 80(3), 1969
PMID: 5786157
Cross-modal bias and perceptual fusion with auditory-visual spatial discordance.
Bertelson P, Radeau M., Percept Psychophys 29(6), 1981
PMID: 7279586
Signal reliability modulates auditory-tactile integration for event counting.
Bresciani JP, Ernst MO., Neuroreport 18(11), 2007
PMID: 17589318
Feeling what you hear: auditory signals can modulate tactile tap perception.
Bresciani JP, Ernst MO, Drewing K, Bouyer G, Maury V, Kheddar A., Exp Brain Res 162(2), 2004
PMID: 15791465
Multisensory integration, perception and ecological validity.
De Gelder B, Bertelson P., Trends Cogn. Sci. (Regul. Ed.) 7(10), 2003
PMID: 14550494
Efficient computation and cue integration with noisy population codes.
Deneve S, Latham PE, Pouget A., Nat. Neurosci. 4(8), 2001
PMID: 11477429
Visual-tactile spatial interaction in saccade generation.
Diederich A, Colonius H, Bockhorst D, Tabeling S., Exp Brain Res 148(3), 2002
PMID: 12541144
The effect of auditory cues on the haptic perception of stiffness in virtual environments
DiFranco, Proc. ASME Dyn. Syst. Contr. Div. 61(), 1997
Merging the senses into a robust percept.
Ernst MO, Bulthoff HH., Trends Cogn. Sci. (Regul. Ed.) 8(4), 2004
PMID: 15050512
The temporal cross-capture of audition and vision.
Fendrich R, Corballis PM., Percept Psychophys 63(4), 2001
PMID: 11436740
Factors involved in visual capture.
Fishkin SM, Pishkin V, Stahl ML., Percept Mot Skills 40(2), 1975
PMID: 1178308
On discriminating the rate of visual flicker and auditory flutter.
GEBHARD JW, MOWBRAY GH., Am J Psychol 72(), 1959
PMID: 13827044
On the nature of intersensory facilitation of reaction time.
Gielen SC, Schmidt RA, Van den Heuvel PJ., Percept Psychophys 34(2), 1983
PMID: 6634374
Audiotactile interactions in roughness perception.
Guest S, Catmur C, Lloyd D, Spence C., Exp Brain Res 146(2), 2002
PMID: 12195518
Tactile dominance in speeded discrimination of textures.
Guest S, Spence C., Exp Brain Res 150(2), 2003
PMID: 12679862
Reaction time as a measure of intersensory facilitation.
HERSHENSON M., J Exp Psychol 63(), 1962
PMID: 13906889
Parchment-skin illusion: sound-biased touch.
Jousmaki V, Hari R., Curr. Biol. 8(6), 1998
PMID: 9512426
Hearing visual motion in depth.
Kitagawa N, Ichihara S., Nature 416(6877), 2002
PMID: 11894093
Perception of material from contact sounds
Klatzky, Presence: Teleoperat. Vir. Environ. 9(), 2000
Functional properties of neurons in the temporo-parietal association cortex of awake monkey.
Leinonen L, Hyvarinen J, Sovijarvi AR., Exp Brain Res 39(2), 1980
PMID: 6772459
Bayesian inference with probabilistic population codes.
Ma WJ, Beck JM, Latham PE, Pouget A., Nat. Neurosci. 9(11), 2006
PMID: 17057707
Auditory capture of vision: examining temporal ventriloquism.
Morein-Zamir S, Soto-Faraco S, Kingstone A., Brain Res Cogn Brain Res 17(1), 2003
PMID: 12763201
Auditory influences on visual temporal rate perception.
Recanzone GH., J. Neurophysiol. 89(2), 2003
PMID: 12574482
Sound alters visual motion perception.
Sekuler R, Sekuler AB, Lau R., Nature 385(6614), 1997
PMID: 9002513
Illusions. What you see is what you hear.
Shams L, Kamitani Y, Shimojo S., Nature 408(6814), 2000
PMID: 11130706
AUDITORY FLUTTER-DRIVING OF VISUAL FLICKER.
SHIPLEY T., Science 145(3638), 1964
PMID: 14173429
Cortex governs multisensory integration in the midbrain.
Stein BE, Wallace MW, Stanford TR, Jiang W., Neuroscientist 8(4), 2002
PMID: 12194499
Integration of proprioceptive and visual position-information: An experimentally supported model.
van Beers RJ, Sittig AC, Gon JJ., J. Neurophysiol. 81(3), 1999
PMID: 10085361
Touch-induced visual illusion.
Violentyev A, Shimojo S, Shams L., Neuroreport 16(10), 2005
PMID: 15973157
Audio-visual integration in temporal perception.
Wada Y, Kitagawa N, Noguchi K., Int J Psychophysiol 50(1-2), 2003
PMID: 14511840
Note on the construction of diagram-balanced latin squares
Wagenaar, Psychol. Bull. 72(), 1969
Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®
Quellen

PMID: 18394521
PubMed | Europe PMC

Suchen in

Google Scholar