Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events
Bresciani J-P, Darnmeier F, Ernst MO (2008)
Brain Research Bulletin 75(6): 753-760.
Zeitschriftenaufsatz
| Veröffentlicht | Englisch
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Autor*in
Bresciani, Jean-Pierre;
Darnmeier, Franziska;
Ernst, Marc O.UniBi
Abstract / Bemerkung
We investigated the interactions between visual, tactile and auditory sensory signals for the perception of sequences of events. Sequences of flashes, taps and beeps were presented simultaneously. For each session, subjects were instructed to count the number of events presented in one modality (Target) and to ignore the stimuli presented in the other modalities (Background). The number of events presented in the background sequence could differ from the number of events in the target sequence. For each session, we quantified the Background-evoked bias by comparing subjects' responses with and without Background (Target presented alone). Nine combinations between vision, touch and audition were tested. In each session but two, the Background significantly biased the Target. Vision was the most susceptible to Background-evoked bias and the least efficient in biasing the other two modalities. By contrast, audition was the least susceptible to Background-evoked bias and the most efficient in biasing the other two modalities. These differences were strongly correlated to the relative reliability of each modality. In line with this, the evoked biases were larger when the Background consisted of two instead of only one modality. These results show that for the perception of sequences of events: (1) vision, touch and audition are automatically integrated; (2) the respective contributions of the three modalities to the integrated percept differ; (3) the relative contribution of each modality depends on its relative reliability (1/variability); (4) task-irrelevant stimuli have more weight when presented in two rather than only one modality. (c) 2008 Elsevier Inc. All rights reserved.
Stichworte
vision;
multimodal integration;
virtual environments;
audition;
touch
Erscheinungsjahr
2008
Zeitschriftentitel
Brain Research Bulletin
Band
75
Ausgabe
6
Seite(n)
753-760
ISSN
0361-9230
Page URI
https://pub.uni-bielefeld.de/record/2288008
Zitieren
Bresciani J-P, Darnmeier F, Ernst MO. Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin. 2008;75(6):753-760.
Bresciani, J. - P., Darnmeier, F., & Ernst, M. O. (2008). Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin, 75(6), 753-760. https://doi.org/10.1016/j.brainresbull.2008.01.009
Bresciani, Jean-Pierre, Darnmeier, Franziska, and Ernst, Marc O. 2008. “Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events”. Brain Research Bulletin 75 (6): 753-760.
Bresciani, J. - P., Darnmeier, F., and Ernst, M. O. (2008). Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin 75, 753-760.
Bresciani, J.-P., Darnmeier, F., & Ernst, M.O., 2008. Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin, 75(6), p 753-760.
J.-P. Bresciani, F. Darnmeier, and M.O. Ernst, “Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events”, Brain Research Bulletin, vol. 75, 2008, pp. 753-760.
Bresciani, J.-P., Darnmeier, F., Ernst, M.O.: Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events. Brain Research Bulletin. 75, 753-760 (2008).
Bresciani, Jean-Pierre, Darnmeier, Franziska, and Ernst, Marc O. “Tri-modal integration of visual, tactile and auditory signals for the perception of sequences of events”. Brain Research Bulletin 75.6 (2008): 753-760.
Daten bereitgestellt von European Bioinformatics Institute (EBI)
21 Zitationen in Europe PMC
Daten bereitgestellt von Europe PubMed Central.
Late Bayesian inference in mental transformations.
Remington ED, Parks TV, Jazayeri M., Nat Commun 9(1), 2018
PMID: 30356049
Remington ED, Parks TV, Jazayeri M., Nat Commun 9(1), 2018
PMID: 30356049
Newly acquired audio-visual associations bias perception in binocular rivalry.
Einhäuser W, Methfessel P, Bendixen A., Vision Res 133(), 2017
PMID: 28237813
Einhäuser W, Methfessel P, Bendixen A., Vision Res 133(), 2017
PMID: 28237813
Central-peripheral differences in audiovisual and visuotactile event perception.
Chen YC, Maurer D, Lewis TL, Spence C, Shore DI., Atten Percept Psychophys 79(8), 2017
PMID: 28815431
Chen YC, Maurer D, Lewis TL, Spence C, Shore DI., Atten Percept Psychophys 79(8), 2017
PMID: 28815431
The dynamics of multimodal integration: The averaging diffusion model.
Turner BM, Gao J, Koenig S, Palfy D, L McClelland J., Psychon Bull Rev 24(6), 2017
PMID: 28275990
Turner BM, Gao J, Koenig S, Palfy D, L McClelland J., Psychon Bull Rev 24(6), 2017
PMID: 28275990
Superadditive and Subadditive Neural Processing of Dynamic Auditory-Visual Objects in the Presence of Congruent Odors.
Ohla K, Höchenberger R, Freiherr J, Lundström JN., Chem Senses 43(1), 2017
PMID: 29045615
Ohla K, Höchenberger R, Freiherr J, Lundström JN., Chem Senses 43(1), 2017
PMID: 29045615
Multisensory integration of redundant trisensory stimulation.
Hagmann CE, Russo N., Atten Percept Psychophys 78(8), 2016
PMID: 27562676
Hagmann CE, Russo N., Atten Percept Psychophys 78(8), 2016
PMID: 27562676
Persistency and flexibility of complex brain networks underlie dual-task interference.
Alavash M, Hilgetag CC, Thiel CM, Gießing C., Hum Brain Mapp 36(9), 2015
PMID: 26095953
Alavash M, Hilgetag CC, Thiel CM, Gießing C., Hum Brain Mapp 36(9), 2015
PMID: 26095953
Spatio-temporal distribution of brain activity associated with audio-visually congruent and incongruent speech and the McGurk Effect.
Pratt H, Bleich N, Mittelman N., Brain Behav 5(11), 2015
PMID: 26664791
Pratt H, Bleich N, Mittelman N., Brain Behav 5(11), 2015
PMID: 26664791
Natural stimuli from three coherent modalities enhance behavioral responses and electrophysiological cortical activity in humans.
Sella I, Reiner M, Pratt H., Int J Psychophysiol 93(1), 2014
PMID: 24315926
Sella I, Reiner M, Pratt H., Int J Psychophysiol 93(1), 2014
PMID: 24315926
Integration of auditory and tactile inputs in musical meter perception.
Huang J, Gamble D, Sarnlertsophon K, Wang X, Hsiao S., Adv Exp Med Biol 787(), 2013
PMID: 23716252
Huang J, Gamble D, Sarnlertsophon K, Wang X, Hsiao S., Adv Exp Med Biol 787(), 2013
PMID: 23716252
Observers can reliably identify illusory flashes in the illusory flash paradigm.
van Erp JB, Philippi TG, Werkhoven P., Exp Brain Res 226(1), 2013
PMID: 23354667
van Erp JB, Philippi TG, Werkhoven P., Exp Brain Res 226(1), 2013
PMID: 23354667
Sex discriminations made on the basis of ambiguous visual cues can be affected by the presence of an olfactory cue.
Hacker G, Brooks A, van der Zwan R., BMC Psychol 1(1), 2013
PMID: 25566362
Hacker G, Brooks A, van der Zwan R., BMC Psychol 1(1), 2013
PMID: 25566362
Spatio-temporal measures of electrophysiological correlates for behavioral multisensory enhancement during visual, auditory and somatosensory stimulation: A behavioral and ERP study.
Wang W, Hu L, Cui H, Xie X, Hu Y., Neurosci Bull 29(6), 2013
PMID: 24293020
Wang W, Hu L, Cui H, Xie X, Hu Y., Neurosci Bull 29(6), 2013
PMID: 24293020
Dynamic characteristics of multisensory facilitation and inhibition.
Wang WY, Hu L, Valentini E, Xie XB, Cui HY, Hu Y., Cogn Neurodyn 6(5), 2012
PMID: 24082962
Wang WY, Hu L, Valentini E, Xie XB, Cui HY, Hu Y., Cogn Neurodyn 6(5), 2012
PMID: 24082962
Feeling music: integration of auditory and tactile inputs in musical meter perception.
Huang J, Gamble D, Sarnlertsophon K, Wang X, Hsiao S., PLoS One 7(10), 2012
PMID: 23119038
Huang J, Gamble D, Sarnlertsophon K, Wang X, Hsiao S., PLoS One 7(10), 2012
PMID: 23119038
A "unity assumption" does not promote intersensory integration.
Misceo GF, Taylor NJ., Exp Psychol 58(5), 2011
PMID: 21592944
Misceo GF, Taylor NJ., Exp Psychol 58(5), 2011
PMID: 21592944
Audiotactile interactions in temporal perception.
Occelli V, Spence C, Zampini M., Psychon Bull Rev 18(3), 2011
PMID: 21400125
Occelli V, Spence C, Zampini M., Psychon Bull Rev 18(3), 2011
PMID: 21400125
The time-course of auditory and visual distraction effects in a new crossmodal paradigm.
Bendixen A, Grimm S, Deouell LY, Wetzel N, Mädebach A, Schröger E., Neuropsychologia 48(7), 2010
PMID: 20385149
Bendixen A, Grimm S, Deouell LY, Wetzel N, Mädebach A, Schröger E., Neuropsychologia 48(7), 2010
PMID: 20385149
Crossmodal congruency effects based on stimulus identity.
Frings C, Spence C., Brain Res 1354(), 2010
PMID: 20674555
Frings C, Spence C., Brain Res 1354(), 2010
PMID: 20674555
Sensory dominance in combinations of audio, visual and haptic stimuli.
Hecht D, Reiner M., Exp Brain Res 193(2), 2009
PMID: 18985327
Hecht D, Reiner M., Exp Brain Res 193(2), 2009
PMID: 18985327
Visual stimulus locking of EEG is modulated by temporal congruency of auditory stimuli.
Schall S, Quigley C, Onat S, König P., Exp Brain Res 198(2-3), 2009
PMID: 19526359
Schall S, Quigley C, Onat S, König P., Exp Brain Res 198(2-3), 2009
PMID: 19526359
50 References
Daten bereitgestellt von Europe PubMed Central.
The ventriloquist effect results from near-optimal bimodal integration.
Alais D, Burr D., Curr. Biol. 14(3), 2004
PMID: 14761661
Alais D, Burr D., Curr. Biol. 14(3), 2004
PMID: 14761661
Maximum Likelihood Integration of rapid flashes and beeps.
Andersen TS, Tiippana K, Sams M., Neurosci. Lett. 380(1-2), 2005
PMID: 15854769
Andersen TS, Tiippana K, Sams M., Neurosci. Lett. 380(1-2), 2005
PMID: 15854769
Integrating physically based sound models in a multimodal rendering architecture
Avanzini, Comput. Anim. Vir. Worlds 17(), 2006
Avanzini, Comput. Anim. Vir. Worlds 17(), 2006
Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition.
Bermant RI, Welch RB., Percept Mot Skills 42(43), 1976
PMID: 980644
Bermant RI, Welch RB., Percept Mot Skills 42(43), 1976
PMID: 980644
Effects of an auditory signal on visual reaction time.
Bernstein IH, Clark MH, Edelstein BA., J Exp Psychol 80(3), 1969
PMID: 5786157
Bernstein IH, Clark MH, Edelstein BA., J Exp Psychol 80(3), 1969
PMID: 5786157
Cross-modal bias and perceptual fusion with auditory-visual spatial discordance.
Bertelson P, Radeau M., Percept Psychophys 29(6), 1981
PMID: 7279586
Bertelson P, Radeau M., Percept Psychophys 29(6), 1981
PMID: 7279586
Vision and touch are automatically integrated for the perception of sequences of events.
Bresciani JP, Dammeier F, Ernst MO., J Vis 6(5), 2006
PMID: 16881788
Bresciani JP, Dammeier F, Ernst MO., J Vis 6(5), 2006
PMID: 16881788
Signal reliability modulates auditory-tactile integration for event counting.
Bresciani JP, Ernst MO., Neuroreport 18(11), 2007
PMID: 17589318
Bresciani JP, Ernst MO., Neuroreport 18(11), 2007
PMID: 17589318
Feeling what you hear: auditory signals can modulate tactile tap perception.
Bresciani JP, Ernst MO, Drewing K, Bouyer G, Maury V, Kheddar A., Exp Brain Res 162(2), 2004
PMID: 15791465
Bresciani JP, Ernst MO, Drewing K, Bouyer G, Maury V, Kheddar A., Exp Brain Res 162(2), 2004
PMID: 15791465
Interaction between sensory spatial after-effects and persistence of response following behavioral compensation.
Collins JK, Singer G., J Exp Psychol 77(2), 1968
PMID: 5655124
Collins JK, Singer G., J Exp Psychol 77(2), 1968
PMID: 5655124
Sensory adaptation and behavioral compensation with spatially transformed vision and hearing.
Day RH, Singer G., Psychol Bull 67(5), 1967
PMID: 6047171
Day RH, Singer G., Psychol Bull 67(5), 1967
PMID: 6047171
Multisensory integration, perception and ecological validity.
De Gelder B, Bertelson P., Trends Cogn. Sci. (Regul. Ed.) 7(10), 2003
PMID: 14550494
De Gelder B, Bertelson P., Trends Cogn. Sci. (Regul. Ed.) 7(10), 2003
PMID: 14550494
Efficient computation and cue integration with noisy population codes.
Deneve S, Latham PE, Pouget A., Nat. Neurosci. 4(8), 2001
PMID: 11477429
Deneve S, Latham PE, Pouget A., Nat. Neurosci. 4(8), 2001
PMID: 11477429
Visual-tactile spatial interaction in saccade generation.
Diederich A, Colonius H, Bockhorst D, Tabeling S., Exp Brain Res 148(3), 2002
PMID: 12541144
Diederich A, Colonius H, Bockhorst D, Tabeling S., Exp Brain Res 148(3), 2002
PMID: 12541144
The effect of auditory cues on the haptic perception of stiffness in virtual environments
DiFranco, Proc. ASME Dyn. Syst. Contr. Div. 61(), 1997
DiFranco, Proc. ASME Dyn. Syst. Contr. Div. 61(), 1997
Humans integrate visual and haptic information in a statistically optimal fashion.
Ernst MO, Banks MS., Nature 415(6870), 2002
PMID: 11807554
Ernst MO, Banks MS., Nature 415(6870), 2002
PMID: 11807554
Merging the senses into a robust percept.
Ernst MO, Bulthoff HH., Trends Cogn. Sci. (Regul. Ed.) 8(4), 2004
PMID: 15050512
Ernst MO, Bulthoff HH., Trends Cogn. Sci. (Regul. Ed.) 8(4), 2004
PMID: 15050512
The temporal cross-capture of audition and vision.
Fendrich R, Corballis PM., Percept Psychophys 63(4), 2001
PMID: 11436740
Fendrich R, Corballis PM., Percept Psychophys 63(4), 2001
PMID: 11436740
Factors involved in visual capture.
Fishkin SM, Pishkin V, Stahl ML., Percept Mot Skills 40(2), 1975
PMID: 1178308
Fishkin SM, Pishkin V, Stahl ML., Percept Mot Skills 40(2), 1975
PMID: 1178308
The case for feedforward multisensory convergence during early cortical processing.
Foxe JJ, Schroeder CE., Neuroreport 16(5), 2005
PMID: 15770144
Foxe JJ, Schroeder CE., Neuroreport 16(5), 2005
PMID: 15770144
On discriminating the rate of visual flicker and auditory flutter.
GEBHARD JW, MOWBRAY GH., Am J Psychol 72(), 1959
PMID: 13827044
GEBHARD JW, MOWBRAY GH., Am J Psychol 72(), 1959
PMID: 13827044
Viewing geometry determines how vision and haptics combine in size perception.
Gepshtein S, Banks MS., Curr. Biol. 13(6), 2003
PMID: 12646130
Gepshtein S, Banks MS., Curr. Biol. 13(6), 2003
PMID: 12646130
On the nature of intersensory facilitation of reaction time.
Gielen SC, Schmidt RA, Van den Heuvel PJ., Percept Psychophys 34(2), 1983
PMID: 6634374
Gielen SC, Schmidt RA, Van den Heuvel PJ., Percept Psychophys 34(2), 1983
PMID: 6634374
Audiotactile interactions in roughness perception.
Guest S, Catmur C, Lloyd D, Spence C., Exp Brain Res 146(2), 2002
PMID: 12195518
Guest S, Catmur C, Lloyd D, Spence C., Exp Brain Res 146(2), 2002
PMID: 12195518
Tactile dominance in speeded discrimination of textures.
Guest S, Spence C., Exp Brain Res 150(2), 2003
PMID: 12679862
Guest S, Spence C., Exp Brain Res 150(2), 2003
PMID: 12679862
What role does multisensory integration play in the visuotactile perception of texture?
Guest S, Spence C., Int J Psychophysiol 50(1-2), 2003
PMID: 14511837
Guest S, Spence C., Int J Psychophysiol 50(1-2), 2003
PMID: 14511837
Reaction time as a measure of intersensory facilitation.
HERSHENSON M., J Exp Psychol 63(), 1962
PMID: 13906889
HERSHENSON M., J Exp Psychol 63(), 1962
PMID: 13906889
Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey.
Hikosaka K, Iwai E, Saito H, Tanaka K., J. Neurophysiol. 60(5), 1988
PMID: 2462027
Hikosaka K, Iwai E, Saito H, Tanaka K., J. Neurophysiol. 60(5), 1988
PMID: 2462027
Hearing cheats touch, but less in congenitally blind than in sighted individuals.
Hotting K, Roder B., Psychol Sci 15(1), 2004
PMID: 14717833
Hotting K, Roder B., Psychol Sci 15(1), 2004
PMID: 14717833
Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys.
Hyvarinen J, Poranen A., Brain 97(4), 1974
PMID: 4434188
Hyvarinen J, Poranen A., Brain 97(4), 1974
PMID: 4434188
Parchment-skin illusion: sound-biased touch.
Jousmaki V, Hari R., Curr. Biol. 8(6), 1998
PMID: 9512426
Jousmaki V, Hari R., Curr. Biol. 8(6), 1998
PMID: 9512426
Perception of material from contact sounds
Klatzky, Presence: Teleoperat. Vir. Environ. 9(), 2000
Klatzky, Presence: Teleoperat. Vir. Environ. 9(), 2000
Functional properties of neurons in the temporo-parietal association cortex of awake monkey.
Leinonen L, Hyvarinen J, Sovijarvi AR., Exp Brain Res 39(2), 1980
PMID: 6772459
Leinonen L, Hyvarinen J, Sovijarvi AR., Exp Brain Res 39(2), 1980
PMID: 6772459
Bayesian inference with probabilistic population codes.
Ma WJ, Beck JM, Latham PE, Pouget A., Nat. Neurosci. 9(11), 2006
PMID: 17057707
Ma WJ, Beck JM, Latham PE, Pouget A., Nat. Neurosci. 9(11), 2006
PMID: 17057707
Auditory capture of vision: examining temporal ventriloquism.
Morein-Zamir S, Soto-Faraco S, Kingstone A., Brain Res Cogn Brain Res 17(1), 2003
PMID: 12763201
Morein-Zamir S, Soto-Faraco S, Kingstone A., Brain Res Cogn Brain Res 17(1), 2003
PMID: 12763201
Temporal characteristics of sensory interaction in choice reaction times.
Morrell LK., J Exp Psychol 77(1), 1968
PMID: 5663585
Morrell LK., J Exp Psychol 77(1), 1968
PMID: 5663585
Intersensory facilitation of reaction time: energy summation or preparation enhancement?
Nickerson RS., Psychol Rev 80(6), 1973
PMID: 4757060
Nickerson RS., Psychol Rev 80(6), 1973
PMID: 4757060
Auditory influences on visual temporal rate perception.
Recanzone GH., J. Neurophysiol. 89(2), 2003
PMID: 12574482
Recanzone GH., J. Neurophysiol. 89(2), 2003
PMID: 12574482
VISION AND TOUCH: AN EXPERIMENTALLY CREATED CONFLICT BETWEEN THE TWO SENSES.
ROCK I, VICTOR J., Science 143(3606), 1964
PMID: 14080333
ROCK I, VICTOR J., Science 143(3606), 1964
PMID: 14080333
Sound alters visual motion perception.
Sekuler R, Sekuler AB, Lau R., Nature 385(6614), 1997
PMID: 9002513
Sekuler R, Sekuler AB, Lau R., Nature 385(6614), 1997
PMID: 9002513
Illusions. What you see is what you hear.
Shams L, Kamitani Y, Shimojo S., Nature 408(6814), 2000
PMID: 11130706
Shams L, Kamitani Y, Shimojo S., Nature 408(6814), 2000
PMID: 11130706
Spatial adaptation and aftereffect with optically transformed vision: effects of active and passive responding and the relationship between test and exposure responses.
Singer G, Day RH., J Exp Psychol 71(5), 1966
PMID: 5939716
Singer G, Day RH., J Exp Psychol 71(5), 1966
PMID: 5939716
Cortex governs multisensory integration in the midbrain.
Stein BE, Wallace MW, Stanford TR, Jiang W., Neuroscientist 8(4), 2002
PMID: 12194499
Stein BE, Wallace MW, Stanford TR, Jiang W., Neuroscientist 8(4), 2002
PMID: 12194499
Integration of proprioceptive and visual position-information: An experimentally supported model.
van Beers RJ, Sittig AC, Gon JJ., J. Neurophysiol. 81(3), 1999
PMID: 10085361
van Beers RJ, Sittig AC, Gon JJ., J. Neurophysiol. 81(3), 1999
PMID: 10085361
Touch-induced visual illusion.
Violentyev A, Shimojo S, Shams L., Neuroreport 16(10), 2005
PMID: 15973157
Violentyev A, Shimojo S, Shams L., Neuroreport 16(10), 2005
PMID: 15973157
Audio-visual integration in temporal perception.
Wada Y, Kitagawa N, Noguchi K., Int J Psychophysiol 50(1-2), 2003
PMID: 14511840
Wada Y, Kitagawa N, Noguchi K., Int J Psychophysiol 50(1-2), 2003
PMID: 14511840
Note on the construction of diagram-balanced latin squares
Wagenaar, Psychol. Bull. 72(), 1969
Wagenaar, Psychol. Bull. 72(), 1969
When sound affects vision: effects of auditory grouping on visual motion perception.
Watanabe K, Shimojo S., Psychol Sci 12(2), 2001
PMID: 11340918
Watanabe K, Shimojo S., Psychol Sci 12(2), 2001
PMID: 11340918
Export
Markieren/ Markierung löschen
Markierte Publikationen
Web of Science
Dieser Datensatz im Web of Science®Quellen
PMID: 18394521
PubMed | Europe PMC
Suchen in