Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity

Kayser S, Kayser C (2018)
Scientific Reports 8(1): 3742.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
OA 2.11 MB
Abstract / Bemerkung
A well-known effect in multisensory perception is that congruent information received by different senses usually leads to faster and more accurate responses. Less well understood are trial-by-trial interactions, whereby the multisensory composition of stimuli experienced during previous trials shapes performance during a subsequent trial. We here exploit the analogy of multisensory paradigms with classical flanker tasks to investigate the neural correlates underlying trial-by-trial interactions of multisensory congruency. Studying an audio-visual motion task, we demonstrate that congruency benefits for accuracy and reaction times are reduced following an audio-visual incongruent compared to a congruent preceding trial. Using single trial analysis of motion-sensitive EEG components we then localize current-trial and serial interaction effects within distinct brain regions: while the multisensory congruency experienced during the current trial influences the encoding of task-relevant information in sensory-specific brain regions, the serial interaction arises from task-relevant processes within the inferior frontal lobe. These results highlight parallels between multisensory paradigms and classical flanker tasks and demonstrate a role of amodal association cortices in shaping perception based on the history of multisensory congruency.
Scientific Reports
Page URI


Kayser S, Kayser C. Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity. Scientific Reports. 2018;8(1): 3742.
Kayser, S., & Kayser, C. (2018). Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity. Scientific Reports, 8(1), 3742. doi:10.1038/s41598-018-22137-8
Kayser, Stephanie, and Kayser, Christoph. 2018. “Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity”. Scientific Reports 8 (1): 3742.
Kayser, S., and Kayser, C. (2018). Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity. Scientific Reports 8:3742.
Kayser, S., & Kayser, C., 2018. Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity. Scientific Reports, 8(1): 3742.
S. Kayser and C. Kayser, “Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity”, Scientific Reports, vol. 8, 2018, : 3742.
Kayser, S., Kayser, C.: Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity. Scientific Reports. 8, : 3742 (2018).
Kayser, Stephanie, and Kayser, Christoph. “Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity”. Scientific Reports 8.1 (2018): 3742.
Alle Dateien verfügbar unter der/den folgenden Lizenz(en):
Creative Commons Namensnennung 4.0 International Public License (CC-BY 4.0):
Access Level
OA Open Access
Zuletzt Hochgeladen
MD5 Prüfsumme

65 References

Daten bereitgestellt von Europe PubMed Central.

Recalibration of temporal order perception by exposure to audio-visual asynchrony.
Vroomen J, Keetels M, de Gelder B, Bertelson P., Brain Res Cogn Brain Res 22(1), 2004
PMID: 15561498
Recalibration of perceived time across sensory modalities.
Hanson JV, Heron J, Whitaker D., Exp Brain Res 185(2), 2008
PMID: 18236035
Rapid recalibration to audiovisual asynchrony.
Van der Burg E, Alais D, Cass J., J. Neurosci. 33(37), 2013
PMID: 24027264
Computational characterization of visually induced auditory spatial adaptation.
Wozny DR, Shams L., Front Integr Neurosci 5(), 2011
PMID: 22069383
Predicting auditory space calibration from recent multisensory experience.
Mendonca C, Escher A, van de Par S, Colonius H., Exp Brain Res 233(7), 2015
PMID: 25795081
Accumulation and decay of visual capture and the ventriloquism aftereffect caused by brief audio-visual disparities.
Bosen AK, Fleming JT, Allen PD, O'Neill WE, Paige GD., Exp Brain Res 235(2), 2016
PMID: 27837258
Multisensory simultaneity recalibration: storage of the aftereffect in the absence of counterevidence.
Machulla TK, Di Luca M, Froehlich E, Ernst MO., Exp Brain Res 217(1), 2011
PMID: 22207361
A model-based comparison of three theories of audiovisual temporal recalibration.
Yarrow K, Minaei S, Arnold DH., Cogn Psychol 83(), 2015
PMID: 26545105
Opposite Effects of Recent History on Perception and Decision.
Fritsche M, Mostert P, de Lange FP., Curr. Biol. 27(4), 2017
PMID: 28162897
Hysteresis in the perception of motion direction as evidence for neural cooperativity.
Williams D, Phillips G, Sekuler R., Nature 324(6094), 1986
PMID: 3785395
Multi-timescale perceptual history resolves visual ambiguity.
Brascamp JW, Knapen TH, Kanai R, Noest AJ, van Ee R, van den Berg AV., PLoS ONE 3(1), 2008
PMID: 18231584
Conflict monitoring and cognitive control.
Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD., Psychol Rev 108(3), 2001
PMID: 11488380
Optimizing the use of information: strategic control of activation of responses.
Gratton G, Coles MG, Donchin E., J Exp Psychol Gen 121(4), 1992
PMID: 1431740
The Gratton effect remains after controlling for contingencies and stimulus repetitions.
Blais C, Stefanidi A, Brewer GA., Front Psychol 5(), 2014
PMID: 25386151
Spatial attention and audiovisual interactions in apparent motion.
Sanabria D, Soto-Faraco S, Spence C., J Exp Psychol Hum Percept Perform 33(4), 2007
PMID: 17683238
Capturing spatial attention with multisensory cues: a review.
Spence C, Santangelo V., Hear. Res. 258(1-2), 2009
PMID: 19409472
Improvement of visual contrast detection by a simultaneous sound.
Lippert M, Logothetis NK, Kayser C., Brain Res. 1173(), 2007
PMID: 17765208
0 + 1 > 1: How adding noninformative sound improves performance on a visual task.
Kim R, Peters MA, Shams L., Psychol Sci 23(1), 2011
PMID: 22127367
The Psychophysics Toolbox.
Brainard DH., Spat Vis 10(4), 1997
PMID: 9176952
FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data.
Oostenveld R, Fries P, Maris E, Schoffelen JM., Comput Intell Neurosci 2011(), 2010
PMID: 21253357
Recipes for the linear analysis of EEG.
Parra LC, Spence CD, Gerson AD, Sajda P., Neuroimage 28(2), 2005
PMID: 16084117
A supramodal accumulation-to-bound signal that determines perceptual decisions in humans.
O'Connell RG, Dockree PM, Kelly SP., Nat. Neurosci. 15(12), 2012
PMID: 23103963
Nonparametric statistical testing of EEG- and MEG-data.
Maris E, Oostenveld R., J. Neurosci. Methods 164(1), 2007
PMID: 17517438
Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates.
Eklund A, Nichols TE, Knutsson H., Proc. Natl. Acad. Sci. U.S.A. 113(28), 2016
PMID: 27357684
Cluster-based computational methods for mass univariate analyses of event-related brain potentials/fields: A simulation study.
Pernet CR, Latinus M, Nichols TE, Rousselet GA., J. Neurosci. Methods 250(), 2014
PMID: 25128255

Y, The Annals of Statistics 29(), 2001
Making sense of all the conflict: a theoretical review and critique of conflict-related ERPs.
Larson MJ, Clayson PE, Clawson A., Int J Psychophysiol 93(3), 2014
PMID: 24950132
The electrophysiological dynamics of interference during the Stroop task.
Hanslmayr S, Pastotter B, Bauml KH, Gruber S, Wimber M, Klimesch W., J Cogn Neurosci 20(2), 2008
PMID: 18275330
Sounds reset rhythms of visual cortex and corresponding human visual perception.
Romei V, Gross J, Thut G., Curr. Biol. 22(9), 2012
PMID: 22503499
Recalibration of audiovisual simultaneity.
Fujisaki W, Shimojo S, Kashino M, Nishida S., Nat. Neurosci. 7(7), 2004
PMID: 15195098
Event Related Potentials Index Rapid Recalibration to Audiovisual Temporal Asynchrony.
Simon DM, Noel JP, Wallace MT., Front Integr Neurosci 11(), 2017
PMID: 28381993

E, Audiovisual temporal recalibration occurs independently at two different time scales 5(), 2015
Can post-error dynamics explain sequential reaction time patterns?
Goldfarb S, Wong-Lin K, Schwemmer M, Leonard NE, Holmes P., Front Psychol 3(), 2012
PMID: 22811671
Sequential effects in two-choice reaction time tasks: decomposition and synthesis of mechanisms.
Gao J, Wong-Lin K, Holmes P, Simen P, Cohen JD., Neural Comput 21(9), 2009
PMID: 19548803
Item-specific adaptation and the conflict-monitoring hypothesis: a computational model.
Blais C, Robidoux S, Risko EF, Besner D., Psychol Rev 114(4), 2007
PMID: 17907873
Specific activation of the V5 brain area by auditory motion processing: an fMRI study.
Poirier C, Collignon O, Devolder AG, Renier L, Vanlierde A, Tranduy D, Scheiber C., Brain Res Cogn Brain Res 25(3), 2005
PMID: 16298112
Tactile and visual motion direction processing in hMT+/V5.
van Kemenade BM, Seymour K, Wacker E, Spitzer B, Blankenburg F, Sterzer P., Neuroimage 84(), 2013
PMID: 24036354
Decoding sound and imagery content in early visual cortex.
Vetter P, Smith FW, Muckli L., Curr. Biol. 24(11), 2014
PMID: 24856208
The classic P300 encodes a build-to-threshold decision variable.
Twomey DM, Murphy PR, Kelly SP, O'Connell RG., Eur. J. Neurosci. 42(1), 2015
PMID: 25925534
Conflict monitoring and anterior cingulate cortex: an update.
Botvinick MM, Cohen JD, Carter CS., Trends Cogn. Sci. (Regul. Ed.) 8(12), 2004
PMID: 15556023
Anterior cingulate and prefrontal cortex: who's in control?
Cohen JD, Botvinick M, Carter CS., Nat. Neurosci. 3(5), 2000
PMID: 10769376
Conflict monitoring versus selection-for-action in anterior cingulate cortex.
Botvinick M, Nystrom LE, Fissell K, Carter CS, Cohen JD., Nature 402(6758), 1999
PMID: 10647008
Neural correlates of motor-sensory temporal recalibration.
Stekelenburg JJ, Sugano Y, Vroomen J., Brain Res. 1397(), 2011
PMID: 21600564
The multifaceted interplay between attention and multisensory integration.
Talsma D, Senkowski D, Soto-Faraco S, Woldorff MG., Trends Cogn. Sci. (Regul. Ed.) 14(9), 2010
PMID: 20675182
Crossmodal attention.
Driver J, Spence C., Curr. Opin. Neurobiol. 8(2), 1998
PMID: 9635209
Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain.
Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M., Neuroimage 15(1), 2002
PMID: 11771995

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®

PMID: 29487374
PubMed | Europe PMC

Suchen in

Google Scholar