Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification

Knoeferle P, Urbach TP, Kutas M (2011)
Psychophysiology 48(4): 495-506.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
DOI
Autor*in
Knoeferle, PiaUniBi; Urbach, Thomas P.; Kutas, Marta
Einrichtung
Fakultät für Linguistik und Literaturwissenschaft
Center of Excellence - Cognitive Interaction Technology CITEC
Abstract / Bemerkung
To re-establish picture-sentence verification-discredited possibly for its over-reliance on post-sentence response time (RT) measures-as a task for situated comprehension, we collected event-related brain potentials (ERPs) as participants read a subject-verb-object sentence, and RTs indicating whether or not the verb matched a previously depicted action. For mismatches (vs. matches), speeded RTs were longer, verb N400s over centro-parietal scalp larger, and ERPs to the object noun more negative. RTs (congruence effect) correlated inversely with the centro-parietal verb N400s, and positively with the object ERP congruence effects. Verb N400s, object ERPs, and verbal working memory scores predicted more variance in RT effects (50%) than N400s alone. Thus, (1) verification processing is not all post-sentence; (2) simple priming cannot account for these results; and (3) verification tasks can inform studies of situated comprehension.
Stichworte
Picture-sentence verification; Situated comprehension; Event-related brain potentials
Erscheinungsjahr
2011
Zeitschriftentitel
Psychophysiology
Band
48
Ausgabe
4
Seite(n)
495-506
ISSN
0048-5772
Page URI
https://pub.uni-bielefeld.de/record/1941460

Zitieren

Knoeferle P, Urbach TP, Kutas M. Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification. Psychophysiology. 2011;48(4):495-506.
Knoeferle, P., Urbach, T. P., & Kutas, M. (2011). Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification. Psychophysiology, 48(4), 495-506. https://doi.org/10.1111/j.1469-8986.2010.01080.x
Knoeferle, Pia, Urbach, Thomas P., and Kutas, Marta. 2011. “Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification”. Psychophysiology 48 (4): 495-506.
Knoeferle, P., Urbach, T. P., and Kutas, M. (2011). Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification. Psychophysiology 48, 495-506.
Knoeferle, P., Urbach, T.P., & Kutas, M., 2011. Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification. Psychophysiology, 48(4), p 495-506.
P. Knoeferle, T.P. Urbach, and M. Kutas, “Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification”, Psychophysiology, vol. 48, 2011, pp. 495-506.
Knoeferle, P., Urbach, T.P., Kutas, M.: Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification. Psychophysiology. 48, 495-506 (2011).
Knoeferle, Pia, Urbach, Thomas P., and Kutas, Marta. “Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification”. Psychophysiology 48.4 (2011): 495-506.

15 Zitationen in Europe PMC

Daten bereitgestellt von Europe PubMed Central.

Semantic Interference and Facilitation: Understanding the Integration of Spatial Distance and Conceptual Similarity During Sentence Reading.
Guerra E, Knoeferle P., Front Psychol 9(), 2018
PMID: 29896138
Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism.
Zhou P, Ma W, Zhan L, Ma H., J Vis Exp (140), 2018
PMID: 30346396
Disentangling stimulus plausibility and contextual congruency: Electro-physiological evidence for differential cognitive dynamics.
Coco MI, Araujo S, Petersson KM., Neuropsychologia 96(), 2017
PMID: 27965026
Validating presupposed versus focused text information.
Singer M, Solar KG, Spear J., Mem Cognit 45(3), 2017
PMID: 27913966
Cortical networks for reference-frame processing are shared by language and spatial navigation systems.
Vukovic N, Shtyrov Y., Neuroimage 161(), 2017
PMID: 28818696
Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events.
Kuperberg GR., Lang Cogn Neurosci 31(5), 2016
PMID: 27570786
Gestalt Reasoning with Conjunctions and Disjunctions.
Dumitru ML, Joergensen GH., PLoS One 11(3), 2016
PMID: 26986760
Visual attention during spatial language comprehension.
Burigo M, Knoeferle P., PLoS One 10(1), 2015
PMID: 25607540
Spatial distance effects on incremental semantic interpretation of abstract sentences: evidence from eye tracking.
Guerra E, Knoeferle P., Cognition 133(3), 2014
PMID: 25215930
Distinct neural correlates for pragmatic and semantic meaning processing: an event-related potential investigation of scalar implicature processing using picture-sentence verification.
Politzer-Ahles S, Fiorentino R, Jiang X, Zhou X., Brain Res 1490(), 2013
PMID: 23103410
Perception and bias in the processing of compound versus phrasal stress: evidence from event-related brain potentials.
McCauley SM, Hestvik A, Vogel I., Lang Speech 56(pt 1), 2013
PMID: 23654115
Throwing the banana away and keeping the peel: neuroelectric responses to unexpected but physically feasible action endings.
Giglio AC, Minati L, Boggio PS., Brain Res 1532(), 2013
PMID: 23948101
Effects of speaker emotional facial expression and listener age on incremental sentence processing.
Carminati MN, Knoeferle P., PLoS One 8(9), 2013
PMID: 24039781
Shared syntax in language production and language comprehension--an FMRI study.
Segaert K, Menenti L, Weber K, Petersson KM, Hagoort P., Cereb Cortex 22(7), 2012
PMID: 21934094
Shared language: overlap and segregation of the neuronal infrastructure for speaking and listening revealed by functional MRI.
Menenti L, Gierhan SM, Segaert K, Hagoort P., Psychol Sci 22(9), 2011
PMID: 21841148

48 References

Daten bereitgestellt von Europe PubMed Central.

Language-mediated eye movements in the absence of a visual world: the 'blank screen paradigm'.
Altmann GT., Cognition 93(2), 2004
PMID: 15147941
Event-related potentials and the semantic matching of pictures.
Barrett SE, Rugg MD., Brain Cogn 14(2), 1990
PMID: 2285513
Sentence comprehension: A psycholinguistic model of verification
Carpenter PA, Just MA., 1975
Models of sentence verification and linguistic comprehension
Carpenter PA, Just MA., 1976
Actions and affordances in syntactic ambiguity resolution
Chambers CG, Tanenhaus MK, Magnuson JS., 2004
On the process of comparing sentences against pictures
Clark HH, Chase WG., 1972

Cohen J, Cohen P., 1983
Individual differences in working memory and reading
Daneman M, Carpenter PA., 1980
Working memory and language comprehension: A meta-analysis.
Daneman M, Merikle PM., Psychon Bull Rev 3(4), 1996
PMID: 24213976
An event-related brain potential study of receptive speech comprehension using a modified Token Test.
D'Arcy RC, Connolly JF., Neuropsychologia 37(13), 1999
PMID: 10617268
A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing
Federmeier KD, Kutas M., 1999
Picture the difference: electrophysiological investigations of picture processing in the two cerebral hemispheres.
Federmeier KD, Kutas M., Neuropsychologia 40(7), 2002
PMID: 11900725
Electrophysiological evidence for the time-course of verifying text ideas.
Ferretti TR, Singer M, Patterson C., Cognition 108(3), 2008
PMID: 18675957
Brain potentials related to stages of sentence verification.
Fischler I, Bloom PA, Childers DG, Roucos SE, Perry NW Jr., Psychophysiology 20(4), 1983
PMID: 6356204
Picture-word differences in a sentence verification task.
Goolkasian P., Mem Cognit 24(5), 1996
PMID: 8870529
Grammatical transformations and speed of understanding
Gough PB., 1965
Difference score correlations in relationship research: A conceptual primer
Griffin D, Murray S, Gonzalez R., 1999
Natural speech processing: An analysis using event-related brain potentials
Holcomb PJ, Neville HJ., 1991

Hutcheson G, Sofroniou N., 1999
Difference score measures of organizational behavior variables: A critique
Johns G., 1981
A capacity theory of comprehension: individual differences in working memory.
Just MA, Carpenter PA., Psychol Rev 99(1), 1992
PMID: 1546114
The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events.
Knoeferle P, Crocker MW, Scheepers C, Pickering MJ., Cognition 95(1), 2005
PMID: 15629475
The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking.
Knoeferle P, Crocker MW., Cogn Sci 30(3), 2006
PMID: 21702823
The influence of recent scene events on spoken comprehension: evidence from eye movements
Knoeferle P, Crocker MW., 2007
Visual scenes trigger immediate syntactic reanalysis: evidence from ERPs during situated spoken comprehension.
Knoeferle P, Habets B, Crocker MW, Munte TF., Cereb. Cortex 18(4), 2007
PMID: 17644830
Structure and process in semantic memory: evidence from event-related brain potentials and reaction times.
Kounios J, Holcomb PJ., J Exp Psychol Gen 121(4), 1992
PMID: 1431739
Reading senseless sentences: brain potentials reflect semantic incongruity.
Kutas M, Hillyard SA., Science 207(4427), 1980
PMID: 7350657
Brain potentials during reading reflect word expectancy and semantic association.
Kutas M, Hillyard SA., Nature 307(5947), 1984
PMID: 6690995
Event-related brain potential studies of language
Kutas M, Van C., 1988
Psycholinguistics Electrified II
Kutas M, Van CK, Kluender R., 2006
Learning to attend: a connectionist model of situated language comprehension.
Mayberry MR, Crocker MW, Knoeferle P., Cogn Sci 33(3), 2009
PMID: 21585477
The effects of physical and semantic incongruities on auditory event-related potentials
McCallum W, Farmer SF, Pocock PV., 1984
Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations.
Meyer DE, Schvaneveldt RW., J Exp Psychol 90(2), 1971
PMID: 5134329
What makes a discourse constraining? Comparing the effects of discourse message and scenario fit on the discourse-dependent N400 effect.
Otten M, Van Berkum JJ., Brain Res. 1153(), 2007
PMID: 17466281
The neural bases of strategy and skill in sentence-picture verification.
Reichle ED, Carpenter PA, Just MA., Cogn Psychol 40(4), 2000
PMID: 10888341
Achieving incremental semantic interpretation through contextual representation.
Sedivy JC, Tanenhaus MK, Chambers CG, Carlson GN., Cognition 71(2), 1999
PMID: 10444906
Verification of text ideas during reading
Singer M., 2006
Eye movements and spoken language comprehension: effects of visual context on syntactic ambiguity resolution.
Spivey MJ, Tanenhaus MK, Eberhard KM, Sedivy JC., Cogn Psychol 45(4), 2002
PMID: 12480476
Studies of interference in serial verbal reactions
Stroop JR., 1935
Electroencephalography effects to semantic and non-semantic mismatch in properties of visually presented single-characters: the N2b and the N400.
Szucs D, Soltesz F, Czigler I, Csepe V., Neurosci. Lett. 412(1), 2006
PMID: 17141414

Tabachnik B, Fidell LS., 2007
Sentence-picture verification models as theories of sentence comprehension: A critique of Carpenter and Just
Tanenhaus MK, Carroll JM, Bever TG., 1976
Integration of visual and linguistic information in spoken language comprehension.
Tanenhaus MK, Spivey-Knowlton MJ, Eberhard KM, Sedivy JC., Science 268(5217), 1995
PMID: 7777863
Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search.
Underwood G, Jebbett L, Roberts K., Q J Exp Psychol A 57(1), 2004
PMID: 14681008
Semantic integration in sentences and discourse: evidence from the N400.
van Berkum JJ, Hagoort P, Brown CM., J Cogn Neurosci 11(6), 1999
PMID: 10601747
Monitoring in language perception: evidence from ERPs in a picture-sentence matching task.
Vissers CT, Kolk HH, van de Meerendonk N, Chwilla DJ., Neuropsychologia 46(4), 2007
PMID: 18199460
Thematic role assignment in patients with Broca's aphasia: sentence-picture matching electrified.
Wassenaar M, Hagoort P., Neuropsychologia 45(4), 2006
PMID: 17005212
Age, working memory, and on-line syntactic processing in sentence comprehension.
Waters GS, Caplan D., Psychol Aging 16(1), 2001
PMID: 11302362
Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®
Quellen

PMID: 20701712
PubMed | Europe PMC

Suchen in

Google Scholar