A model for production, perception, and acquisition of actions in face-to-face communication

Kroeger BJ, Kopp S, Lowit A (2010)
Cognitive Processing 11(3): 187-205.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
DOI
Autor*in
Kroeger, Bernd J.; Kopp, StefanUniBi ; Lowit, Anja
Einrichtung
Technische Fakultät > AG Kognitive Systeme und soziale Interaktion
Technische Fakultät > AG Wissensbasierte Systeme
Center of Excellence - Cognitive Interaction Technology CITEC
Research Institute for Cognition and Robotics
Abstract / Bemerkung
The concept of action as basic motor control unit for goal-directed movement behavior has been used primarily for private or non-communicative actions like walking, reaching, or grasping. In this paper, literature is reviewed indicating that this concept can also be used in all domains of face-to-face communication like speech, co-verbal facial expression, and co-verbal gesturing. Three domain-specific types of actions, i.e. speech actions, facial actions, and hand-arm actions, are defined in this paper and a model is proposed that elucidates the underlying biological mechanisms of action production, action perception, and action acquisition in all domains of face-to-face communication. This model can be used as theoretical framework for empirical analysis or simulation with embodied conversational agents, and thus for advanced human-computer interaction technologies.
Stichworte
Co-verbal behavior; Speech; Face-to-face communication; expression; Action; Facial
Erscheinungsjahr
2010
Zeitschriftentitel
Cognitive Processing
Band
11
Ausgabe
3
Seite(n)
187-205
ISSN
1612-4782
eISSN
1612-4790
Page URI
https://pub.uni-bielefeld.de/record/1794428

Zitieren

Kroeger BJ, Kopp S, Lowit A. A model for production, perception, and acquisition of actions in face-to-face communication. Cognitive Processing. 2010;11(3):187-205.
Kroeger, B. J., Kopp, S., & Lowit, A. (2010). A model for production, perception, and acquisition of actions in face-to-face communication. Cognitive Processing, 11(3), 187-205. https://doi.org/10.1007/s10339-009-0351-2
Kroeger, Bernd J., Kopp, Stefan, and Lowit, Anja. 2010. “A model for production, perception, and acquisition of actions in face-to-face communication”. Cognitive Processing 11 (3): 187-205.
Kroeger, B. J., Kopp, S., and Lowit, A. (2010). A model for production, perception, and acquisition of actions in face-to-face communication. Cognitive Processing 11, 187-205.
Kroeger, B.J., Kopp, S., & Lowit, A., 2010. A model for production, perception, and acquisition of actions in face-to-face communication. Cognitive Processing, 11(3), p 187-205.
B.J. Kroeger, S. Kopp, and A. Lowit, “A model for production, perception, and acquisition of actions in face-to-face communication”, Cognitive Processing, vol. 11, 2010, pp. 187-205.
Kroeger, B.J., Kopp, S., Lowit, A.: A model for production, perception, and acquisition of actions in face-to-face communication. Cognitive Processing. 11, 187-205 (2010).
Kroeger, Bernd J., Kopp, Stefan, and Lowit, Anja. “A model for production, perception, and acquisition of actions in face-to-face communication”. Cognitive Processing 11.3 (2010): 187-205.

6 Zitationen in Europe PMC

Daten bereitgestellt von Europe PubMed Central.

You cannot speak and listen at the same time: a probabilistic model of turn-taking.
Donnarumma F, Dindo H, Iodice P, Pezzulo G., Biol Cybern 111(2), 2017
PMID: 28265753
Modeling Interactions between Speech Production and Perception: Speech Error Detection at Semantic and Phonological Levels and the Inner Speech Loop.
Kröger BJ, Crawford E, Bekolay T, Eliasmith C., Front Comput Neurosci 10(), 2016
PMID: 27303287
Crosslinguistic application of English-centric rhythm descriptors in motor speech disorders.
Liss JM, Utianski R, Lansford K., Folia Phoniatr Logop 65(1), 2013
PMID: 24157596
Human sensorimotor communication: a theory of signaling in online social interactions.
Pezzulo G, Donnarumma F, Dindo H., PLoS One 8(11), 2013
PMID: 24278201
Social signals: a framework in terms of goals and beliefs.
Poggi I, D'Errico F., Cogn Process 13 Suppl 2(), 2012
PMID: 22797857
Embodied Gesture Processing: Motor-Based Integration of Perception and Action in Social Artificial Agents.
Sadeghipour A, Kopp S., Cognit Comput 3(3), 2011
PMID: 21957433

131 References

Daten bereitgestellt von Europe PubMed Central.


AUTHOR UNKNOWN, 0

MW, J Memory Lang 44(), 2001

J, 1976
Deciphering the enigmatic face: the importance of facial dynamics in interpreting subtle facial expressions.
Ambadar Z, Schooler JW, Cohn JF., Psychol Sci 16(5), 2005
PMID: 15869701
Synthetic brain imaging: grasping, mirror neurons and imitation.
Arbib MA, Billard A, Iacoboni M, Oztop E., Neural Netw 13(8-9), 2000
PMID: 11156205

G, Speech Commun 22(), 1997
Facial motion in the perception of faces and of emotional expression.
Bassili JN., J Exp Psychol Hum Percept Perform 4(3), 1978
PMID: 681886

AUTHOR UNKNOWN, 0

N, 1967
From the perception of action to the understanding of intention.
Blakemore SJ, Decety J., Nat. Rev. Neurosci. 2(8), 2001
PMID: 11483999
Investigating action understanding: inferential processes versus action simulation.
Brass M, Schmitt RM, Spengler S, Gergely G., Curr. Biol. 17(24), 2007
PMID: 18083518

C, Phonology 6(), 1989
Articulatory phonology: an overview.
Browman CP, Goldstein L., Phonetica 49(3-4), 1992
PMID: 1488456
Modeling sensorimotor learning with linear dynamical systems.
Cheng S, Sabes PN., Neural Comput 18(4), 2006
PMID: 16494690

JF, 2007

JF, 2007

F, J Acoust Soc Am 24(), 1952
The selection of intended actions and the observation of others' actions: a time-resolved fMRI study.
Cunnington R, Windischberger C, Robinson S, Moser E., Neuroimage 29(4), 2005
PMID: 16246592
Construction and control of a physiological articulatory model.
Dang J, Honda K., J. Acoust. Soc. Am. 115(2), 2004
PMID: 15000197

AUTHOR UNKNOWN, 0

AUTHOR UNKNOWN, 0

AUTHOR UNKNOWN, 0

AUTHOR UNKNOWN, 0
Forward modeling allows feedback control for fast reaching movements.
Desmurget M, Grafton S., Trends Cogn. Sci. (Regul. Ed.) 4(11), 2000
PMID: 11058820
Speech perception.
Diehl RL, Lotto AJ, Holt LL., Annu Rev Psychol 55(), 2004
PMID: 14744213

P, Env Psychol Nonverbal Behav 1(), 1976

P, 1978

P, Annu Rev Psychol 30(), 1979
Electrophysiology of action representation.
Fadiga L, Craighero L., J Clin Neurophysiol 21(3), 2004
PMID: 15375346
Once more on the equilibrium-point hypothesis (lambda model) for motor control.
Feldman AG., J Mot Behav 18(1), 1986
PMID: 15136283

TM, 1984
The coordination of arm movements: an experimentally confirmed mathematical model.
Flash T, Hogan N., J. Neurosci. 5(7), 1985
PMID: 4020415
Immediate compensation in bite-block speech.
Fowler CA, Turvey MT., Phonetica 37(5-6), 1981
PMID: 7280033

V, J Conscious Stud 7(), 2000
Audio-visual enhancement of speech in noise.
Girin L, Schwartz JL, Feng G., J. Acoust. Soc. Am. 109(6), 2001
PMID: 11425143

L, 2006
Dynamic action units slip in speech production errors.
Goldstein L, Pouplier M, Chen L, Saltzman E, Byrd D., Cognition 103(3), 2006
PMID: 16822494
Evidence for a distributed hierarchy of action representation in the brain.
Grafton ST, Hamilton AF., Hum Mov Sci 26(4), 2007
PMID: 17706312
Fitts's law holds for action perception.
Grosjean M, Shiffrar M, Knoblich G., Psychol Sci 18(2), 2007
PMID: 17425525
Cortical interactions underlying the production of speech sounds.
Guenther FH., J Commun Disord 39(5), 2006
PMID: 16887139
A theoretical investigation of reference frames for the planning of speech movements.
Guenther FH, Hampson M, Johnson D., Psychol Rev 105(4), 1998
PMID: 9830375
Neural modeling and imaging of the cortical interactions underlying syllable production.
Guenther FH, Ghosh SS, Tourville JA., Brain Lang 96(3), 2005
PMID: 16040108

M, First Language 28(), 2008

G, Trends Cogn Sci 4(), 2007
An organizing principle for a class of voluntary movements.
Hogan N., J. Neurosci. 4(11), 1984
PMID: 6502203
Sensorimotor adaptation of speech I: Compensation and adaptation.
Houde JF, Jordan MI., J. Speech Lang. Hear. Res. 45(2), 2002
PMID: 12003512
Neural mechanisms of imitation.
Iacoboni M., Curr. Opin. Neurobiol. 15(6), 2005
PMID: 16271461
The spatial and temporal signatures of word production components.
Indefrey P, Levelt WJ., Cognition 92(1-2), 2004
PMID: 15037128
Dynamical simulation of speech cooperative articulation by muscle linkages.
Ito T, Gomi H, Honda M., Biol Cybern 91(5), 2004
PMID: 15449066
Willed action and its impairments.
Jahanshahi M., Cogn Neuropsychol 15(6-8), 1998
PMID: 22448836
Learning to discriminate complex movements: biological versus artificial trajectories.
Jastorff J, Kourtzi Z, Giese MA., J Vis 6(8), 2006
PMID: 16895459
Neural simulation of action: a unifying mechanism for motor cognition.
Jeannerod M., Neuroimage 14(1 Pt 2), 2001
PMID: 11373140

M, Q J Exp Psychol 52A(), 1999

MI, 1995
Internal models for motor control and trajectory planning.
Kawato M., Curr. Opin. Neurobiol. 9(6), 1999
PMID: 10607637
Trajectory formation of arm movement by cascade neural network model based on minimum torque-change criterion.
Kawato M, Maeda Y, Uno Y, Suzuki R., Biol Cybern 62(4), 1990
PMID: 2310782
Functionally specific articulatory cooperation following jaw perturbations during speech: evidence for coordinative structures.
Kelso JA, Tuller B, Vatikiotis-Bateson E, Fowler CA., J Exp Psychol Hum Percept Perform 10(6), 1984
PMID: 6239907

JAS, J Phon 14(), 1986

A, 2004
Hearing sounds, understanding actions: action representation in mirror neurons.
Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G., Science 297(5582), 2002
PMID: 12161656

AUTHOR UNKNOWN, 0

S, J Comput Anim Virtual Worlds 15(), 2004

S, 2006

S, 2007

S, 2008
A gestural production model and its application to reduction in German.
Kroger BJ., Phonetica 50(4), 1993
PMID: 8295928

BJ, 2007

BJ, J Acoust Soc Am 98(), 1995

BJ, Speech Commun 51(), 2009

BJ, 2009

K, J Acoust Soc Am 73(), 1984
Evolution of Motor Control: From Reflexes and Motor Programs to the Equilibrium-Point Hypothesis.
Latash ML., J Hum Kinet 19(19), 2008
PMID: 19823595

ML, Kinesiology 40(), 2008
Neural substrates for action understanding at different description levels in the human brain.
Lestou V, Pollick FE, Kourtzi Z., J Cogn Neurosci 20(2), 2008
PMID: 18275338

WJM, J Memory Lang 24(), 1985
A theory of lexical access in speech production.
Levelt WJ, Roelofs A, Meyer AS., Behav Brain Sci 22(1), 1999
PMID: 11301520
The motor theory of speech perception revised.
Liberman AM, Mattingly IG., Cognition 21(1), 1985
PMID: 4075760

B, J Acoust Soc Am 35(), 1963

B, 1983

D, 1992
Imitation of facial and manual gestures by human neonates.
Meltzoff AN, Moore MK., Science 198(4312), 1977
PMID: 17741897
Imitation in Newborn Infants: Exploring the Range of Gestures Imitated and the Underlying Mechanisms.
Meltzoff AN, Moore MK., Dev Psychol 25(6), 1989
PMID: 25147405
Somatosensory precision in speech production.
Nasir SM, Ostry DJ., Curr. Biol. 16(19), 2006
PMID: 17027488
Speech motor learning in profoundly deaf adults.
Nasir SM, Ostry DJ., Nat. Neurosci. 11(10), 2008
PMID: 18794839

T, J Acoust Soc Am 80(), 1986

AT, Acoust Res Lett Online 5(), 2004
Physical principles for economies of skilled movements.
Nelson WL., Biol Cybern 46(2), 1983
PMID: 6838914
The role of the cerebellum for predictive control of grasping.
Nowak DA, Topka H, Timmann D, Boecker H, Hermsdorfer J., Cerebellum 6(1), 2007
PMID: 17366262

M, Image Vis Comput 18(), 2000

Y, Speech Commun 22(), 1997

J, Speech Commun 22(), 1997

P, ZAS Pap Linguist 40(), 2005

AUTHOR UNKNOWN, 0
The equilibrium point hypothesis and its application to speech motor control.
Perrier P, Ostry DJ, Laboissiere R., J Speech Hear Res 39(2), 1996
PMID: 8729923
Influences of tongue biomechanics on speech movements during the production of velar stop consonants: a modeling study.
Perrier P, Payan Y, Zandipour M, Perkell J., J. Acoust. Soc. Am. 114(3), 2003
PMID: 14514212
Perception of American sign language in dynamic point-light displays.
Poizner H, Bellugi U, Lutes-Driscoll V., J Exp Psychol Hum Percept Perform 7(2), 1981
PMID: 6453935
Adaptive control of vowel formant frequency: evidence from real-time formant manipulation.
Purcell DW, Munhall KG., J. Acoust. Soc. Am. 120(2), 2006
PMID: 16938984
Muscle recruitment by the min/max criterion -- a comparative numerical study.
Rasmussen J, Damsgaard M, Voigt M., J Biomech 34(3), 2001
PMID: 11182135
The mirror-neuron system.
Rizzolatti G, Craighero L., Annu. Rev. Neurosci. 27(), 2004
PMID: 15217330
The speech focus position effect on jaw-finger coordination in a pointing task.
Rochet-Capellan A, Laboissiere R, Galvan A, Schwartz JL., J. Speech Lang. Hear. Res. 51(6), 2008
PMID: 18695015

MJ, First Lang 24(), 2004
Point-light facial displays enhance comprehension of speech in noise.
Rosenblum LD, Johnson JA, Saldana HM., J Speech Hear Res 39(6), 1996
PMID: 8959601
The planning and control of reaching movements.
Sabes PN., Curr. Opin. Neurobiol. 10(6), 2000
PMID: 11240283
Obstacle avoidance and a perturbation sensitivity model for motor planning.
Sabes PN, Jordan MI., J. Neurosci. 17(18), 1997
PMID: 9278546

AUTHOR UNKNOWN, 0

E, J Math Psychol 20(), 1979

E, Hum Mov Sci 19(), 2000
Skilled actions: a task-dynamic approach.
Saltzman E, Kelso JA., Psychol Rev 94(1), 1987
PMID: 3823306

E, Ecol Psychol 1(), 1989
Is imitation learning the route to humanoid robots?
Schaal S., Trends Cogn. Sci. (Regul. Ed.) 3(6), 1999
PMID: 10354577

KL, Am J Phys Anthropol 116(S33), 2002
Signal characteristics of spontaneous facial expressions: automatic movement in solitary and social smiles.
Schmidt KL, Cohn JF, Tian Y., Biol Psychol 65(1), 2003
PMID: 14638288
MOVEMENT DIFFERENCES BETWEEN DELIBERATE AND SPONTANEOUS FACIAL EXPRESSIONS: ZYGOMATICUS MAJOR ACTION IN SMILING.
Schmidt KL, Ambadar Z, Cohn JF, Reed LI., J Nonverbal Behav 30(1), 2006
PMID: 19367343
Comparison of Deliberate and Spontaneous Facial Movement in Smiles and Eyebrow Raises.
Schmidt KL, Bhattacharya S, Denlinger R., J Nonverbal Behav 33(1), 2009
PMID: 20333273

JP, Exp Brain Res 80(), 2007

JL, 2007
Adaptive representation of dynamics during learning of a motor task.
Shadmehr R, Mussa-Ivaldi FA., J. Neurosci. 14(5 Pt 2), 1994
PMID: 8182467

JB, Mot Control 3(), 1999
Multisensory integration during motor planning.
Sober SJ, Sabes PN., J. Neurosci. 23(18), 2003
PMID: 12904459
Flexible strategies for sensory integration during motor planning.
Sober SJ, Sabes PN., Nat. Neurosci. 8(4), 2005
PMID: 15793578

L, Connect Sci 20(), 2008
Dynamic specification of coarticulated vowels.
Strange W, Jenkins JJ, Johnson TL., J. Acoust. Soc. Am. 74(3), 1983
PMID: 6630725

Q, 1987

YL, 2005
Optimality principles in sensorimotor control.
Todorov E., Nat. Neurosci. 7(9), 2004
PMID: 15332089

AUTHOR UNKNOWN, 0
Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements.
Todorov E, Jordan MI., J. Neurophysiol. 80(2), 1998
PMID: 9705462
A new look at infant pointing.
Tomasello M, Carpenter M, Liszkowski U., Child Dev 78(3), 2007
PMID: 17516997

MT, 1977
Motor prediction.
Wolpert DM, Flanagan JR., Curr. Biol. 11(18), 2001
PMID: 11566114
Perspectives and problems in motor learning.
Wolpert DM, Ghahramani Z, Flanagan JR., Trends Cogn. Sci. (Regul. Ed.) 5(11), 2001
PMID: 11684481
Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®
Quellen

PMID: 20012153
PubMed | Europe PMC

Suchen in

Google Scholar