Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance

Mitev N, Renner P, Pfeiffer T, Staudte M (2018)
Cognitive Research: Principles and Implications (3): 51.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Autor/in
; ; ;
Abstract / Bemerkung
Referential success is crucial for collaborative task-solving in shared environments. In face-to-face interactions, humans, therefore, exploit speech, gesture, and gaze to identify a specific object. We investigate if and how the gaze behavior of a human interaction partner can be used by a gaze-aware assistance system to improve referential success. Specifically, our system describes objects in the real world to a human listener using on-the-fly speech generation. It continuously interprets listener gaze and implements alternative strategies to react to this implicit feedback. We used this system to investigate an optimal strategy for task performance: providing an unambiguous, longer instruction right from the beginning, or starting with a shorter, yet ambiguous instruction. Further, the system provides gaze-driven feedback, which could be either underspecified (“No, not that one!”) or contrastive (“Further left!”). As expected, our results show that ambiguous instructions followed by underspecified feedback are not beneficial for task performance, whereas contrastive feedback results in faster interactions. Interestingly, this approach even outperforms unambiguous instructions (manipulation between subjects). However, when the system alternates between underspecified and contrastive feedback to initially ambiguous descriptions in an interleaved manner (within subjects), task performance is similar for both approaches. This suggests that listeners engage more intensely with the system when they can expect it to be cooperative. This, rather than the actual informativity of the spoken feedback, may determine the efficiency of information uptake and performance.
Stichworte
Human–computer interaction; Natural language generation; Listener gaze; Referential success; Multimodal systems
Erscheinungsjahr
2018
Zeitschriftentitel
Cognitive Research: Principles and Implications
Ausgabe
3
Art.-Nr.
51
eISSN
2365-7464
Page URI
https://pub.uni-bielefeld.de/record/2932893

Zitieren

Mitev N, Renner P, Pfeiffer T, Staudte M. Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance. Cognitive Research: Principles and Implications. 2018;(3): 51.
Mitev, N., Renner, P., Pfeiffer, T., & Staudte, M. (2018). Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance. Cognitive Research: Principles and Implications(3), 51. doi:10.1186/s41235-018-0148-x
Mitev, N., Renner, P., Pfeiffer, T., and Staudte, M. (2018). Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance. Cognitive Research: Principles and Implications:51.
Mitev, N., et al., 2018. Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance. Cognitive Research: Principles and Implications, (3): 51.
N. Mitev, et al., “Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance”, Cognitive Research: Principles and Implications, 2018, : 51.
Mitev, N., Renner, P., Pfeiffer, T., Staudte, M.: Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance. Cognitive Research: Principles and Implications. : 51 (2018).
Mitev, Nikolina, Renner, Patrick, Pfeiffer, Thies, and Staudte, Maria. “Towards efficient human–machine collaboration. Effects of gaze-driven feedback and engagement on performance”. Cognitive Research: Principles and Implications 3 (2018): 51.
Alle Dateien verfügbar unter der/den folgenden Lizenz(en):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Link(s) zu Volltext(en)
Access Level
OA Open Access

Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®

Quellen

PMID: 30594976
PubMed | Europe PMC

Suchen in

Google Scholar