Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning

Voß H, Wersing H, Kopp S (2021)
In: Companion Publication of the 2021 International Conference on Multimodal Interaction. Hammal Z (Ed); New York, NY: Association for Computing Machinery : 317-323.

Konferenzbeitrag | Veröffentlicht | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Autor*in
Herausgeber*in
Hammal, Zakia
Abstract / Bemerkung
Detecting mental states of human users is crucial for the development of cooperative and intelligent robots, as it enables the robot to understand the user's intentions and desires. Despite their importance, it is difficult to obtain a large amount of high quality data for training automatic recognition algorithms as the time and effort required to collect and label such data is prohibitively high. In this paper we present a multimodal machine learning approach for detecting dis-/agreement and confusion states in a human-robot interaction environment, using just a small amount of manually annotated data. We collect a data set by conducting a human-robot interaction study and develop a novel preprocessing pipeline for our machine learning approach. By combining semi-supervised and supervised architectures, we are able to achieve an average F1-score of 81.1\% for dis-/agreement detection with a small amount of labeled data and a large unlabeled data set, while simultaneously increasing the robustness of the model compared to the supervised approach.
Erscheinungsjahr
2021
Titel des Konferenzbandes
Companion Publication of the 2021 International Conference on Multimodal Interaction
Seite(n)
317-323
eISBN
978-1-4503-8471-1
Page URI
https://pub.uni-bielefeld.de/record/2957396

Zitieren

Voß H, Wersing H, Kopp S. Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning. In: Hammal Z, ed. Companion Publication of the 2021 International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery ; 2021: 317-323.
Voß, H., Wersing, H., & Kopp, S. (2021). Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning. In Z. Hammal (Ed.), Companion Publication of the 2021 International Conference on Multimodal Interaction (pp. 317-323). New York, NY: Association for Computing Machinery . https://doi.org/10.1145/3461615.3486575
Voß, H., Wersing, H., and Kopp, S. (2021). “Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning” in Companion Publication of the 2021 International Conference on Multimodal Interaction, Hammal, Z. ed. (New York, NY: Association for Computing Machinery ), 317-323.
Voß, H., Wersing, H., & Kopp, S., 2021. Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning. In Z. Hammal, ed. Companion Publication of the 2021 International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery , pp. 317-323.
H. Voß, H. Wersing, and S. Kopp, “Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning”, Companion Publication of the 2021 International Conference on Multimodal Interaction, Z. Hammal, ed., New York, NY: Association for Computing Machinery , 2021, pp.317-323.
Voß, H., Wersing, H., Kopp, S.: Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning. In: Hammal, Z. (ed.) Companion Publication of the 2021 International Conference on Multimodal Interaction. p. 317-323. Association for Computing Machinery , New York, NY (2021).
Voß, Hendric, Wersing, Heiko, and Kopp, Stefan. “Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning”. Companion Publication of the 2021 International Conference on Multimodal Interaction. Ed. Zakia Hammal. New York, NY: Association for Computing Machinery , 2021. 317-323.

Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Suchen in

Google Scholar