PUB - Publikationen an der Universität Bielefeld

Understanding “understanding”: Presenting a richly annotated multimodal corpus of dyadic interaction Leonie Schade, Nico Dallmann, Olcay T ̈urk, Petra Wagner Bielefeld University, Germany leonie.schade@uni-bielefeld.de Abstract This paper presents the MUNDEX corpus (MUltimodal UNDerstanding of EXplanations) together with past and current investigations using its data. The corpus is constructed to observe the dynamics of co-constructed communication and the understanding of explanations on multiple modalities in dyadic interactions. These modalities are annotated on several levels, including orthographic transcriptions, acoustic information, annotations of head movement, gaze, manual gestures, and further non-verbal behaviour as well as discourse annotations. Present and past projects are also concerned with adding further to these annotations. The interlocutors’ level of understanding is currently investigated in regard to several verbal and non-verbal behaviour markers of both the explaining and listening side.