PUB - Publikationen an der Universität Bielefeld

Our research aims at cognitive modelling of joint attention for artificial agents, such as virtual agents or robots. With the current study, we are focusing on the observation of interaction patterns of joint attention and their time course. For this, we recorded and analyzed twenty sessions of two interacting participants using two mobile binocular eye-tracking systems. A key contribution of our work addresses methodological aspects of mobile eye-tracking studies with scene camera recordings in general. The standard procedure for the analysis of such gaze videos requires a manual annotation. This time consuming process often exceeds multiple times the duration of the original recordings (e.g. 30 times). This doubles if, as in our case, the gaze of two interlocutors is recorded simultaneously. In our approach, we build upon our EyeSee3D approach for marker-based tracking and registration of the environment and a 3D reconstruction of the relevant stimuli. We extend upon the previous approach in supporting more than one participant and dynamically changing stimuli, here the faces and eyes of the interlocutors. The full analysis of the time course of both interlocutor’s gaze is done in real-time and available for analysis right after the experiment without the requirement for manual annotation.