PUB - Publikationen an der Universität Bielefeld

In this paper we introduce two new methods for real-time sonification of head movements and head gestures. Head gestures such as nodding, shaking the head, etc. are important non-verbal back-channelling signals which facilitate coordination and alignment of communicating interaction partners. Visually impaired persons cannot interpret such non-verbal signals, same as people in mediated communication (e.g. on the phone), or cooperating users whose visual attention is focussed elsewhere. We introduce the idea, the approach, our sensing setup and two different sonification methods. A first preliminary study on the recognition of signals shows that subjects understand gesture type even without prior explanation, and can estimate gesture intensity and frequency with no or little training. #### Sound example S1 continuous excitatory sonification of head gestures of different type, intensity and velocity #### Sound example S2 event-based sonification of head gestures of different type, intensity and velocity #### Video example S3