PUB - Publikationen an der Universität Bielefeld

This paper presents an interactive audio-haptic human-computer interface for controlling an atomic force microscope (AFM) in force spectroscopy experiments on single molecules. The sensor data used are proportional to the force that is applied to a single molecule. These forces are measured in real-time by using the reflection of a laser beam from a cantilever. We present a system that involves (a) a visual display of the data (b) a force-feedback joystick for navigating the sample and providing a tactile feedback of the forces, and (c) an auditory display to monitor the measured data while interactively moving the sample. The sonification we have developed integrates information at various levels of detail, including audifications of the high-frequency cantilever movement, and an auditory stream that communicates the instantaneous deviation of forces between the approach and retract phase of the sample. The sonification design and offline-computed sonifications will be presented and discussed. We further report on our first experiences with this interactive multi-modal control interface for manipulation of individual DNA molecules. ### Audifications of DNA measurements: + (a) data files where DNA actually binds to the cantilever
[S_a1 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703102) [S_a2 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703103) [S_a3 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703104) [S_a4 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703105)
[S_a5 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703107) [S_a6 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703106) [S_a7 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703108) [S_a8 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703109) + (b) data files where nothing happens (no binding)
[S_b1 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703110) [S_b2 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703111) [S_b3 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703114) [S_b4 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703112)
[S_b5 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703113) [S_b6 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703115) [S_b7 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703116) [S_b8 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703117) + (c) audifications of unknown things going on at the cantilever
[S_c1 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703119) [S_c2 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703118) [S_c3 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703120) [S_c4 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703121)
[S_c5 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703124) [S_c6 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703122) [S_c7 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703123) [S_c8 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703125) + (d) audifications of sticktip events
[S_d1 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703126) [S_d2 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703127) [S_d3 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703129) [S_d4 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703128)
[S_d5 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703130) [S_d6 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703131) [S_d7 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703132) [S_d8 (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703133) ### Interactive Sonifications using the Simulator System in Neo/NST The sounds contain 3 layers: (a) a wind-sound pattern that corresponds to the absolute force measured, (b) a repeating tone, whose pitch is determined by the net binding force, (c) marker sounds that are played on certain events connected to a specific physical meaning, like e.g. the rupture of a binding. The sound examples represent typical interactions using the simulator, a picture shows a screenshot of the system. Currently, the slider is used to navigate the cantilever, in the real AFM system, here the force feedback joystick comes into play. + [Sonification example 1 - wind sound layer (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703134) + [Sonification example 2 - as above plus tone sound layer (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703135) + [Sonification example 3 - as above plus marker sound layer (mp3)](https://pub.uni-bielefeld.de/download/2703101/2703136)