PUB - Publikationen an der Universität Bielefeld

This paper introduces Wave Space Sonification (WSS), a novel class of sonification techniques for time- (or space-) indexed data. WSS doesn't fall into the classes of Audification, Parameter-Mapping Sonification or Model-based Sonification and thus constitutes a novel class of sonification techniques. It realizes a different link between data and their auditory representation, by scanning a scalar field – defined as wave space – along a data-driven trajectory. This allows both the highly controlled definition of the auditory representation for any area of interest, as well as subtle yet acoustically complex sound variations as the overall pattern changes. To illustrate Wave Space Sonification (WSS), we introduce three different WSS instances, (i) the Static Canonical WSS, (ii) Data-driven Localized WSS and (iii), Granular Wave Space Sonification (GWSS), and we demonstrate the different methods with sonification examples from various data domains. We discuss the technique and its relation to other sonification approaches and finally outline productive application areas. ##### **Canonical Wave Space Sonification of a circle trajectory** *S1-Canonic-WSS-Circle.wav* The sonification exhibits 3 cylces of the circling trajectory, see article for further details. ##### **Canonical Wave Space Sonification of periodic cycles using axis-aligned timbres** *S2-Canonic-WSS-timbre-axis.wav* This sonification uses a delay embedding (s(t), s(t-k)) of s(t) = sin(2*pi*t)+0.18*sin(2*pi*2*t) with t in [0, 0.01,...10] with k=20 using a wave space definition with 1st, 3rd, 5th harmonic along x-axis and 1st, 2nd, 3rd harmonic along y-axis ##### **Canonical Wave Space Sonification of the monthly number of sunspots dataset (k=12, rate=30)** *S3.1-Canonic-WSS-sunspots-k12-rate30.wav* In this WSS, the overall evolution of sunspots variations over 150 years can be heard in few seconds. The 11-year cycle of varying strength and its change over centuries can be perceived. ##### **Canonical Wave Space Sonification of the monthly number of sunspots dataset (k=12, rate=300)** *S3.2-Canonic-WSS-sunspots-k12-rate300.wav* In this WSS, the overall evolution of sunspots variations over 150 years can be heard in 30 seconds. The variations between the 11-year cycles can be perceived. ##### **Canonical Wave Space Sonification of the monthly number of sunspots dataset (k=24, rate=30)** *S3.3-Canonic-WSS-sunspots-k24-rate30.wav* In this WSS, the overall evolution of sunspots variations over 150 years can be heard in few seconds. The 11-year cycle of varying strength and its change over centuries can be perceived. Compared to S3.1, the delay embedding uses offset k=24, which affects the timbre. ##### **Canonical Wave Space Sonification of the monthly number of sunspots dataset (k=24, rate=300)** *S3.4-Canonic-WSS-sunspots-k24-rate300.wav* In this WSS, the overall evolution of sunspots variations over 150 years can be heard in 30 seconds. The variations between the 11-year cycles can be perceived. Compared to S3.2, the delay embedding uses offset k=24, which affects the timbre. ##### **Static Sample based Wave Space Sonification of the monthly number of sunspots dataset (k=24, rate=144)** *S4.1-Sample-based-WSS-sunspots-hp5-k24-rate144.wav* WSS of sunspots data set trajectories where the first wave space dimension is filled with the spoken vowels ‘a-e-i-o-u’ (vowels as in bath-bear-bee-beau-boo) with a short gap in-between, and the second wave space dimension is left silent. The vowels are reproduced at trajectory speed depending rate, accessing different vowels according to oscillation amplitude. ##### **Static Sample based Wave Space Sonification of the monthly number of sunspots dataset (k=24, rate=30)** *S4.2-Sample-based-WSS-sunspots-hp5-k24-rate30.wav* WSS of sunspots data set trajectories where the first wave space dimension is filled with the spoken vowels ‘a-e-i-o-u’ (vowels as in bath-bear-bee-beau-boo) with a short gap in-between, and the second wave space dimension is left silent. The vowels are reproduced at trajectory speed depending rate, accessing different vowels according to oscillation amplitude. Different from S4.1, here the rate is approx. 4 times higher, resulting in a shorter overall sonification. ##### **Static Sample based Wave Space Sonification of ECG (k=24, rate=40)** *S4.3-Sample-based-WSS-ecg-hp5-k24-rate40.wav* WSS of ECG time series delay embedding (k=24) trajectory where the first wave space dimension is filled with the spoken vowels ‘a-e-i-o-u’ (vowels as in bath-bear-bee-beau-boo) with a short gap in-between, and the second wave space dimension is left silent. 8 heart beats can be heard and their pattern becomes a clearly recognizable auditory gestalt. ##### **Static Sample based Wave Space Sonification of ECG (k=24, rate=160)** *S4.4-Sample-based-WSS-ecg-hp5-k24-rate160.wav* WSS of ECG time series delay embedding (k=24) trajectory where the first wave space dimension is filled with the spoken vowels ‘a-e-i-o-u’ (vowels as in bath-bear-bee-beau-boo) with a short gap in-between, and the second wave space dimension is left silent. Compared to S4.3, the trajectory speed is 4x slower, allowing more detailed attention to the patterns within the cycles. ##### **Samples used for Localized Wave Space Sonification** *S5.0-samples-r-s-whistle.wav* Introduction to three samples used for Localized Wave Space Sonification ##### **Data-Driven Localized Wave Space Sonification of ECG (k=16, rate=40)** *S5.1-LocWSS-ecg-hp45-k16-rate40-logs-1.7-mode2-fn4.wav* WSS of the ECG cycle depicted in Fig. 6 of the corresponding article, using a sound sample of spoken letter ’r’ and ’s’ for the outgoing (horizontal towards the right) and returning (vertical, downward) trajectory of the QRS-complex. A whistling sound is added to the origin with a 45 degree orientation, which is turned into a discernable pattern by the other parts of the ECG (u-wave, p- wave, etc.). ##### **Data-Driven Localized Wave Space Sonification of ECG (k=16, rate=160)** *S5.2-LocWSS-ecg-hp45-k16-rate160-logs-2.1-mode2-fn4.wav* This WSS is similar to S5.1, but the trajectory speed is slowed down by a factor of 4, allowing more time to follow details of the time series in between subsequent QRS-complexes. ##### **Granular Wave Space Sonification of ECG (paricle rate 3 Hz, particle duration 20 msec)** *S6.1-GWWS-hp45-k15-rate2-logsc3.3-mode0-fn0-p_rate3-p_dur-0.02-p_speed3.wav* This sonification shows how grains sound differently as they are spawned from different locations along the trajectory. ##### **Granular Wave Space Sonification of ECG (paricle rate 3 Hz, particle duration 100 msec)** *S6.2-GWWS-hp45-k15-rate2-logsc3.3-mode0-fn0-p_rate3-p_dur-0.10-p_speed3.wav* Compared to S6.1. the grain duration is increased to 100ms, allowing to perceive the inner temporal organization of grains ##### **Granular Wave Space Sonification of ECG (paricle rate 20 Hz, particle duration 100 msec)** *S6.3-GWWS-hp45-k15-rate2-logsc3.3-mode0-fn0-p_rate20-p_dur-0.10-p_speed3.wav* Compared to S2 now the grain rate is increased to 20Hz so that it can be better heard how the timbre changes with movement. Apparently this is not sufficient to discern the hear beats. ##### **Granular Wave Space Sonification of ECG (paricle rate 40 Hz, particle duration 30 msec)** *S6.4-GWWS-hp45-k15-rate2-logsc3.3-mode0-fn0-p_rate40-p_dur-0.03-p_speed3.wav* Increasing the rate to 40 grains/s, the heart beat pattern becomes salient, 5 heart beats become audible ##### **Granular Wave Space Sonification of ECG (paricle rate 40 Hz, particle duration 30 msec, speed 0.33)** *S6.5-GWWS-hp45-k15-rate6-logsc3.3-mode0-fn0-p_rate40-p_dur-0.03-p_speed3.wav* This 3*slower sonification gives more auditory access to variations within the beats. Without the granularization, the pitch would have dropped by more than one octave, rendering the timbre and pitch variations less perceivable. ##### **Granular Wave Space Sonification in a localized WSS of ECG (paricle rate 40 Hz, particle duration 60 msec, speed 0.5)** *S6.6-GWWS-hp45-k15-rate4-logsc2.3-mode0-fn3-p_rate40-p_dur-0.06-p_speed2.wav* In this sonification, the localized WSS definition using the vowels are used. Tuning the particle speed allows to maintain a particle speed at which the vowels remain discernable, independent of the overall trajectory rate. The auditory gestalt of 10 heart beats can be heard to be quite stable over time. ##### **Granular Wave Space Sonification in a localized WSS of ECG (paricle rate 40 Hz, particle duration 60 msec, speed 1)** *S6.7-GWWS-hp45-k15-rate2-logsc2.3-mode0-fn3-p_rate40-p_dur-0.06-p_speed2.wav* Different to sonification example S6.6. the trajectory rate is halfed, resulting in a 2x speedup, but now (and different strom standard WSS) without affecting the pitch.