Sonification of Simulations in Computational Physics (Thesis, K. Vogt)
Soniﬁcation is the translation of information for auditory perception, excluding speech itself. The cognitive performance of pattern recognition is striking for sound, and has too long been disregarded by the scientiﬁc mainstream. Examples of ‘spontaneous soniﬁcation’ and systematic research for about 20 years have proven that soniﬁcation provides a valuable tool for the exploration of scientiﬁc data.
The data in this thesis stem from computational physics, where numerical simulations are applied to problems in physics. Prominent examples are spin models and lattice quantum ﬁeld theories. The corresponding data lend themselves very well to innovative display methods: they are structured on discrete lattices, often stochastic, high-dimensional and abstract, and they provide huge amounts of data. Furthermore, they have no inherently perceptual dimension.
When designing the soniﬁcation of simulation data, one has to make decisions on three levels, both for the data and the sound model: the level of meaning (phenomenological; metaphoric); of structure (in time and space), and of elements (‘display units’ vs. ‘gestalt units’). The design usually proceeds as a bottom-up or top-down process.
This thesis provides a ‘toolbox’ for helping in these decisions. It describes tools that have proven particularly useful in the context of simulation data. An explicit method of top-down soniﬁcation design is the metaphoric soniﬁcation method, which is based on expert interviews. Furthermore, qualitative and quantitative evaluation methods are presented, on the basis of which a set of evaluation criteria is proposed. The translation between a sci-
entiﬁc and the sound synthesis domain is elucidated by a soniﬁcation operator. For this formalization, a collection of notation modules is provided.
Showcases are discussed in detail that have been developed in the interdisciplinary research projects SonEnvir and QCD-audio, during the second Science By Ear workshop and during a short-term research visit at CERN. They show diverse applications of soniﬁcation for data exploration.
Meaning, structure and elements: in both the sound model and in the data model, decisions at these three levels have to be made. The levels are ordered hierarchically, along the auditory perception, on the sound side, and the mathematical treatment, on the data side.
The metaphoric content of the sound should fit the phenomenological meaning of the data. Hypothesized `audiables' face observables. In structure, sound and data decisions differ from each other; sound is at least always structured in time and in some sonic space, while the simulation data is defined globally on a lattice, and often has local entities of interest. Also the elements, the `gestalt units' as smallest perceivable entities, and the `display units' that are actually used in the sonification, are not necessarily equivalent to each other.
For structure and elements, also the implementation (choice of interface, playback mode and interaction possibilities) plays a role. Finally, evaluation should influence all decisions in an iterative design process.
The sound of a soniﬁcation has, like any sound, a metaphoric content. Ideally, the sound is designed in a way that it ﬁts the metaphors of the ﬁnal users. We suggest a metaphoric soniﬁcation method in order to explore the most intuitive mapping choices with the right polarities. The method is based on recorded interviews, asking experts in a ﬁeld what they expect data properties to sound like. Language metaphors and sounds of the recordings are then interpreted by the soniﬁcation designer. The method has been used for developing an ‘Acoustic Standard Model of particle physics’ with physicists at CERN.
(see also >> ICAD paper)
The sonification operator was recently suggested by J. Rohrhuber. It consists of a formalization of the sonification, mking the mapping between the domain science and the sound synthesis more explicit. I developed notation modules, that are described in chapter 5 of the thesis and used in chapter 6 to describe various sonification examples. (>>thesis)
As new evaluation approach, the Multi-criteria decision aid (MCDA; originally a method of politics, economy and social science) was introduced to the field of sonification. In the workshop Science By Ear II, evaluation criteria were developed and weighted according to their importance to a `good' sonification. Results can be found here.