Home
Sonification of Simulations in Computational Physics (Thesis, K. Vogt)
>> download thesis (pdf, 25MB)
Abstract
Sonification is the translation of information for auditory perception, excluding speech itself. The cognitive performance of pattern recognition is striking for sound, and has too long been disregarded by the scientific mainstream. Examples of ‘spontaneous sonification’ and systematic research for about 20 years have proven that sonification provides a valuable tool for the exploration of scientific data.
The data in this thesis stem from computational physics, where numerical simulations are applied to problems in physics. Prominent examples are spin models and lattice quantum field theories. The corresponding data lend themselves very well to innovative display methods: they are structured on discrete lattices, often stochastic, high-dimensional and abstract, and they provide huge amounts of data. Furthermore, they have no inherently perceptual dimension.
When designing the sonification of simulation data, one has to make decisions on three levels, both for the data and the sound model: the level of meaning (phenomenological; metaphoric); of structure (in time and space), and of elements (‘display units’ vs. ‘gestalt units’). The design usually proceeds as a bottom-up or top-down process.
This thesis provides a ‘toolbox’ for helping in these decisions. It describes tools that have proven particularly useful in the context of simulation data. An explicit method of top-down sonification design is the metaphoric sonification method, which is based on expert interviews. Furthermore, qualitative and quantitative evaluation methods are presented, on the basis of which a set of evaluation criteria is proposed. The translation between a sci-
entific and the sound synthesis domain is elucidated by a sonification operator. For this formalization, a collection of notation modules is provided.
Showcases are discussed in detail that have been developed in the interdisciplinary research projects SonEnvir and QCD-audio, during the second Science By Ear workshop and during a short-term research visit at CERN. They show diverse applications of sonification for data exploration.
Design Decisions
Meaning, structure and elements: in both the sound model and in the data model, decisions at these three levels have to be made. The levels are ordered hierarchically, along the auditory perception, on the sound side, and the mathematical treatment, on the data side.
The metaphoric content of the sound should fit the phenomenological meaning of the data. Hypothesized `audiables' face observables. In structure, sound and data decisions differ from each other; sound is at least always structured in time and in some sonic space, while the simulation data is defined globally on a lattice, and often has local entities of interest. Also the elements, the `gestalt units' as smallest perceivable entities, and the `display units' that are actually used in the sonification, are not necessarily equivalent to each other.
For structure and elements, also the implementation (choice of interface, playback mode and interaction possibilities) plays a role. Finally, evaluation should influence all decisions in an iterative design process.
Metaphoric Sonification
The sound of a sonification has, like any sound, a metaphoric content. Ideally, the sound is designed in a way that it fits the metaphors of the final users. We suggest a metaphoric sonification method in order to explore the most intuitive mapping choices with the right polarities. The method is based on recorded interviews, asking experts in a field what they expect data properties to sound like. Language metaphors and sounds of the recordings are then interpreted by the sonification designer. The method has been used for developing an ‘Acoustic Standard Model of particle physics’ with physicists at CERN.
(see also >> ICAD paper)
Notation Modules
The sonification operator was recently suggested by J. Rohrhuber. It consists of a formalization of the sonification, mking the mapping between the domain science and the sound synthesis more explicit. I developed notation modules, that are described in chapter 5 of the thesis and used in chapter 6 to describe various sonification examples. (>>thesis)
Evaluation
As new evaluation approach, the Multi-criteria decision aid (MCDA; originally a method of politics, economy and social science) was introduced to the field of sonification. In the workshop Science By Ear II, evaluation criteria were developed and weighted according to their importance to a `good' sonification. Results can be found here.
