SysSon - A systematic procedure to develop sonifications

Sonification is the acoustic analogue of data visualization. Research issues include parameter mapping, audification, model-based sonification, and iconographic representations (e.g., auditory icons). Different combinations of these methods take advantage of human perceptual and cognitive capabilities. Sonification is especially suited to the preliminary exploration of complex, dynamic, and multidimensional data sets. Still, the sonification community lacks vigorous case studies of the application of sonification in current exploratory research. The amount of data being processed today is steadily increasing, and both scientists and society need new ways to understand scientific data and their implications. In SysSon, we will develop a systematic procedure to develop sonifications for the foremost scientific domains. A Ph.D. thesis will apply this systematics with climate data in order to establish a case study of a systematic sonification of exploratory data. The SysSon approach includes three crucial steps addressing all relevant obstacles that are met when introducing sonification in a new scientific domain: Firstly, we will analyze the metaphors of the domain science and deduce a sound library specified for the field within a Master thesis. This will enable the interdisciplinary communication about the project results. Furthermore, the new sonifications will compensate for the inadequacies of existing visualization tools. Secondly, prototypes will repeatedly be evaluated by four stakeholder groups (domain scientists and aesthetically trained persons will evaluate the sounds in cycles within three test groups; sonification experts will assess the project results during a concluding workshop; and the general public will indirectly act as test subjects during a media installation). Thirdly, we will provide an open source, easy-to-use sonification tool for climate scientists, which fits their perceptualization needs and is compatible to their usual software environment. The research will be carried out at the Institute of Electronic Music and Acoustics (IEM), which has extensive experience in interactive sonification with multidimensional data sets. Cooperation partners are the Wegener Center for Climate and Global Change, an international leader in climate research, and the Centre for Systematic Musicology, responsible for the aesthetic part of the evaluation and the sound library. SysSon will lead to a more objective, straightforward development of sonifications and enhance their application in different fields. Conventions shall be established not in the sense of ready-made sonification advices, but regarding the process of developing sonifications, communicating their potential, and quality control.

The SysSon project developed an access to sonification for climate science. Sonification is the acoustic analogue to visualization. Climate scientists depend on visual displays to get an insight in their huge and multi-dimensional data sets. But visualization is limited for conceptual and perceptual reasons. Therefore, sonification may be used complementarily, using the human sense of hearing in order to explore climate processes. The projects focus was on the development of a systematic procedure to enable climate scientists to create their own sonifications. The technical barrier of compatibility between the data structures and sound synthesis programs was met by developing the open source software SysSon (freely downloadable from https://github.com/iem-projects/sysson). Furthermore, the interdisciplinary work process tackled communicational barriers, as, e.g., climate scientists are not used to working with sound and sonification experts are not used to climate science terminology. Therefore a series of evaluations was implemented, based on User Centered Design. This procedure of evaluations, design, and development lead to a process of systematic sonification that can be generalized to other scientific domains as well. Sonification is on the edge between science and the arts. Therefore, two artistic projects have been co-developed within the project. Klima|Anlage is a joint project with a German radio station and other partners (http://www.klima-anlage.org). Climate data were pre- processed to control an interactive climate sound machine. The installation has been shown in Berlin; further exhibitions are planned. Furthermore, the art installation Turbulence a climate sound portrait was presented in November 2014 in Forum Stadtpark, Graz, Austria, conveying the climate topic to the public.