SoniVis: Data Analytics using Sonification and Visualization

Data analysis, such as the analysis of production processes, includes a number of activities that cannot be automated completely. Therefore, the work of human analysts is often required. Both, information visualization and sonification the representation of data with images or sounds are considered effective methods to involve humans in data analysis. These two fields utilize the highly sophisticated visual and auditory information processing capabilities of humans. While information visualization approaches explore computer-based, interactive visual representations of data, sonification illustrates information through the use of nonspeech audio. With the increasing amount and complexity of data, both approaches have reached their limits, which is also the case for the human perceptual capacities serving as the basis for interpreting images and sounds. Although extensive research has been carried out on auditory and visual represent ation of data, there have been few attempts to combine these two channels in a systematic and complementary manner. So far, way too little attention has been paid to the interrelation of these modalities. Existing research on combinations has often focused on one modality while neglecting the other. A methodical approach to combine information visualization and sonification with regard to a complementary framework is still missing. Due to the rapidly increasing amounts of complex data to be analyzed, each individual approach reaches its limits. For this reason, more capable methods are urgently needed. Our project SoniVis aims to bridge the gap between sonification and visualization. To do this, we are working on creating a basis for a unified design theory for audio-visual analytics that combines these two modalities in a complementary way for exploratory data analysis. Within the framework of this project, we will (1) outline the range of design options for complementary audio-visual data analysis approaches; (2) examine the practical feasibility in the form of a design study based on the analysis of production processes; and (3) conduct a controlled experiment to empirically compare different complementary audio-visual analytics approaches. We will follow a human- centered and problem-driven research process and we will closely link design and evaluation phases to review our approaches in a timely manner, thus reducing the risk of unreliable results.

Data analysis, such as the analysis of production processes, includes activities that cannot be automated completely. Therefore, the work of human analysts is often required. Both, information visualization and sonification, the representation of data with images or sounds, are considered effective methods to involve humans in data analysis. These two fields utilize the highly sophisticated visual and auditory information processing capabilities of humans. While information visualization approaches explore computer-based, interactive visual representations of data, sonification communicates information using nonspeech audio. With the increasing amount and complexity of data, both approaches have reached their limits, which is also the case for the human perceptual capacities serving as the basis for interpreting images and sounds. Although extensive research has been carried out on auditory and visual representation of data, there have been few attempts to combine these two channels in a systematic and complementary manner. So far, way too little attention has been paid to the interrelation of these modalities. Existing research on combinations has often focused on one modality while neglecting the other. Therefore, there is still no established methodological approach to combine information visualization and sonification in the form of a complementary design theory. The SoniVis project explored the possibilities of analysis tools that integrate visualization and sonification, to form a type of display that can be "more than the sum of the two". For the establishment of a design theory of audio-visual data analytics, the project was able to provide research contributions on several levels: - Theory: a novel theoretical model for the formal description of audio-visual data analytics. - Design and Implementation: Two designs of audio-visual analytics idioms, called SoniScope and Parallel Chords, both intended to support users with the audio-visual exploration of multivariate data. - State-of-the-Art: an extensive state-of-the-art report providing a broad perspective on the field and the research community of audio-visual analytics. - Evaluation: the empirical evaluation of the Parallel Chords design introduced a methodological strategy for the evaluation of audio-visual analytics idioms. - Community Work: between 2021 and 2025, community bridging events at top-tier international conferences such as IEEE VIS, EG EuroVis, ACM Audio Mostly, ACM AVI, and ICAD, as well as a Dagstuhl seminar were organized. Providing the community with the option to meet and interact, these workshops and panel discussions led to several publications and collaborations.