Computer Science, Artificial Intelligence, Music

Gerhard Widmer is a scientist who takes a clearly interdisciplinary approach, and he has gained worldwide renown as a pioneer in research at the intersection between information science, artificial intelligence (AI) and music; in fact, Widmer has played a key role in establishing, developing and gaining recognition for this field of research. With his research teams in Linz and Vienna, Widmer is working on new computer-based methods for intelligent music processing. One particular sensation was his research on the use of novel computerised methods for the quantitative analysis and modelling of artistic nuances in expressive music performances. He launched those research efforts with great panache in the course of his FWF START project: Among other achievements, he succeeded in quantifying and classifying subtle details in the styles of great pianists, and his intelligent computer solutions even managed to reveal and describe several fundamental principles of expressive tempo and dynamics on the basis of huge quantities of measurement data. That was probably the first time that a computer had independently made a truly new contribution to music research. Widmer`s work is purely cognition-oriented basic research and has attracted a great deal of attention worldwide (including popular media such as the New York Times and Wired magazine). The most recent highlight in Widmer`s work was winning all three main prizes at a scientific computer performance rendering competition in Sapporo, Japan (Sept. 2008) with a programme that autonomously attempts to play a given musical piece "expressively". At the same time, Widmer and his teams are also developing music analysis methods with practical applications for the new world of digital music, including algorithms which recognise musically relevant patterns in audio signals or which simulate humans` perception of similarities in music. In this field as well, Widmer`s work yielded another scientific first: In March 2009, a renowned audio equipment producer launched the world`s first digital hi-fi system which automatically analyses stored music based on musical criteria and then independently compiles playlists on the basis of those analyses. The technologies developed by Widmer`s team for the recognition of similarities in music are at the cutting edge worldwide, and two patents were recently registered in the US. Therefore, Widmer`s research interests cover a wide range, from pure fundamental research to technologies with impressive practical applications. At the international level, Widmer is recognised as a pioneer in the field of artificial intelligence and music, and his research teams are among the most productive groups worldwide in this field. With the funds from his Wittgenstein Award, Widmer now plans to raise his research to a new level of quality by addressing a difficult and scientifically risky topic: His vision is to implement substantial "understanding" of music in a computer, that is, to develop methods which enable a computer to describe and "understand" music on an abstract and semantically meaningful level comparable to that observed in human listeners. This would narrow the chasm between raw audio signals and musical meaning, also known as the "semantic gap" in the research world. On this basis, Widmer envisages the development of machines which can, for example, understand abstract characterisations of pieces of music and generate such characterisations themselves, which can track and respond to musical performances in real time, or which can identify and correctly classify expressive elements in music recordings. Ultimately, Widmer hopes this will also create the basis for novel applications in music searches, production and performance as well as music education.

FWF Project Z159 is the result of a Wittgenstein Prize awarded to Gerhard Widmer in 2009. The project was supported by the generous sum of EUR 1.4 million, and its purpose was to greatly advance our research in the intersection of computer science, Artificial Intelligence (AI), and music.The goal of our research is to develop computer systems that can listen' to music, develop a basic `understanding' of the contents and meaning of musical signals, and learn to recognise, classify, synchronise, and manipulate music in `intelligent' ways and this way support many important practical applications in the digital music world.This particular research project focused on two grand goals: one, to teach computers to recognise musically relevant patterns and structure in music recordings, much as human listeners do when listening to music, at many different levels - e.g., to recognise event onsets in music, identify beat, rhythm, tempo, metrical structure, harmonies, instruments, voices -- and to recognize musical pieces and track them (i.e., follow along in the sheet music, as musicians do) in real time (live). The outcome of this work are numerous new computer listening algorithms that are among the (or are the) best in the world for these tasks, as was also shown by winning first prizes in many international scientific competitions (see www.cp.jku.at/awards). Some of these algorithms have also found their way into real commercial applications in the digital media world (e.g., automatic media monitoring, radio broadcast analysis, and music search and recommendation services.)The second goal was to go a level `deeper', developing computer methods that can help us get a deeper understanding of the meaning' of music, its expressive aspects, how music becomes human music' through the artistic act of interpretation and performance. Here, we greatly advanced previous work on computer systems that investigate the art of expressive music performance, analysing performances by great human musicians and learning to describe and predict how music needs to be played (e.g., in terms of timing, dynamics, articulation) so as to sound musical' and natural' to us. The result of this strand of research are computer programs that have helped discover and describe interesting details about the art of great pianists (these were also published in the international world of musicology), and programs that can learn themselves to play music in musically meaningful and `expressive' ways, winning, among other things, an international Computer Piano Performance Contest (RENCON 2011). The last result in this respect (Spring 2017) is that in a blind listening test, our computer's performance of a piano piece was judged by human listeners as more `human' than that of an actual concert pianist ...The work started in the Wittgenstein Project Z159 will be continued and brought to a great synthesis in a new long-term project funded by the European Research Council (ERC) (www.cp.jku.at/research/projects/ConEspressione) - a project that would not have been possible without the support of the Wittgenstein project, and the FWF.