Combining Audio and Symbolic Approaches for Music Similarity Search

The ability to search music collections for occurrences of a given melody or similar musical material can be useful for several purposes. Examples include: finding items in libraries, music stores, or personal music collections; musicological research into how melodies spread and evolve; author attribution for anonymous works; and raising or resolving disputes about plagiarism. The topics considered in this project fall under the heading of Music Information Retrieval. There are three broad goals: combining symbolic and audio approaches to content-based music search, improving the state of the art for searching symbolic music data, and improving the speed of algorithms by advancing the so-called vantage indexing method. As part of my dissertation, I have developed a melody search algorithm for symbolically encoded music (MIDI files or scores). This algorithm has been demonstrated to be a leading algorithm. 1 However, it cannot be directly applied to audio collections because the automatic transcription of polyphonic audio recordings into scores will remain an unsolved problem for the foreseeable future. To apply my symbolic algorithm to audio data, one can, however, combine it with work on beat detection by Simon Dixon, who until recently worked at the inviting research institution, the Austrian Research Institute for Artificial Intelligence (OFAI). To use my algorithm for searching audio data for melodies, a new method for extracting melodies from audio data needs to be developed. My symbolic algorithm can still be substantially improved. I will quantify the effects of better voice splitting on the result quality, for example by combining a voice splitting algorithm by Madsen and Widmer (OFAI) with my algorithm, as well as one by Kilian and Hoos. Also, I will investigate how the search can be limited to relevant musical material, e. g., melodic lines, but not the accompaniment. With vantage indexing, information retrieval can be sped up if the underlying distance measure obeys the triangle inequality. This method can thus also be useful for other retrieval problems such as image retrieval or video retrieval. However, not much is known about how one can pick good vantage objects or how one could construct even better objects. It is also unknown whether combining multiple vantage indices can alleviate problems that arise if only a weak version of the triangle inequality holds, or if the triangle inequality only holds with a certain probability. I will investigate how good vantage objects can be constructed for the purpose of indexing music, and whether combinations of vantage indices are useful for distance measures for which the triangle inequality does not always hold. This project will deliver new methods for searching audio music data, an improved algorithm for searching music scores, and new insights about an indexing technique that is useful for music, but also for other data. 1 For the "Symbolic Melodic Similarity" results of MIREX 2006, see http://rainer.typke.org/mirex06.0.html