Probabilistic Graphical Models for Time-Series Signal Mixtures

Robustness against reverberation, noise, and interfering audio signals is one of the grand challenges in speech recognition, speech understanding, and audio analysis technology. One avenue to approach this challenge is single- channel audio separation. Recently, factorial hidden Markov have won the single-channel speech separation and recognition challenge. These models are capable of modeling acoustic scenes with multiple sources interacting over time. While these models reach super-human performance on specific tasks, there are still serious limitations restricting the applicability in many areas. We aim to generalize these models and enhance their applicability in several aspects: (i) Introduction of discriminative large margin learning techniques. This allows to focus the model specification on the most salient differences, i.e. discriminating information, between interfering sources. (ii) Development of efficient inference approaches. Efficient inference is needed since the computational demands of exact inference in factorial hidden Markov models scale exponentially with the number of sources, i.e. inference is intractable in tasks with many interacting sources. (iii) We are interested in adapting the model parameters during separation to the specific situation (e.g. actual speakers, gain, etc.) using only speech mixture data. Therefore, an expectation-maximization- like iterative adaptation framework initialized with universal models, e.g. speaker independent models, is proposed. This greatly increases the utility of this model. Currently, source-specific monaural data is required to learn the model. The models and methods derived are applied to single-channel speech separation, tracking of the fundamental frequency of concurrent speakers, and benchmark classification scenarios. The overall goal is to devise methods for next generation time-series models well-suited for monaural audio data generated from multiple interacting sources. These models are also appealing to related fields requiring signal separation. Examples are resolving interactions in brain-scan images or seismic data.

Robustness against reverberation, noise, and interfering audio signals is one of the grand challenges in speech recognition, speech understanding, and audio analysis technology. One avenue to approach this challenge is single-channel audio separation. Recently, factorial hidden Markov have won the single-channel speech separation and recognition challenge. These models are capable of modeling acoustic scenes with multiple sources interacting over time. While these models reach super-human performance on specific tasks, there are still serious limitations restricting the applicability in many areas. In this project we developed self-adaptation methods for separating single-channel signal mixtures. Furthermore, we developed discriminative learning methods to increase the performance of signal separation. For increasing the resource-efficiency we investigated reduced precision analysis for classification.These methods for speech enhancement are important in many telecommunication applications. Improving speech intelligibility and quality has been an active field of research for many decades.