Frames and Linear Operators for Acoustical Modeling and Parameter Estimation

We live in the age of information where the analysis, classification, and transmission of information is of essential importance. Signal processing tools and algorithms form the backbone of important technologies like MP3, digital television, mobile phones and wireless networking. Many signal processing algorithms have been adapted for applications in audio and acoustics, also taking into account the properties of the human auditory system. The mathematical concept of frames describes the theoretical background for signal processing in a very general and flexible way. Frames are generalizations of orthonormal bases which allow for more freedom in the analysis and modification of information - however, this concept is still not firmly rooted in applied research. A link between mathematical frame theory, signal processing algorithms, their implementations and finally applications in acoustics is a very promising, synergetic combination of research in different fields. Therefore the main goal of this multidisciplinary project is to establish frame theory as the theoretical backbone of acoustical modeling in particular in psychoacoustics, phonetics, and computational acoustics. Based on this connection of disciplines, FLAME will produce substantial impact on both theory and applied research. The theoretical part of FLAME focuses on three topics. First, we will develop frame analysis and reconstruction methods beyond classical approaches; we will develop algorithms to approximately reconstruct with complete Bessel sequence, a non- stationary Gabor transform motivated by human auditory perception and a perceptually motivated, continuous time-frequency transform related to alpha-modulation spaces. Secondly, we will substantially extend the knowledge about frame multipliers; in particular, we will develop new algorithms for the inversion of frame multipliers, we will consider phase modifications in Gabor multipliers to optimize amplitude modifications with positive masks, we will propose new methods to find the best approximation with Gabor multipliers by using sub-band approaches, and we will establish new methods for system approximation and estimation using generalized frame multipliers. The last topic in the theory-based part, frame representation of operators, is motivated by computational acoustics: we will find sufficient conditions for operators which are represented sparsely by curvelets and shearlets, we will extend the idea of operator representation by frames to Banach, Gelfand, and fusion frames, and we will connect the concepts of localized frames and hierarchical matrices. The application-oriented part of FLAME consists also of three topics. First, we will improve the irrelevance algorithm by using a perceptual-based Gabor transform and by incorporating recent psycho-acoustical findings about time-frequency masking. We will also improve the irrelevance method by inclusion of an algorithm that separates a signal into tonal, transient, and noise parts. We will invent a "perceptual norm" for signals based on the well-known model for computational auditory signal processing and perception. In the second topic, we will apply advanced frame methods to the analysis and classification of speech. We will develop a pole-and-zero tracking algorithm based on a perceptually motivated non-stationary Gabor transform, a system-identification algorithm to separate the distortion by transmission via mobile device and the speech-production filter, and a speech/speaker-classification method based on Gabor multipliers. In the last topic, we will use advanced frame methods for signal enhancement and system estimation: We will formulate denoising algorithms for the improvement of system estimation when using sparse signals like sweeps by providing a statistical description of colored noise in an overcomplete non-stationary Gabor analysis. This ample and ambitious plan will be realized with the support of a strong network of international, national and internal cooperation partners.

Signal processing tools and algorithms form the backbone of technological advances in audio and acoustics. The mathematical concept of frames describes the theoretical background in a very general and flexible way. The link between the mathematical frame theory, signal processing, implementations and acoustical applications had not been fully recognized and was the promising, multi-disciplinary research environment of this project. The ambitious goal of this project was to establish frame theory as theoretical backbone of acoustical modeling. The meta-goals of FLAME can be considered fully achieved: Frame theory and its applicability were significantly advanced, complying with the principles of application-oriented mathematics: Provide better methods for applications and advance the theory. The multi-disciplinary approach has enabled FLAME to study many mathematical concepts motivated by applications in acoustics. FLAME provided many results in frame theory, new mathematical concepts, and the integration of frame theory in audio signal processing, acoustical applications and beyond. The considerable scientific impact of FLAME is evidenced by many publications in top-level journals and the participation in and management of workshops and conferences. All resulting algorithms were implemented in an open-source-toolobox. The scientific ripple generated by FLAME will be felt for years to come in the form of inspiring internal and external research topics and a flow of related publications. Let us mention a few exemplary results: Frames are particularly useful when one wants a reconstruction without the standard basis approach. We have introduced a generalization called reproducing pairs, where this is possible with even less restrictions. Many if not all audio processing tasks will be presented to and judged by a human listener. We have therefore developed a perception-motivated time-frequency representation that is still stable and invertible, the AUDlet transform. In many applications a time-varying change of the frequency content of a signal is done, for example when a sound engineer records a song. One way to describe this process is by a frame multiplier, a focus of this project. We found that, for example, the inverse of such a process can be nicely represented by a particular dual frame. For many time-frequency representations for decades only the magnitude has been considered, while in the last years the phase has been recognized as being relevant. Currently a very active research topic is how to generate the phase out of the magnitude, so called phase-retrieval. We developed a fast version of the famous Griffin-Lim algorithm to solve this task. We have linked the matrix representation of operators, needed for example for the numerical simulation of acoustical waves, to frame theory. We did this also in the context of frame-like sequences of subspaces, so called fusion frames, aiming at solving the domain overlapping problem in the numerical solution of operator equations.