Acoustic modeling and transformation of varieties for speech synthesis

Our main goal in this research project is the advancement of variety modeling for speech synthesis through the optimal use of available data resources. The fact of having phonetically similar data within different social (sociolects) or regional (dialects) varieties and the potential to use statistical parametric synthesis to adapt models with a relatively small amount of data from background models will yield synthesis methods that can model varieties using only a few minutes of speech adaptation data. To reach this overall goal we focus on three topics that are highly relevant for variety modeling and that represent new scientific challenges, namely average voice models for varieties, modeling of variety transformation, and modeling of varieties with incomplete training data. In average voice modeling for varieties we will investigate variety and speaker adaptive training as a new training method for average voice models. In variety transformation we will develop techniques to create a speaker`s voice in a certain variety when only having speech data of the speaker in a similar variety. Furthermore we will investigate methods to create a speaker`s voice from incomplete speech data sets, which can be used to synthesize historic dialect states. Speech synthesis is becoming more and more important as an output interface in cognitive user interfaces. While it is possible to achieve natural sounding speech synthesis for neutral style speech with today`s technology, the fast adaptation of speech synthesis systems to different contexts and situations is still a problem, something to which both speakers and listeners are used to. While emotional speech and natural intonation are an area of active research, relatively little research is devoted to language varieties. Within this project we will develop the necessary methods to develop speech synthesis systems that can be easily adapted to social and regional language varieties. To achieve this we search for optimal ways to use the available training data by exploiting similarities within social and regional varieties using different layers of abstraction.

Our main goal in this research project was the advancement of variety modeling for speech synthesis. To reach this overall goal, we focused on three topics that are highly relevant for variety modeling and that represented new scientific challenges, namely modeling of variety transformation, average voice models for varieties, and modeling of varieties with incomplete training data. In modeling of variety transformation, we developed a method for unsupervised interpolation of language varieties that automatically creates in-between varieties by generating gradual transitions between two varieties, be it two dialects/sociolects, or a dialect and a standard. Furthermore, we developed a cross-variety speaker transformation method that can create a speakers voice in a certain variety even if only speech data of another variety of the speaker are available. In average voice modeling, we investigated different adaptation methods like dialect-adaptive training and dialect clustering that exploit the common phone sets of dialects and standard and applied an adaptive modelling method that uses one variety as background and one as adaptation variety to Albanian dialects.On modeling of varieties with incomplete training data we evaluated the perception of foreign-accented natural and synthetic speech in comparison to automatically accent-reduced synthetic speech. The applied method does not use an average voice model but only the phonetically incomplete accented speech data.Speech synthesis is becoming increasingly important as an output interface in cognitive user interfaces. While emotional speech and natural intonation are an area of active research, less attention has been paid to the investigation of language varieties in the context of speech synthesis. Within this project we developed methods for speech synthesis systems that can be easily adapted to social and regional language varieties.