Cross-layer prosodic models for conversational speech

Automatic speech recognition (ASR) systems were originally designed to cope with carefully pronounced speech. Most real world applications of ASR systems, however, require the recognition of spontaneous, conversational speech (e.g., dialogue systems, voice input aids for physically disabled, medical dictation systems, etc.). Compared to prepared speech, conversational speech contains utterances that might be considered `ungrammatical` and contain disfluencies such as ...oh, well, I think ahm exactly . Moreover, in spontaneous conversation, a word like yesterday may sound like yeshay and the German word haben (to have) may sound like ham. The pronunciation of the words depends on well-known factors, for instance on the regional background of the speakers and the formality of the situation. Highly influential, but not so well studied factors are those reflecting the prosodic characteristics of the word in the utterance. These prosodic characteristics describe the rhythm and melody of a sentence, and for instance whether a word is accented or not. The proposed project aims at investigating which role prosody plays for pronunciation variation from a linguistic point of view and at incorporating gained knowledge into an ASR system. In our investigations, we will use speech material from German and Austrian speakers. In contrast to most research in the field of prosody which used read sentences or prepared speech, we will annotate and analyze speech from free conversations between speakers who know each other well. Such speech material is not only more naturalistic, but also richer in pronunciation variation. In sum, our project will deliver the first prosodically annotated database for conversational Austrian German, automatic tools for the creation of prosodic annotations and a prosody-dependent ASR system for conversational speech from German and Austrian German speakers.

Cross-layer prosodic models for conversational speech Automatic speech recognition (ASR) systems were originally designed to cope with carefully pronounced speech. Most real world applications of ASR systems, however, require the recognition of spontaneous, conversational speech (e.g., dialogue systems, voice input aids for physically disabled, medical dictation systems, etc.). Compared to prepared speech, conversational speech contains utterances that might be considered 'ungrammatical' and contain disfluencies such as "...oh, well, I think ahm exactly ". Moreover, in spontaneous conversation, a word like "yesterday" may sound like "yeshay" and the German word "haben" ("to have") may sound like "ham". The pronunciation of the words depends on well-known factors, for instance on the regional background of the speakers and the formality of the situation. Highly influential, but not so well studied factors are those reflecting the prosodic characteristics of the word in the utterance. These prosodic characteristics describe the rhythm and melody of a sentence, and for instance whether a word is accented or not. This Elise Richter project investigating which role prosody plays for pronunciation variation from a linguistic point of view and investigated statistical methods for its quantitative analyis. In our investigations, we used speech material from German and Austrian speakers. In contrast to most research in the field of prosody which used read sentences or prepared speech, we annotated and analyzed speech from casual, free conversations between speakers who know each other well. Such speech material is not only more naturalistic, but also richer in pronunciation variation, coming with the challenge of more variation and the requirement of more complex statistic techniques. One of our main findings was that from a speech-melody, rhythm point of view, German and Austrian German conversations show a more similar pattern than Austrian read and Austrian conversational speech, leading us to the conclusion that with respect to prosodic phrasing, speaking style is more relevant than the regional background of the speakers. One main deliverable of the project was the first prosodically annotated database for conversational Austrian German, along with automatic tools for the creation of prosodic annotations. These will continued to be used to develop an automatic speech recognition system for conversational speech from German and Austrian German speakers. What is more, the speech database is already in use by linguists and speech technologists by national and international academic research institutions