Objective differentiation of dysphonic voice quality types

Voice problems are not completely understood yet, partly because conventional methods for clinical examination are limited. Aim of this research project is to create computerized methods which automatically recognize and identify reliably voice problems in microphone recordings of patients voices. Videos of the throat and microphone recordings of 230 patients with voice problems are obtained. Super slow motion videos with 4000 images per second are used, because the vocal folds, which are located in the larynx, normally vibrate very fast during voice production, more often than 100 times per second. To visualize irregularities, videos showing two seconds of vocal fold vibration are slowed down to a playing time of five minutes. The researchers investigate how the vocal fold vibration relates to the sound of the voice. In particular, three voice types are payed attention to. First, a so called vocal fry is a very low pitched type of voice. This voice type may be evocative of the Strohbass singing register. Sometimes, it is possible to hear individual pulses of the vocal folds in these voices. Second, extrapulsed voices are investigated. These may be compared to a common phenomenon in human heart beating, i.e., extra systoles. Just as a human heart may sometimes stumble every now and then, extra pulses may occur in the voice. Frequently occurring extra pulses may be a sign of a voice problem, may be perceived to sound raspy. Third, phase differences between the left and the right vocal fold may occur in voice disorders. To understanding phase differences, one could try to bounce two basketballs with the left and the right hand simultaneously, and notice that the balls will hardly ever hit the ground at exactly same time, and two separate bouncing sounds will be audible every time the balls hit the ground. Such timing differences also occur in vocal folds, and are a sign of a voice problem. Interestingly, instead of hearing the two vocal folds separately, listeners have reported to hear a certain type of rumbling in voices with phase differences. This is mainly because the frequency of the vocal folds is much higher than the frequency of basketballs. A diverse approach is chosen to investigate how the vocal folds vibrate, and how abnormal voices sound to listeners. The project comprises computer-based science, patient data, and auditory experiments. The results may be applied for improving clinical voice quality assessment.

Verbal communication is one of the most significant human achievements, relying on the functioning of the voice box, particularly the vibration of the vocal cords. This vibration gives the voice its tone, similar to how a vibrating string gives a guitar its sound. Voice disorders may disrupt this normal vibration, making speaking difficult. Clinicians use cameras to examine patient's vocal folds and listening to the nuances of the voice. However, vocal cords vibrate rapidly, making them difficult to see, and both visual and auditory assessments can be subjective. This project aimed to address these challenges through innovative technology and methods. The innovative techniques and significant findings are the following. First, researchers used high-speed cameras to slow down voice recordings by a factor of 160. This allowed for observation of minute details otherwise missed, especially features of irregularity. Second, microphone recordings were analyzed to understand how vocal sounds relate to vibrations. This led to improved understanding of how different vocal fold conditions affect voice sound. Third, the project involved advanced simulations of vocal fold vibrations and the auditory process, further pinpointing critical features of phonatory dysfunctions. Finally, the project leveraged artificial intelligence and machine learning (AIML). In particular, recent advances in speech technology (cf. Siri, Alexa, etc.) were adapted to create more realistic simulations of pathological voices, and AIML mimicking human vision was used to automate video analysis, reducing the need for manual review and facilitating the implementation of slow-motion video analysis in clinical settings. Specific voice types were investigated. First, diplophonia is a condition where different regions of the vocal folds vibrate at distinct rates, causing a doubled voice. Software was developed to measure the frequency of this occurrence in speech objectively. Second, vocal fry and creaky voice are characterized by separated sound pulses, similar to the sounds of a frying pan, a creaky door, or the making of popcorn. This research clarified that such voices either have a low vibration rate, or other disturbances creating only the illusion of pulse separation. Third, researchers investigated timing differences between vocal fold regions and extra pulses similar to extra systoles in heartbeats. In summary, this research has greatly enhanced our understanding of vocal fold mechanics and voice perception. By combining high-speed video technology, computer simulations, and AI, the project tackles key challenges in diagnosing and treating voice disorders. The findings have the potential to transform clinical practices, providing more accurate and reliable diagnostics via digital twinning and decision support, ultimately leading to better treatment outcomes and an improved quality of life for individuals with voice problems.