Automation of live mediated communication

Recent progress in what is generally referred to as Artificial Intelligence (AI) has accelerated automation in a wide range of human activities. One of the tasks for which computer systems are widely considered capable of replacing humans is translation. Machine translation (MT) from one written language to another has indeed shown impressive improvement but still requires human post-editing to ensure high levels of quality and reliability. Translation between spoken languages, which requires speech recognition and speech synthesis in addition to MT, poses even greater challenges, particularly when it is done in real time for immediate use in communicative interaction. Such live translation to enable communication between speakers of different languages has traditionally been performed by interpreters, in settings ranging from doctorpatient interviews to conference presentations. Although automatic speech-to-speech translation (AST) still has a number of limitations, it is increasingly being viewed as an alternative to employing interpreters. However, the extent to which AST systems can be relied on to mediate interlingual communication with satisfactory levels of accuracy and comprehensibility remains largely unclear. The project therefore aims to compare the performance of AST systems and interpreters in realistic communication scenarios requiring live translation between spoken languages. These include counseling by health and social care providers as well as international events with presentations in languages other than German (or English). Consequently, the AST systems investigated in the project will include dialogue translation apps run on mobile devices as well as cloud-based solutions for simultaneous speech translation. The performance of the respective AST systems will be compared to that of professional interpreters providing short-consecutive or simultaneous interpreting on site or via a videoconference link. A fundamental component of this project is the development and validation of a consistent method for assessing the quality of live translation. Quality, broadly understood in line with functionalist translation theories as fitness for purpose, comprises both translational performance and user reception. The live- translation outputs of AST systems and interpreters will therefore be assessed for fidelity, formal correctness and fluency and also investigated from the perspective of users with regard to comprehensibility and usability. In order to account for the situatedness of live mediated communication, the research design envisages multiple case studies of live mediated interaction and takes a multi-method approach involving the analysis of audio and video recordings and transcriptions as well as questionnaires and interviews. The research findings will show the potential and limitations of AST systems and can inform decisions on their use as an alternative to professional interpreters in multilingual communication.