Brainstorming Meetings with Blind and Sighted Persons

The following research question is addressed in the proposed project: How can appropriate IT based means improve participation of the blind in workplace situations that require intense cooperation with the sighted? We plan scientific contributions in the areas of (a) novel interaction devices and (b) novel interaction techniques, combined with research in the area of (c) eAccessibility. Well matching research labs with considerable experience had to be found for each of the areas (a) - (c); they turned out to be spread over all three countries participating in the "lead agency" cooperation line: Switzerland (ETH Zürich, (a)), Germany (TU Darmstadt, (b)), and Austria (TU Linz, (c)). The project is motivated by a key goal on the path towards truly barrier-free work life, study and everyday life: equality of the impaired when cooperating closely with the non-impaired. In this context, the project addresses a particular category of working situations, namely team working sessions (creative team meetings). Such situations are highly relevant for the blind, in particular those exposed to the knowledge based society: they are both frequent and barrierintensive. These barriers concern both communication / coordination among session participants and subjects addressed in the meetings. Regarding communication / coordination among session participants, the sighted tend to use virtually all available perceptive entities either in parallel or in frequent alteration; in particular, the highly important implicit communication is marked by dynamic visual aspects. Even dedicated training for the sighted and years of experience in mixed settings are insufficient for eliminating these barriers. Although quite a number of means have been devised for mapping visual information to the haptic and acoustic channels, the effectiveness of these means in face of spontaneous or implicit communication / coordination remains insufficient. Regarding the subject addressed, there has been substantial progress in recent years regarding textual content. The situation is much worse regarding multidimensional structures as frequently used, e.g., for representing subjects of dynamic, abstract, and/or complex networked nature. Such subjects prevail in output oriented team working sessions. In this respect, the project selects mindmaps as an example for concrete empirical work. The aim of the project is scientific advancement in IT supported concepts and mechanisms for better integration of the blind in work places of the knowledge society. Team working sessions are selected as an exemplary application case. In a first step, typical work processes of blind participants in such sessions will be studied. Existing and innovative processes will be modeled, fundamental new concepts for interaction devices and techniques will be researched and realized as research prototypes, and evaluated in real applications with blind persons.

Collaborative Meetings are part of our professional lives. The joint project of partners ETH, JKU and TUD set its goal in enabling better accessibility and participation of blind persons in such meetings. To overcome barriers, pure visual information cues have to be made accessible for blind meeting participants. Two central layers of visual information exchange were identified: a) Artefacts as for instance text on a blackboard, slide or bubble in a mind map and b) nonverbal communication as for instance pointing gestures to highlight or nodding to agree or disagree. As these are vast domains with many diverse scenarios, ideation meetings using mind maps, displayed and arranged using a touch table, got chosen as a representative research scenario as they include many of the dynamic interaction elements used in everyday communication. For instance change of speaker, pointing/deictic gestures and dynamic modifications of the mind map artefacts are typical in such meetings and both are fundamental elements of non-verbal communication.As a representative example pointing/deictic gestures got selected for in-depth studies. A prototype (test scenario) was defined and implemented. Sighted meeting participants are able to manipulate the mind map using a touch table (Microsoft Surface). Blind meeting participants use their Assistive Technologies (AT; ScreenReader, Braille display, speech output) to handle the mind map in a synchronised tree view. LeapMotion devices are used to detect deictic gestures above the table to track and present their target by focussing it in the synchronised tree view. Blind users can also handle the focus with their ICT/AT what is synchronised in the visual display. For a usable application the key R&D questions are:a) Tracking the artefact focus and non-verbal communication elements, b) Pre-selection of important elements based on reasoning for immediate display and c) Non-disturbing presentation of non-verbal information elements, as much as possible in parallel to the communication flow, to avoid overloading and disturbance.The user driven design approach including end users in all stages showed that blind meeting participants considerably benefit and enjoy this support when these three quality aspects are in place and lead to a non-obtrusive presentation and interaction. Therefore this research project is seen as a starting point into the field of access to nonverbal communication for blind people and other target groups (e.g. learning disabilities, migration background) as well as for a general tool for supporting frictionless communication and cooperation.Besides state of the art AT we also examined prototypes of new devices as for instance two-dimensional tactile displays. It turned out that such devices support blind meeting participants in understanding two-dimensional relations in parallel without taking too much attention however it makes exploring and manipulating of mind maps more time-consuming what again disturbs application in practice. Also such alternative devices are not ready for the market and very expensive.