Localization of sound sources with cochlear implants

Electrical stimulation of the cochlear nerve by means of Cochlear Implant (CI) systems has been shown to successfully substitute basic functions of the inner ear: it enables deafened people to understand speech. Current CI systems, however, can not replace all functions of the normal auditory system. While localization of sound sources in the left/right dimension has been shown to be possible by means of CI stimulation at both ears, localization in the front/back and up/down dimensions (within the so-called sagittal planes) still remains difficult because of insufficient transmission of relevant cues by current CI systems. In normal hearing, localization in the sagittal planes is achieved by the evaluation of spectral colouring of the incoming sound induced by the external ear (pinna) at high frequencies (pinna cue or spectral cue). Current CI systems, however, do not transmit pinna cues due to the placement of the microphone behind the pinna and the restricted bandwidth of the signal analysis stage of the processor. While these technical limitations can be handled, the following basic questions remain: a) How many frequency channels are required to transmit pinna cues relevant for sound localization in the sagittal planes? b) To what extent can CI listeners learn to localize sound sources in the sagittal planes using high-frequency pinna cues that are mapped to tonotopic places with lower characteristic frequencies, as usually covered by CI electrodes? In this project these questions are studied by conducting psychoacoustic experiments with CI and normal hearing subjects, the latter being presented with acoustic simulations of the perception of a cochlear implant listener. The ultimate goal of the project is to enable CI listeners to localize sound sources in the sagittal planes. First, a series of experiments determines the effects of parameters, such as the number and tonotopic range of the stimulating electrodes, on the coding of pinna cues and the speech signal. In a second stage, the ability of both subject groups to learn localization in the sagittal planes is studied using pinna cues that are frequency-shifted to match the tonotopic range associated with the basal electrodes in CI listeners. The subjects learn to associate auditory stimuli with spatial positions by means of audio-visual stimulus presentation. CI listeners are expected to benefit from the coding of pinna cues with respect to localization of sound sources, in particular regarding front/back discrimination.

Current cochlear implants (CIs) allow implantees some ability to localize sounds in the left/right dimension. However, localization in the vertical planes, involving front/back and up/down localization, is more complicated because of insufficient transmission of relevant cues. In normal hearing (NH), localization in the vertical planes is achieved by the evaluation of spectral coloring of the incoming sound induced by the external ear at high frequencies. In this project we addressed several basic questions on the feasibility of providing spectral localization cues with CIs, with the ultimate aim to allow CI listeners to localize in the vertical planes. First, we developed methods to train listeners with new localization cues, involving a virtual audio-visual environment. Next, we studied the ability of bilateral CI listeners to localize in 3-D space with their clinical devices. As expected, vertical-plane localization was much worse than in NH listeners and relied solely on absolute level cues instead of spectral cues. Next, we explored the frequency channels and spectral range required for high speech intelligibility in quiet and background with a 12-channel CI. At least eight channels and frequencies up to about 3 kHz were found to be sufficient, allowing to encode spectral localization cues via the remaining four channels. Next, we addressed the question if CI listeners are able to discriminate between different spectral shapes, which is required for vertical-plane localization. All six CI listeners were sensitive to spectral peaks and notches imposed on a constant-loudness background, but the sensitivity declined dramatically when listeners were prevented from using absolute level cues. This is in contrast to NH listeners, who are sensitive even without absolute level cues. Given that NH listeners receive both spectral and temporal cues, the latter having been excluded in our CI study, we speculate on the importance of temporal cues. Next, we studied how many frequency channels are required for robust vertical-plane sound localization after an audio-visual training. NH subjects, listening to acoustic CI simulations, showed robust localization in a configuration with only four channels within the frequency range containing spectral localization cues. In the last project stage, we hypothesized that the auditory localization mechanisms can adapt to a warping of spectral localization information towards lower frequencies. This is important because in electric hearing the upper limit of the stimulation range is considerably lower than in acoustic hearing. Fifteen NH listeners completed a localization training experiment, involving two-hour daily training over a period of three weeks. After an initial strong increase of the localization error relative to the baseline performance without frequency-warping, the subjects` localization performance improved steadily, although not reaching the baseline performance at the end of the training period. The combination of results from the different project stages shows that it may be possible to enable vertical-plane localization in recipients of future CI systems by properly encoding spectral localization cues. While our results indicate that directional features can be mapped to the limited range of spectral channels available in electric hearing, one major challenge towards this goal will be to improve the CI listeners` spectral shape discriminate ability, possibly by means of including temporal cues.