Audiovisual Speech Perception at Various Presentation Levels in Mandarin-Speaking Adults with Cochlear Implants

et al. (2014) Audiovisual Speech Perception at Various Presentation Levels in Mandarin-Speaking Adults with Cochlear Implants. PLoS ONE 9(9): e107252. doi:10.1371/journal.pone.0107252 Audiovisual Speech Perception at Various Presentation Levels in Mandarin-Speaking Adults with Cochlear Implants Shu-Yu Liu 0 Grace Yu 0 Li-Ang Lee 0 Tien-Chen Liu 0 Yung-Ting Tsou 0 Te-Jen Lai 0 Che-Ming Wu 0 Christian Friedrich Altmann, Human Brain Research Center, Japan 0 1 Institute of Medicine, Chung Shan Medical University , Taichung, Taiwan , 2 School of Speech Language Pathology and Audiology, Chung Shan Medical University , Taichung, Taiwan , 3 Department of Otolaryngology, Chung Shan Medical University Hospital , Taichung, Taiwan , 4 Department of Otolaryngology - Head and Neck Surgery, National University Health System of Singapore , Singapore, Singapore , 5 Department of Otolaryngology, Chang-Gung Memorial Hospital, College of Medicine, Chang-Gung University , Linkou, Taiwan , 6 Department of Otolaryngology, National Taiwan University Hospital , Taipei, Taiwan , 7 Department of Psychiatry, Chung Shan Medical University Hospital , Taichung , Taiwan Objectives: (1) To evaluate the recognition of words, phonemes and lexical tones in audiovisual (AV) and auditory-only (AO) modes in Mandarin-speaking adults with cochlear implants (CIs); (2) to understand the effect of presentation levels on AV speech perception; (3) to learn the effect of hearing experience on AV speech perception. Methods: Thirteen deaf adults (age = 29.1613.5 years; 8 male, 5 female) who had used CIs for .6 months and 10 normalhearing (NH) adults participated in this study. Seven of them were prelingually deaf, and 6 postlingually deaf. The Mandarin Monosyllablic Word Recognition Test was used to assess recognition of words, phonemes and lexical tones in AV and AO conditions at 3 presentation levels: speech detection threshold (SDT), speech recognition threshold (SRT) and 10 dB SL (re:SRT). Results: The prelingual group had better phoneme recognition in the AV mode than in the AO mode at SDT and SRT (both p = 0.016), and so did the NH group at SDT (p = 0.004). Mode difference was not noted in the postlingual group. None of the groups had significantly different tone recognition in the 2 modes. The prelingual and postlingual groups had significantly better phoneme and tone recognition than the NH one at SDT in the AO mode (p = 0.016 and p = 0.002 for phonemes; p = 0.001 and p,0.001 for tones) but were outperformed by the NH group at 10 dB SL (re:SRT) in both modes (both p, 0.001 for phonemes; p,0.001 and p = 0.002 for tones). The recognition scores had a significant correlation with group with age and sex controlled (p,0.001). Conclusions: Visual input may help prelingually deaf implantees to recognize phonemes but may not augment Mandarin tone recognition. The effect of presentation level seems minimal on CI users' AV perception. This indicates special considerations in developing audiological assessment protocols and rehabilitation strategies for implantees who speak tonal languages. - Funding: This study was supported by research grants from the National Science Council of the Executive Yuan of Taiwan (NSC 100-2815-C-040-039-B). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. " SYL and CMW are co-first authors on this work. Verbal information transmitted to listeners via dual-modal (i.e., audiovisual, AV) stimulation is often thought to be more efficient than uni-modal (auditory-only, AO) stimulation [12]. Listeners, whether hearing-impaired or not, automatically watch talkers facial, lip and jaw movements, especially when auditory information was degraded, distorted or noise-masked [35]. In fact, optical cues also provide useful information when auditory stimuli are clear [6]. For example, English listeners distinguish threat from fret better by observing the location of teeth and tongue of the talkers. Cochlear implantation has been proven as an effective treatment to restore the hearing of patients with severe-toprofound sensorineural hearing loss [7]. It was reported that deaf patients with cochlear implants (CIs) made use of visual information to supplement the auditory stimulation they received from the CIs and in this way optimized their speech perception in daily communication (e.g., [810]). Their speech recognition was significantly better in the AV condition than in the AO condition [10]. Higher AV gain was observed in the CI users than in the NH controls who were tested in the simulated or noise-masked conditions as a result of CI users greater capability to integrate visual information with degraded auditory signals [11]. This AV integration ability in CI users was reported to correlate with the duration of the implant experience rather than the duration of deafness [12]. The neuroplasticity involving speechrelated network in our brain seems to allow a more efficient AV integration of speech after cochlear implantation [13]. Yet, although visual speech perceptual skills that developed during periods of deafness could have positive implications for later perception of auditory speech signals [14], visual take-over found in the auditory cortex in some CI users may also lead to incomplete reversal of this deafness-induced cortical reorganization [15]. Due to the inconsistent results from the past studies, the effect of auditory experience on AV perception in CI patients is still in question. However, although extensive research has been undertaken in non-tonal language users with CIs regarding AV speech processing, little information is available for the patients who speak tonal languages such as Mandarin Chinese. In Mandarin Chinese, each monosyllabic word comprises two lexical components: phoneme(s) and lexical tone. Words could be semantically different solely because of the lexical tone variations. Smith and Burnham [16] and Chen and Massaro [17] were the only ones we found who investigated the tone perception ability of Mandarin-speaking adults in the AV condition. They used normal-hearing participants and focused only on lexical tone discrimination. The authors found that visual information seemed less informative for Mandarin Chinese listeners than for non-tonal language users when discriminating Mandarin tones, meaning that native listeners of Mandarin Chinese depended more on auditory signals than visual ones to distinguish lexical tones. The presentation levels of speech signals could also affect speech recognition performance in listeners with normal hearing [18] and with CIs [1920]. In general, speech stimuli were more difficult to recognize at soft levels, and listeners often reported to take advantage of visual cues when auditory input was unreliable [35]. Thus, the degree of dependency on visual cues to distinguish spee (...truncated)