Eyes-closed hybrid brain-computer interface employing frontal brain activation

RESEARCH ARTICLE Eyes-closed hybrid brain-computer interface employing frontal brain activation Jaeyoung Shin1, Klaus-Robert Müller2,3,4☯*, Han-Jeong Hwang5☯* 1 Department of Biomedical Engineering, Hanyang University, Seoul, Korea, 2 Machine Learning Group, Berlin Institute of Technology (TU Berlin), Berlin, Germany, 3 Department of Brain and Cognitive Engineering, Korea University, Seoul, Korea, 4 Max Planck Institute for Informatics, Stuhlsatzenhausweg, Saarbrücken, Germany, 5 Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Kumi, Korea a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Shin J, Müller K-R, Hwang H-J (2018) Eyes-closed hybrid brain-computer interface employing frontal brain activation. PLoS ONE 13 (5): e0196359. https://doi.org/10.1371/journal. pone.0196359 Editor: Xu Lei, School of Psychology, CHINA Received: October 7, 2017 Accepted: April 11, 2018 Published: May 7, 2018 Copyright: © 2018 Shin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All data used in this study are fully available without any restriction from figshare: https://doi.org/10.6084/m9.figshare. 5900842.v1. Funding: This work was supported by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00451) and by National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. 2017R1A6A3A01003543) and the National Research Foundation of Korea (NRF) grant funded ☯ These authors contributed equally to this work. * (KRM); (HJH) Abstract Brain-computer interfaces (BCIs) have been studied extensively in order to establish a nonmuscular communication channel mainly for patients with impaired motor functions. However, many limitations remain for BCIs in clinical use. In this study, we propose a hybrid BCI that is based on only frontal brain areas and can be operated in an eyes-closed state for end users with impaired motor and declining visual functions. In our experiment, electroencephalography (EEG) and near-infrared spectroscopy (NIRS) were simultaneously measured while 12 participants performed mental arithmetic (MA) and remained relaxed (baseline state: BL). To evaluate the feasibility of the hybrid BCI, we classified MA- from BL-related brain activation. We then compared classification accuracies using two unimodal BCIs (EEG and NIRS) and the hybrid BCI in an offline mode. The classification accuracy of the hybrid BCI (83.9 ± 10.3%) was shown to be significantly higher than those of unimodal EEGbased (77.3 ± 15.9%) and NIRS-based BCI (75.9 ± 6.3%). The analytical results confirmed performance improvement with the hybrid BCI, particularly for only frontal brain areas. Our study shows that an eyes-closed hybrid BCI approach based on frontal areas could be applied to neurodegenerative patients who lost their motor functions, including oculomotor functions. Introduction Brain-computer interfaces (BCIs) have in the past enabled patients to control external devices directly without the help of muscular movements [1–5]. Thus, many research groups have explored BCI technology and considerably improved the performance of BCI systems [6–12]. Various BCI paradigms based on electroencephalography (EEG) have been introduced to implement BCIs for physically challenged patients. These paradigms include motor imagery [13–15], P300 [16–18], steady-state visual evoked potential (SSVEP) [19], and others. However, these paradigms have limitations with respect to severely motor-impaired patients such as late-stage amyotrophic lateral sclerosis (ALS). For example, some of them cannot generate PLOS ONE | https://doi.org/10.1371/journal.pone.0196359 May 7, 2018 1 / 16 Eyes-closed hybrid brain-computer interface by the Korea government (Ministry of Science, ICT & Future Planning) (No. 2017R1C1B5017909). Competing interests: The authors have declared that no competing interests exist. reliable sensorimotor rhythms for motor-imagery-based BCI [20–22]. Also, as BCIs based on exogenous paradigms such as conventional visual P300 and SSVEP generally require moderate oculomotor functions, these exogenous paradigms cannot be fully exploited for those with oculomotor dysfunctions that are often presented in late-stage ALS or completely locked-in state (CLIS) patients [23]. To overcome these constraints, previous studies introduced BCIs based on cognitive tasks instead of motor imagery tasks and validated its feasibility with both healthy subjects and ALS patients [20–22]. Also, an eyes-closed (EC) SSVEP-based BCI was recently introduced [19], which validated the feasibility of an SSVEP-based BCI under EC conditions for healthy participants and for an (ALS) patient with partially impaired oculomotor functions. A more recent study introduced a novel EC BCI paradigm based on visual P300 and demonstrated its effectiveness [23]. In our previous study, we first proposed an EC BCI system using a representative endogenous BCI paradigm, namely, mental arithmetic (MA), to check whether an endogenous BCI paradigm can also be used in an EC condition [24]. In [24], we used near-infrared spectroscopy (NIRS) signals of prefrontal cortex (PFC) areas, which represents a promising alternative to EEG for BCI research, as its sensitivity to physiological artifacts (e.g., electrooculogram (EOG)) is limited. It has been well documented that EEG signals significantly change under an EC state (e.g., α-rhythm). However, because PFC hemodynamic changes are irrelevant to an EC condition [25], the feasibility of the EC NIRS-BCI could be successfully verified. Moreover, because PFC is essentially below the hair-free region of the skull, we could shorten preparation time and speed up the experiment. In fact, many NIRS-BCI studies have focused on PFC hemodynamic changes, as PFC areas are free from one of the critical drawbacks of a NIRSbased BCI: signal amplitude attenuation as a result of dense, long, and dark hairs blocking light penetration to the scalp [26–32]. Although we successfully demonstrated the feasibility of an EC NIRS-BCI system, the classification accuracy was relatively low compared to those reported in standard EEG-BCI studies [19, 23]. One possible means of improving classification accuracy is to use a hybrid approach that combines two brain-imaging modalities (e.g., EEG and NIRS [33–36]). That the hybrid BCI can increase the reliability of BCI systems in terms of performance has already been demonstrated [37, 38]. In particular, the performance of BCI systems could be enhanced by integrating two kinds of BCI systems or using complementary information of brain activations measured with different modalities. As an example of the forme (...truncated)