Midfrontal theta oscillation encodes haptic delay

www.nature.com/scientificreports OPEN Midfrontal theta oscillation encodes haptic delay Haneen Alsuradi1, Wanjoo Park2 & Mohamad Eid2* Haptic technologies aim to simulate tactile or kinesthetic interactions with a physical or virtual environment in order to enhance user experience and/or performance. However, due to stringent communication and computational needs, the user experience is influenced by delayed haptic feedback. While delayed feedback is well understood in the visual and auditory modalities, little research has systematically examined the neural correlates associated with delayed haptic feedback. In this paper, we used electroencephalography (EEG) to study sensory and cognitive neural correlates caused by haptic delay during passive and active tasks performed using a haptic device and a computer screen. Results revealed that theta power oscillation was significantly higher at the midfrontal cortex under the presence of haptic delay. Sensory correlates represented by beta rebound were found to be similar in the passive task and different in the active task under the delayed and synchronous conditions. Additionally, the event related potential (ERP) P200 component is modulated under the haptic delay condition during the passive task. The P200 amplitude significantly reduced in the last 20% of trials during the passive task and in the absence of haptic delay. Results suggest that haptic delay could be associated with increased cognitive control processes including multi-sensory divided attention followed by conflict detection and resolution with an earlier detection during the active task. Additionally, haptic delay tends to generate greater perceptual attention that does not significantly decay across trials during the passive task. Many systems that involve human–machine interaction, such as teleoperation or virtual reality, are incorporating haptic information to enhance perception and manipulation of the e nvironment1. One perceptual attribute that provides an essential basis for haptic-visual integration is haptic delay due to stringent computational and communication needs associated with haptic d ata2. Haptic delay can be defined as the temporal difference between the actual haptic feedback and the expected one. Delayed haptic feedback can seriously disrupt many aspects of the interaction, such as the completion time of manipulation tasks3, quality of teleoperation4, and the perception of physical properties such as stiffness and f riction5. The perceptual consequences of haptic delay have received significant attention through psychophysical studies. Detection thresholds for haptic delays may vary substantially, between 20 and 200 ms, based on the application and the type of haptic interaction (force feedback versus tactile, discrete versus continuous force feedback, and active versus passive interaction)6. A haptic-visual study showed that a discrete haptic feedback is noticed as delayed if the delay exceeded 110 ms6. In a collaborative virtual environment where haptic information is bi-directionally communicated, haptic feedback delay could be perceived starting from around 50 ms7. It is also reported that humans do not perceive delays below 30 ms during continuous haptic i nteraction8. Understanding the experience of perceiving haptic delay is an essential consideration for haptic technology designers to optimize the realism of the haptic experience. Neurohaptics is an emerging field that strives to understand the complex neural representations provoked in response to touch s timulation9. Neural imaging techniques such as fMRI and EEG offer the potential to examine brain activities associated with haptic delay to provide objective, real-time assessment of the haptic d elay9,10. Compared to other neural imaging techniques such as fMRI, EEG is preferable due to the compatibility with electric devices, relatively low cost, and the ability to measure brain responses with high temporal r esolution9. Previous studies utilized EEG to examine brain correlates associated with unexpected or mismatched visual stimulation. For instance, an event-related potential (ERP) study found that a negative potential around 200 ms (N200) is pronounced after seeing a visual stimulus that was not e xpected11. Furthermore, in an object selection task in a virtual environment, it was found that the prediction error negativity component of the ERP signal was more pronounced when the user’s hand had unrealistic r epresentation12. 1 Tandon School of Engineering, New York University, New York City, NY 11201, USA. 2Engineering Division, New York University Abu Dhabi, Saadiyat Island, Abu Dhabi 129188, United Arab Emirates. *email: mohamad.eid@ nyu.edu Scientific Reports | (2021) 11:17074 | https://doi.org/10.1038/s41598-021-95631-1 1 Vol.:(0123456789) www.nature.com/scientificreports/ Converging EEG studies have unraveled the functional roles that the central and midfrontal areas, and in particular the theta frequency band, play in conflict analysis at the stimulus/response l evel13,14, at the semantic/ cognitive level15,16 and during cross-modal prediction error processing17. An early study found that theta oscillation at the midfrontal region is closely related to cognitive control processes that are needed to evaluate the stream of information from perceived stimuli and prepare the brain’s response a ccordingly18. These processes include multisensory divided attention19, conflict detection and resolution and selective suppression20. In multimodal interaction such as audiovisual, increased central theta power was observed following incongruent audiovisual stimuli compared with congruent audiovisual s timuli21. Studies on the haptic modality are scarce. A few studies reported that theta synchronization at the midfrontal cortex reflects conflict monitoring and resolution to visual stimuli involving a motor r esponse20,22 or an initiated motor movement23. Theta oscillation was reported to be more pronounced under incongruent cross-modal stimulation as compared to congruent stimulation under visuotactile matching paradigm using Braille stimulator and a computer s creen24. In a similar study, visuotactile congruency was tested using two LEDs (visual stimuli) and two vibrotactile motors (tactile stimuli) placed on the thumb and the forefinger; incongruent stimulation induced a significantly greater theta band activity during 300–500 ms after stimulation25. This paper focuses on examining neural correlates associated with delayed force feedback during active and passive tasks. A delay value of 220 ms has been selected for this experiment based on the previously mentioned literature6,26; an easily recognizable delay is needed to clearly identify the neural correlates. This decision is complimented by a pilot study that was conducted to make sure the delay is clearly perceivable by the majority of participants (beyond 90% recognition accuracy). ERP and event-related spectral (...truncated)