Emotion Analysis using Different Stimuli with EEG Signals in Emotional Space

NESciences, 2017, 2(2): 1-10 -RESEARCH ARTICLE- Emotion Analysis using Different Stimuli with EEG Signals in Emotional Space Yasar Dasdemir1*, Esen Yildirim2, Serdar Yildirim3 1 2 Iskenderun Technical University, Computer Engineering, Hatay, Turkey Adana Science and Technology University, Electric-Electronics Engineering, Adana, Turkey 3 Adana Science and Technology University, Computer Engineering, Adana, Turkey Abstract Automatic detection for human-machine interfaces of the emotional states of the people is one of the difficult tasks. EEG signals that are very difficult to control by the person are also used in emotion recognition tasks. In this study, emotion analysis and classification study were conducted by using EEG signals for different types of stimuli. The combination of the audio and video information has been shown to be more effective about the classification of positive/negative (high/low) emotion by using wavelet transform from EEG signals, and true positive rate of 81.6% was obtained in valence dimension. Information of audio was found to be more effective than the information of video at classification that is made in arousal dimension, and true positive rate of 73.7% was obtained when both stimuli of audio and audio+video are used. Four class classification performance has also been examined in the space of valence-arousal. Keywords: EEG, stimuli types, emotion, emotion space model, valence, arousal Article history: Received 07 March 2017, Accepted 05 June 2017, Available online 20 June 2017 * Corresponding Author: Yasar Dasdemir, e-mail: Natural and Engineering Sciences 2 Introduction Emotions have a key role in communication and they are required to understand human behavior. Emotion is among the topics of research of several disciplines such as neuroscience, psychology, and linguistics. Especially in the field of psychology, there are different approaches to emotion modeling (Gunes & Pantic, 2010). Categorical and dimensional approaches of these approaches are the most widely used models for labeling emotion in the studies about emotions recognition. Emotional state in categorical approach has been identified as the mood that is expressed as discrete. Although there is a wide range of defined emotion categories, anger, fear, surprise, happiness, sadness, disgust feeling classes proposed by Ekman et al. (Ekman, 1999) have been accepted as universal emotions. In the dimensional approach, emotions are not limited to a small number of discrete emotion classes, instead of this, it is defined as points in a multi-dimensional space. In this approach, diversity of emotions is considered in 3 dimensions. These dimensions are valence, arousal, and dominance. When valence determines the range from negative to positive of emotions, arousal denotes the range from calmness to exciting of emotions. The dominance dimension is associated with the control of the environment with the feeling. Dimensional approach is used for the representation and the labeling of the emotions, and studies were performed about valence and arousal dimensions that are widely used in the literature. Effects of emotion analysis and classification of the type of stimuli were also investigated using three different stimuli types as Audio (sound), Video (visual) and Audio + Video (both sound and visual) to reveal the feelings of the participants in the database created under this study. Literature Review Many studies have used accepted sense stimulus (movies, pictures, sound, smell) to elicit some emotions. For example, IAPS (International Affective Picture System) (Lang, Bradley, & Cuthbert, 1997) is a dataset that is frequently used in studies on emotion. 716 natural colored images like landscapes, people, and objects which are taken by professional photographers are found inside it, and it is widely used with EEG for emotion recognition studies (Yohanes vd., 2012), (Xu & Plataniotis, 2012), (Ramirez & Vamvakousis, 2012). There are also studies that use movie clips for stimulation of feelings (Rottenberg, Ray, & Gross, 2007). 3-dimensional valence-arousal-dominance or pleasure-arousal-dominances fields are used in cognitive theory. These fields are frequently used in emotion processing studies (Yoon & Chung, 2011), (Liu & Sourina, 2012), (Ahmed, 2014), (Al-galal, Taha, & Wahab, 2015), (Chen & Han, 2015), (Atkinson & Campos, 2016), (Huang vd., 2016). Ramirez and Vamvakousis have proposed a new method for emotion recognition by using Emotiv EPOC device (Ramirez & Vamvakousis, 2012). They have used some sounds from IADS (International Affective Digitized Sounds) (Bradley & Lang, 1999) sound library which consisted of labeled emotional sounds as stimulators. They tried to classify high/low arousal and high/low valence emotions with various machine learning techniques, using the valence and arousal plane. They did EEG measurements from AF3, AF4, F3 and F4 channels in the section prefrontal cortex. They have used beta/alpha ratio as an arousal status indicator. They have achieved classification performances which are 77.82% for high/low arousal and 80.11% for positive/negative valence. The performance values obtained by SVM with radial basis function kernel classifier. Natural and Engineering Sciences 3 The achievement was obtained 64.84% and 61.17% ratio about the classification of high/low arousal and valence of emotions, respectively. The experiment is made with bispectrum analysis of EEG signals by using valence-arousal emotion space (Kumar, Khaund, & Hazarika, 2016). Emotion recognition studies have been conducted on DEAP dataset (Koelstra vd., 2012) obtained by collecting video stimuli and EEG records using wavelet-based attributes (Srinivas, Rama, & Rao, 2016) and emotion characteristics emerged in the delta, theta, alpha, beta and gamma band are classified with MLP (Multi-Layer Perceptron) and RBF (Radial Basis Function). The best results in the frequency field have been obtained with 54.54% of the RBF and 63.63% of the MLP. Emotion analysis and classification studies are made in this study using EEG signals, and performance of the system was tested in the 4-quadrant emotion of field. Necessity database has been established within this scope to perform studies. As the database is created, emotion targeted using only audio, only video, and audio+video stimuli have been triggered. Short videos obtained from domestic and foreign films were used as stimuli within this scope. Each video is selected as 60-second segments, and 3 different versions of the video (only audio, only video, and both audio and video) are used as stimuli. Background EEG signals and facial expressions collected from 25 volunteers are used in this study. In the database (Duygu-DB) created in the study, EEG signals were recorded using an Emotive EPOC wireless EEG device and the facial expression videos were recorded using a smartphone with 1920x1080 HD 30 fps resolution. The videos are not used in this (...truncated)