EEG based Emotion Recognition and Classification: a Review

Emotion plays a vital role in medical research and interpersonal communication. Essentially feeling can be communicated verbally like discourse or non-verbally like outward appearance and physiological signals. A human emotion is complex physiological state which involves a physiological response, a person’s experience and behavioral change. EEG measures electric current that are generated due to neuronal activities in the human brain. This paper provides an overview of comparative study of various techniques of emotion recognition from EEG signals. Our analysis is based on extracted features and classification methods of emotion recognition. We intended that, this study will be useful for newly researchers those entering in the field of emotion recognition.


Introduction
Essentially feeling can be communicated verbally like discourse or non-verbally like outward appearance and physiological signals. Physiological signals are bio electrical signals that are control by the automatic nerves system [1,2]. The human computer interaction system (HCI) find lots of applications in biomedical engineering, neuromarketing, neuroscience and other areas of human life, which are affected by emotions. Because of increasing demand of HCI and automatic human emotion recognition. Currently emotional research focuses more in the diagnosis of depression, mental illness, and assists health care professionals to make accurate diagnosis [3][4][5]. Human emotion can be recognize through physiological signal like Electromyography (EMG), Electrocardiography(ECG), Gelvonic skin response and Electroencephalography(EEG) from all of these, EEG is more effective because it provide more accurate, non-invasive and convenient way of capturing brain signal. And also EEG is portable device [6,9]. Diagnosis of EEG signal varies with expertise. EEG recording require lot of time for manual inspection. And some time results may not be accurate because of artifact in the signals. So the processing an analysis of EEG signals can be done with the help of HCI technologies to get fast and accurate results. Using these technologies diagnosis of neuropsychiatric and neurological disorder such as Alzheimer, epilepsy and depressive disorder will be done [31]. Nowadays, researcher has shows extensive interest to make human emotion interaction with machine by proper brain computer interface [23].Literature survey covered recent methods used in physiological signal based emotion recognition; it will help for researcher working in this field. The remaining portion of this paper is organized as follows: Emotion is described in Section II. Characteristics and various rhythms of EEG signals is describes in section III. Section IV describes the various databases used and preprocessing of signals. Extracted features and various classification techniques have been describe in section Conclusion and future direction gives in section 2. Emotion A human emotion is complex physiological state which involves a physiological response, a person's experience and behavioral change. An emotion is necessary component of human and significantly affects human's day by day activities including interaction, communication and learning etc.There are two perspectives towards representation of emotion. The first category indicate basic emotions through natural selection. There are eight basic emotions; fear, anger, disgust, sadness, curiosity, surprise, joy and acceptance. In the second category emotions are mapped into the Arousal, Valence and dominance [18]. Wast research on techniques of emotion recognition have been done over the past years and classified in to three catagories:  Periphery physiological signals  Facial expressions and voice  Brain signal generated by central nervous system. Some people cannot truly reflect their emotions into their facial expressions, because of some special reasons and a patient who suffers facial paralysis cannot express their filling. To solve these problems researchers had proposed emotion recognition methods based on Electrophysiological (EEG) signals and compared with other traditional methods [2].

Electroencephalogram (EEG)
The cortex is divided into the frontal, temporal, parietal, and occipital lobes(see fig.1) [18]. The frontal lobe is responsible for the conscious thought. The temporal lobe is responsible for the senses of smell and sound; The parietal lobe is responsible for integrating sensory information from various senses and the manipulation of objects and the at last, occipital lobe is responsible for the sense of sight. EEG estimates voltage fluctuations because of ionic current flows through neurons of the brain. A adult EEG signal, measured from the scalp is of range 10-100µV. These signals are divided into specific ranges, namely the delta(1-4Hz), theta (4-7Hz), alpha(8-13Hz), beta(13-30Hz) and gamma(>30Hz) bands [18]. Delta waves are related with the unconscious brain, and occur during deep sleep. Theta waves are related with subconscious mind, and occur during sleeping and dreaming. Alpha waves are associated to relaxed mental state. Beta waves are an active state of mind. International 10/20 system (IS) (see fig.2) [18] are used for standard sets of locations of electrodes on the skull. The number 10 and 20 shows the distance between neibouring electrodes. The procedure of emotion recognition is as follows:  User is exposed to the stimulus are tested.  Changes in the voltage of human mind are recorded.  From the EEG signals artifacts are removed.
 After the removal of noise and artifact the signal is analyzed and the relevant features are extracted.
 Select appropriate classifier and train them from the training dataset and compute features and predict the signal.

Fig.3: Procedure of EEG emotion recognition system 4. Dataset and Data Preprocessing
There are number of datasets are available publically that can be used for emotion recognition. Researchers mostly used DEAP, SEED and AMIGOS datasets. a. DEAP [39] is widely used dataset in the EEG based emotion recognition. There are 32 subjects were participated in the Electrophysiological recording. They watch 40 one minute music videos. In the DEAP dataset total 32 channels were used to record. b. SEED[40] 15 volunteers were participated in which each one was watching a small clips having distinct emotions. In SEED dataset total 62 channels were used for EEG recording. c. AMIGOS [3] 40 volunteers were participated in which each one was watched a set of sixteen short videos. Each volunteers rated each video on the scale of two dimensional (valence and arousal). Raw signal collected from EEG have noise and artifacts. Firstly we should remove this noise and artifacts from the raw signal. Artifacts removal is one of the most important step in emotion recognition. Artifacts are generated due to internal factors such as eye blinking, heart rate, muscles contraction and other environment factors like electrode position cable and recorders. There are many methods used for artifact removal, like linear regression, empirical mode decomposition, principal and component analysis etc.

Feature Extraction and Classification
Feature Extraction-For the better performance of biomedical (EEG) signals feature extractions are required. The main aim of feature extraction is to determine most important informative set of features to increase accuracy of the classifiers. A suitable feature extraction method converts one of many signals into a feature vector. In the machine learning algorithm researchers believe that suitable feature extraction is the key to make an efficient predictive model. There are several types of features based on frequency domain, Time domain and frequency time domain for the EEG signal diagnosis and analysis [23]. Feature Classification-Machine learning is a subject of artificial intelligence. In the machine learning we train some network to build the predictive model. There are many supervised machine learning algorithm exist. There are various feature extraction methods are used by researchers to perform frequency domain, time domain and time frequency domain analysis of EEG signals. Fourier transform(FT), empirical mode decomposition(EMD), wavelet packet decomposition(WPD), wavelet transform, mRMR, etc are some of the feature extraction methods [31]. Extracted features are further classified using machine learning algorithms. Some of feature extraction techniques and classification techniques are discussed in this paper. In this section we have discuss machine learning techniques for emotion recognition. In[2] used an Ensemble Convolutional Neural Network (ECNN) model, which is used to automatically mine the correlation between multi-channel EEG signals and peripheral physiological signals to improve the emotion recognition accuracy. Author design five convolution networks and use global average pooling (GAP) layers instead of fully connected layers, and then the plurality voting strategy is adopted to establish these model. In [3] a deep belief-conditional random field (DBN-CRF) framework is used which is improved version of deep belief networks with glia chains (DBN-GC) and conditional random field. In this, feature vector sequence is extracted first from the EEG signals. Then, parallel DBN-GC models are utilized to obtain the feature sequence from EEG signals. And then conditional random field (CRF) model is used to generates the emotion state sequence. And at the last, K-nearest neighbor algorithm(KNN) is used to estimate the emotion state. DEAP, SEED & AMIGOS dataset is used in this. In [4] subject independent emotion recognition system using Discrete Wavelet Transform and MLP is used. This method is better than the earlier methods as it has used publically available dataset which contains data from 32 subjects from both the men and women. DEAP database is used and accuracy obtained is 58.50%. In [6] a feature extraction technique based on double tree complex wavelet transform (DTCWT) and machine learning algorithm is used. In this a Neuroscan device is used for 16 subjects with video stimuli. Then band pass filter is used to remove artifects. And at the last, support vector machine (SVM) is used to classify emotions: calm, happy, and sad. And obtained the classification accuracy of 90.61%. In [7] EMD method is utilized to decompose EEG signals with channels F3 and C4. A series of IMFs obtained by EMD are used to calculate SampEn values and to form feature vectors. These vectors are fed into SVM classifier for training and testing. The average accuracy of this method is 94.98% for DEAP dataset. In [11] a hierarchical bidirectional GRU model with attention mechanism (H-ATT-BGRU) is used for classification of emotion. The first layer of the model encodes the local correlation among the samples in an epoch, and the second layer encodes the temporal correlation among the EEG epochs in a sequence. The model uses attention mechanism at both sample and epoch levels. In [13] tunable-Q wavelet transform (TQWT) is used for the classification of various emotions of EEG signals. tunable-Q wavelet transform divides EEG signal into subbands and then time-domain features are extracted from these subbands. These features are used as an input to machine learning classifier for the classification of happy, fear, sad, and relax emotions. In [14] phase locking value (PLV) graph convolutional neural networks [P-GCNN] is used which is the improved model of PLV and GCNN. The PLV has the ability to segregate phase and amplitude component in the EEG signal and determine inter channel correlation information. In [16] a hybrid feature extraction method is used in Empirical Mode Decomposition (EMD) domain with combination of Sequence Backward Selection (SBS) for EEG emotion recognition. In this detail information of multi scale components of EEG signal was extracted and optimal features was selected for emotion classification. Model is tested on DEAP dataset, in which the Valance and Arousal dimension emotional states are classified by Knearest neighbor (KNN) and support vector machine (SVM). In [19] a feature extraction technique Teager-Kaiser Energy Operator (TKEO) is used along with k-nearest neighbor (KNN), neural network (NN) and Classification Tree (CT) classifiers for the emotion recognition from EEG signal. This study determines the performance and accuracy of emotion recognition which is further used for stress identification. In [20] dual tree complex wavelet transform (DT-CWT) was used to decomposed the EEG into five different sub bands from which different features were extracted using frequency, time and non linear analysis. In this deep simple recurrent units (SRU) network is adopted which is not only able to processing a sequence data but also has the ability to resolve the problem of long term dependencies occurs in normal recurrent neural network(RNN). Positive neutral and negative emotions were identified from EEG signals from emotion recognition system. In [23] a hybrid method of principal component analysis (PCA) and t-statistics is used for feature extraction. This spatial PCA was implemented to reduce signal dimensionality and select features based on the t-statistical. In [27] a GPSO algorithm for tuning the hyperparameters in the CNN model is used, and applies the optimized CNN model to the EEG emotion recognition. In [29] mRMR feature selection method is used after the preprocessing step of EEG signal to improve the accuracy of SVM emotion classifier on twodimension(Valence and Arousal) emotions model. In [30] Liquid State Machines (LSM) algorithm is method of feature extraction and used to recognize the emotion state of an individual from EEG dataset. In this author identified arousal, valence and liking emotional states from the available dataset. [35][36][37][38][39][40][41].

Conclusion and Future Direction
In the analysis of brain signal we use EEG signals, as it is non-invasiveness high temporal resolution and safe nature. In this study we have surveyed the various methods of emotion recognition from EEG signals. A comparative analysis of EEG signals have been done in which feature extraction and classification methods are discussed. Our main focus on classification techniques and extracted features. From the analysis, we concluded that each and every stage has its own role in EEG signals analysis. Each and every state play a very important role in pre-processing EEG signals and developing novel model. In pre-processing unwanted signals, noise and artifacts are removed from the raw EEG signals. In second state, feature extraction algorithms are used to represent high dimensional signal into discrete features. Then, if data suffers from overfitting problem, feature reduction techniques can be used to reduce the computational cost. And at last signals are classified with different machine learning algorithms and deep learning algorithms for emotion recognition. From the study of various research papers it is concluded that accuracy of deep learning algorithm is better than machine learning algorithms. In future, we will made efforts in more advanced algorithms for feature extractions and classification techniques from the publically available datasets.