Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition

Abstract

In electroencephalography (EEG)-based emotion recognition systems, the distribution between the training samples and the testing samples may be mismatched if they are sampled from different experimental sessions or subjects because of user fatigue, different electrode placements, varying impedances, etc. Therefore, it is difficult to directly classify the EEG patterns with a conventional classifier. The domain adaptation method, which is aimed at obtaining a common representation across training and test domains, is an effective method for reducing the distribution discrepancy. However, the existing domain adaptation strategies either employ a linear transformation or learn the nonlinearity mapping without a consistency constraint; they are not sufficiently powerful to obtain a similar distribution from highly non-stationary EEG signals. To address this problem, in this paper, a novel component, called the subspace alignment auto-encoder (SAAE), is proposed. Taking advantage of both nonlinear transformation and a consistency constraint, we combine an auto-encoder network and a subspace alignment solution in a unified framework. As a result, the source domain can be aligned with the target domain together with its class label, and any supervised method can be applied to the new source domain to train a classifier for classification in the target domain, as the aligned source domain follows a distribution similar to that of the target domain. We compared our SAAE method with six typical approaches using a public EEG dataset containing three affective states: positive, neutral, and negative. Subject-to-subject and session-to-session evaluations were performed. The subject-to-subject experimental results demonstrate that our component achieves a mean accuracy of 77.88% in comparison with a state-of-the-art method, TCA, which achieves 73.82% on average. In addition, the average classification accuracy of SAAE in the session-to-session evaluation for all the 15 subjects in a dataset is 81.81%, an improvement of up to 1.62% on average as compared to the best baseline TCA. The experimental results show the effectiveness of the proposed method relative to state-of-the-art methods. It can be concluded that SAAE is a useful and effective tool for decreasing domain discrepancy and reducing performance degradation across subjects and sessions in the EEG-based emotion recognition field.

Keywords: Auto-encoder; Domain adaptation; EEG; Emotion recognition; MMD.