A Transfer Learning Framework for EEG Emotion Recognition

doi:10.12068/j.issn.1005-3026.2023.07.001

Abstract

Abstract: As the most representative signal of human emotion， EEG signal is becoming the mainstream signal source of emotion recognition. Transfer learning can overcome the problem of distribution difference between physiological signal source domain and target domain. The traditional transfer learning will have a negative impact on the transfer effect due to the lack of the selection process of samples and features， resulting in a low recognition rate. In order to improve the migration effect， this paper optimizes the migrated data in two aspects: samples and characteristics. A case selection method based on like value and an automatic feature selection method based on particle swarm optimization are introduced. Using JDA， a transfer learning framework for emotion recognition is proposed. Finally， two transfer learning tasks are constructed on the SEED dataset for verification. The results show that the framework can effectively improve the transfer effect and improve the accuracy of cross domain emotion recognition.

Key words: emotion recognition; transfer learning; sample selection; feature selection; EEG signal

CLC Number:

TP181

ZHAO Hai， WANG Xiang， SHI Han， CHEN Jia-wei. A Transfer Learning Framework for EEG Emotion Recognition[J]. Journal of Northeastern University(Natural Science), 2023, 44(7): 913-921.

References

[1]Fan X L，Yan Y，Wang X M，et al.Emotion recognition measurement based on physiological signals［C］//2020 13th International Symposium on Computational Intelligence and Design (ISCID).Hangzhou，2020:81-86.
[2]Li M，Lu B L.Emotion classification based on gamma-band EEG［C］//2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Minneapolis，2009:1223-1226.
[3]Fadi F，Elmachot A，Ali M，et al.A deep-learning model for subject-independent human emotion recognition using electrodermal activity sensors ［J］.Sensors，2019，19(7):1-14.
[4]Yin Y Q，Zheng X W，Hu B，et al.EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM［J］.Applied Soft Computing，2021，21(100):106954.
[5]龙明盛.迁移学习问题与方法研究［D］.北京:清华大学，2014.(Long Ming-sheng.Transfer learning:problems and methods［D］.Beijing:Tsinghua University，2014.)
[6]李永盛，何佳洲，赵国清，等.关于迁移学习中负迁移方向研究［J］.指挥控制与仿真，2020，42(4):28-33.(Li Yong-sheng，He Jia-zhou，Zhao Guo-qing，et al.The research on negative transfer in transfer learning［J］.Command Control & Simulation，2020，42(4):28-33.)
[7]Kumar V，Minz S.Multi-view ensemble learning:an optimal feature set partitioning for high-dimensional data classification［J］.Knowledge and Information Systems，2016，18(49):1-59.
[8]Feng K，Chaspari T.A review of generalizable transfer learning in automatic emotion recognition［J］.Frontiers in Computer Science，2020，14(2):1-9.
[9]Hazarika D，Poria S，Zimmermann R，et al.Conversational transfer learning for emotion recognition［J］.Information Fusion，2021，20(65):1-12.
[10]Zhang W，Song P.Transfer sparse discriminant subspace learning for cross-corpus speech emotion recognition［J］.IEEE/ACM Transactions on Audio Speech and Language Processing，2019，6(28):307-318.
[11]Zheng W L，Lu B L.Personalizing EEG-based affective models with transfer learning［C］ //Proceedings of the Twenty-fifth International Joint Conference on Artificial Intelligence.New York，2016:2732-2738.
[12]Zheng W L，Zhang Y Q，Zhu J Y，et al.Transfer components between subjects for EEG-based emotion recognition［C］// 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).Xi’an:IEEE，2015:917-922.
[13]Pan S J，Yang Q.A survey on transfer learning［J］.IEEE Transactions on Knowledge and Data Engineering，2010，22(10):1345-1359.
[14]Borgwardt K M，Gretton A，Rasch M J，et al.Integrating structured biological data by kernel maximum mean discrepancy［J］.Bioinformatics，2006，22(14):49-57.
[15]Pan S J，Tsang I W，Kwok J T，et al.Domain adaptation via transfer component analysis［J］.IEEE Transactions on Neural Networks，2011，22(2):199-210.
[16]Lan Z，Sourina O，Wang L，et al.Domain adaptation techniques for EEG-based emotion recognition:a comparative study on two public datasets［J］.IEEE Transactions on Cognitive and Developmental Systems，2018，11(1):85-94.
[17]Long M S，Wang J M，Ding G G，et al.Transfer feature learning with joint distribution adaptation［C］ //Proceedings of the IEEE International Conference on Computer Vision.Sydney，2013:2200-2207.
[18]Chai X，Wang Q，Zhao Y，et al.Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition［J］.Computers in Biology and Medicine，2016，46(79):205-214.
[19]Lin Y P，Jung T P.Improving EEG-based emotion classification using conditional transfer learning［J］.Frontiers in Human Neuroscience，2017，10(11):334.
[20]Chai X，Wang Q，Zhao Y，et al.A fast，efficient domain adaptation technique for cross-domain electroencephalography (EEG)-based emotion recognition ［J］.Sensors，2017，17(5):1014.
[21]Lin Y P.Constructing a personalized cross-day EEG-based emotion-classification model using transfer learning［J］.IEEE Journal of Biomedical and Health Informatics，2019，24(5):1255-1264.
[22]Li J，Qiu S，Shen Y Y，et al.Multisource transfer learning for cross-subject EEG emotion recognition［J］.IEEE Transactions on Cybernetics，2019，50(7):3281-3293.
[23]Zhou J，Chu S，Li X，et al.An EEG emotion recognition method based on transfer learning and echo state network for HilCPS［J］.Microprocessors and Microsystems，2021，87(11):1-10.
[24]Long M S，Wang J M，Ding G G，et al.Transfer joint matching for unsupervised domain adaptation［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus，2014:1410-1417.
[25]Peng H C，Long F H，Chris D.Feature selection based on mutual information criteria of max-dependency，max-relevance，and min-redundancy［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2005，27(8):1226-1238.
[26]Sun S L，Shi H L.Bayesian multi-source domain adaptation［C］//2013 International Conference on Machine Learning and Cybernetics.Tianjin，2013:24-28.
[27]Zheng W L，Lu B L.Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks［J］.IEEE Transactions on Autonomous Mental Development，2015，7(3):162-175.
[28]Zheng W L，Liu W，Lu Y F，et al.EmotionMeter:a multimodal framework for recognizing human emotions［J］.IEEE Transactions on Cybernetics，2019，49(3):1110-1122.
[29]Wang J，Chen Y，Hao S，et al.Balanced distribution adaptation for transfer learning［C］// 2017 IEEE International Conference on Data Mining (ICDM).New Orleans，2017:1129-1134.
[30]Gong B，Shi Y，Sha F，et al.Geodesic flow kernel for unsupervised domain adaptation［C］//2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence，2012:2066-2073.［31］娄晓光，陈兰岚，宋振振.基于多源域迁移学习的脑电情感识别［J］.计算机工程与设计，2020，41(7):2011-2018.(Lou Xiao-guang，Chen Lan-lan，Song Zhen-zhen.EEG emotion recognition based on multi-source domain transfer learning［J］.Computer Engineering and Design，2020，41(7):2011-2018.)