一种应用于脑电情感识别的迁移学习框架

doi:10.12068/j.issn.1005-3026.2023.07.001

摘要/Abstract

摘要： 脑电信号作为最能表征人体情绪的信号，正在成为情感识别的主流信号源.利用迁移学习可以克服生理信号源域、目标域间存在分布差异的问题.传统迁移学习由于缺少对样本、特征的选择过程，会对迁移效果产生负影响，致使识别率较低.为提升迁移效果，在样本、特征两个方面对迁移数据进行优化.介绍了一种基于Like值的实例筛选方法，以及基于粒子群优化的自动特征选择方法，并使用联合分布适配(joint distribution adaptation，JDA)，提出了一种应用于情感识别的迁移学习框架.在SEED数据集上构建了两个迁移任务并进行验证，结果表明，该框架可以有效提升迁移效果，提高跨域情感识别准确率.

关键词: 情感识别；迁移学习；样本筛选；特征选择；脑电信号

Abstract: As the most representative signal of human emotion， EEG signal is becoming the mainstream signal source of emotion recognition. Transfer learning can overcome the problem of distribution difference between physiological signal source domain and target domain. The traditional transfer learning will have a negative impact on the transfer effect due to the lack of the selection process of samples and features， resulting in a low recognition rate. In order to improve the migration effect， this paper optimizes the migrated data in two aspects: samples and characteristics. A case selection method based on like value and an automatic feature selection method based on particle swarm optimization are introduced. Using JDA， a transfer learning framework for emotion recognition is proposed. Finally， two transfer learning tasks are constructed on the SEED dataset for verification. The results show that the framework can effectively improve the transfer effect and improve the accuracy of cross domain emotion recognition.

Key words: emotion recognition; transfer learning; sample selection; feature selection; EEG signal

中图分类号:

TP181

赵海，王相，施瀚，陈佳伟. 一种应用于脑电情感识别的迁移学习框架[J]. 东北大学学报（自然科学版）, 2023, 44(7): 913-921.

ZHAO Hai， WANG Xiang， SHI Han， CHEN Jia-wei. A Transfer Learning Framework for EEG Emotion Recognition[J]. Journal of Northeastern University(Natural Science), 2023, 44(7): 913-921.

参考文献

[1]Fan X L，Yan Y，Wang X M，et al.Emotion recognition measurement based on physiological signals［C］//2020 13th International Symposium on Computational Intelligence and Design (ISCID).Hangzhou，2020:81-86.
[2]Li M，Lu B L.Emotion classification based on gamma-band EEG［C］//2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Minneapolis，2009:1223-1226.
[3]Fadi F，Elmachot A，Ali M，et al.A deep-learning model for subject-independent human emotion recognition using electrodermal activity sensors ［J］.Sensors，2019，19(7):1-14.
[4]Yin Y Q，Zheng X W，Hu B，et al.EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM［J］.Applied Soft Computing，2021，21(100):106954.
[5]龙明盛.迁移学习问题与方法研究［D］.北京:清华大学，2014.(Long Ming-sheng.Transfer learning:problems and methods［D］.Beijing:Tsinghua University，2014.)
[6]李永盛，何佳洲，赵国清，等.关于迁移学习中负迁移方向研究［J］.指挥控制与仿真，2020，42(4):28-33.(Li Yong-sheng，He Jia-zhou，Zhao Guo-qing，et al.The research on negative transfer in transfer learning［J］.Command Control & Simulation，2020，42(4):28-33.)
[7]Kumar V，Minz S.Multi-view ensemble learning:an optimal feature set partitioning for high-dimensional data classification［J］.Knowledge and Information Systems，2016，18(49):1-59.
[8]Feng K，Chaspari T.A review of generalizable transfer learning in automatic emotion recognition［J］.Frontiers in Computer Science，2020，14(2):1-9.
[9]Hazarika D，Poria S，Zimmermann R，et al.Conversational transfer learning for emotion recognition［J］.Information Fusion，2021，20(65):1-12.
[10]Zhang W，Song P.Transfer sparse discriminant subspace learning for cross-corpus speech emotion recognition［J］.IEEE/ACM Transactions on Audio Speech and Language Processing，2019，6(28):307-318.
[11]Zheng W L，Lu B L.Personalizing EEG-based affective models with transfer learning［C］ //Proceedings of the Twenty-fifth International Joint Conference on Artificial Intelligence.New York，2016:2732-2738.
[12]Zheng W L，Zhang Y Q，Zhu J Y，et al.Transfer components between subjects for EEG-based emotion recognition［C］// 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).Xi’an:IEEE，2015:917-922.
[13]Pan S J，Yang Q.A survey on transfer learning［J］.IEEE Transactions on Knowledge and Data Engineering，2010，22(10):1345-1359.
[14]Borgwardt K M，Gretton A，Rasch M J，et al.Integrating structured biological data by kernel maximum mean discrepancy［J］.Bioinformatics，2006，22(14):49-57.
[15]Pan S J，Tsang I W，Kwok J T，et al.Domain adaptation via transfer component analysis［J］.IEEE Transactions on Neural Networks，2011，22(2):199-210.
[16]Lan Z，Sourina O，Wang L，et al.Domain adaptation techniques for EEG-based emotion recognition:a comparative study on two public datasets［J］.IEEE Transactions on Cognitive and Developmental Systems，2018，11(1):85-94.
[17]Long M S，Wang J M，Ding G G，et al.Transfer feature learning with joint distribution adaptation［C］ //Proceedings of the IEEE International Conference on Computer Vision.Sydney，2013:2200-2207.
[18]Chai X，Wang Q，Zhao Y，et al.Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition［J］.Computers in Biology and Medicine，2016，46(79):205-214.
[19]Lin Y P，Jung T P.Improving EEG-based emotion classification using conditional transfer learning［J］.Frontiers in Human Neuroscience，2017，10(11):334.
[20]Chai X，Wang Q，Zhao Y，et al.A fast，efficient domain adaptation technique for cross-domain electroencephalography (EEG)-based emotion recognition ［J］.Sensors，2017，17(5):1014.
[21]Lin Y P.Constructing a personalized cross-day EEG-based emotion-classification model using transfer learning［J］.IEEE Journal of Biomedical and Health Informatics，2019，24(5):1255-1264.
[22]Li J，Qiu S，Shen Y Y，et al.Multisource transfer learning for cross-subject EEG emotion recognition［J］.IEEE Transactions on Cybernetics，2019，50(7):3281-3293.
[23]Zhou J，Chu S，Li X，et al.An EEG emotion recognition method based on transfer learning and echo state network for HilCPS［J］.Microprocessors and Microsystems，2021，87(11):1-10.
[24]Long M S，Wang J M，Ding G G，et al.Transfer joint matching for unsupervised domain adaptation［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus，2014:1410-1417.
[25]Peng H C，Long F H，Chris D.Feature selection based on mutual information criteria of max-dependency，max-relevance，and min-redundancy［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2005，27(8):1226-1238.
[26]Sun S L，Shi H L.Bayesian multi-source domain adaptation［C］//2013 International Conference on Machine Learning and Cybernetics.Tianjin，2013:24-28.
[27]Zheng W L，Lu B L.Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks［J］.IEEE Transactions on Autonomous Mental Development，2015，7(3):162-175.
[28]Zheng W L，Liu W，Lu Y F，et al.EmotionMeter:a multimodal framework for recognizing human emotions［J］.IEEE Transactions on Cybernetics，2019，49(3):1110-1122.
[29]Wang J，Chen Y，Hao S，et al.Balanced distribution adaptation for transfer learning［C］// 2017 IEEE International Conference on Data Mining (ICDM).New Orleans，2017:1129-1134.
[30]Gong B，Shi Y，Sha F，et al.Geodesic flow kernel for unsupervised domain adaptation［C］//2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence，2012:2066-2073.［31］娄晓光，陈兰岚，宋振振.基于多源域迁移学习的脑电情感识别［J］.计算机工程与设计，2020，41(7):2011-2018.(Lou Xiao-guang，Chen Lan-lan，Song Zhen-zhen.EEG emotion recognition based on multi-source domain transfer learning［J］.Computer Engineering and Design，2020，41(7):2011-2018.)