基于残差卷积自注意力神经网络的铝电解过热度识别方法

doi:10.12068/j.issn.1005-3026.2023.01.002

摘要/Abstract

摘要： 过热度是反映铝电解槽当前生产效率的重要指标，由于过热度难以在线实时测量，本文提出一种基于残差卷积自注意力神经网络的过热度识别方法.针对铝电解生产过程数据为时间序列数据且具有多源异构特性，设计异构数据的同构表示方法.在此基础上建立残差卷积自注意力神经网络模型以提取同构时间序列数据的全局与局部特征.针对过热度数据标签少且类别分布不均匀问题，采用基于自动编码器的无监督预训练方法与加权交叉熵损失函数以提高过热度识别任务的性能.在基准数据集上进行仿真对比实验以验证本文所提方法的有效性，然后在只包含少量不平衡标签的铝电解过热度数据集上进行实验验证，结果表明本文构建的过热度识别模型相较与其他现有模型不仅提高了过热度识别准确率，而且在训练样本较少时保证了模型的泛化能力.

关键词: 过热度识别；多源异构；残差卷积自注意力机制；无监督预训练；铝电解过程

Abstract: Superheat is an important indicator to reflect the current production efficiency of aluminium electrolytic cells. Due to the difficulty of superheat online real-time measurement， this paper proposes a superheat identification method based on residual convolution self-attention neural network (RCSANN). As the production data in aluminium electrolysis process is time series data and featured with multi-source heterogeneous characteristics， the isomorphic representation method is designed for heterogeneous data. On this basis， the RCSANN superheat model is proposed to extract the global and local features of the isomorphic time series data. Aiming at the problem of few labels and uneven category distribution of superheat data， the unsupervised pre-training method based on auto-encoder and the weighted cross-entropy loss function are used to improve the performance of the superheat identification task. The validity of the proposed method is verified by simulation and comparison experiments on the benchmark dataset. Then， experiments are carried out on the dataset of superheat in aluminium electrolysis with only a few unbalanced labels. The results show that not only the accuracy of superheat identification is improved compared with other existing models， but also the generalization ability can be guaranteed under few training labeled-samples.

Key words: superheat identification; multi-source heterogeneous; residual convolutional self-attention mechanism; unsupervised pre-training; aluminium electrolysis process

中图分类号:

TP277

林清扬，陈晓方，谢永芳. 基于残差卷积自注意力神经网络的铝电解过热度识别方法[J]. 东北大学学报（自然科学版）, 2023, 44(1): 8-17.

LIN Qing-yang， CHEN Xiao-fang， XIE Yong-fang. An Superheat Identification Method in Aluminium Electrolysis Based on Residual Convolutional Self-Attention Neural Network[J]. Journal of Northeastern University(Natural Science), 2023, 44(1): 8-17.

参考文献

[1]桂卫华，岳伟超，谢永芳，等.铝电解生产智能优化制造研究综述［J］.自动化学报，2018，44(11):1957-1970.(Gui Wei-hua，Yue Wei-chao，Xie Yong-fang，et al.A review of intelligent optimal manufacturing for aluminum reduction production［J］.Acta Automatica Sinica，2018，44(11):1957-1970.)
[2]Li J，Liu Y X，Huang Y Z，et al.Bath temperature model for point-feeding aluminium reduction cells［J］.Transactions of Nonferrous Metals Society of China，1994，4(1):26-32.
[3]Drengstig T，Ljungquist D，Foss B A.On the AlF3 and temperature control of an aluminum electrolysis cell［J］.IEEE Transactions on Control Systems Technology，1998，6(2):157-171.
[4]郭英杰，胡峰，于洪，等.基于时间粒的铝电解过热度预测模型［J］.南京大学学报(自然科学)，2019，55(4):624-632.(Guo Ying-jie，Hu Feng，Yu Hong，et al.Prediction model of superheat in aluminum electrolysis based on time granularity［J］.Journal of Nanjing University(Natural Science)，2019，55(4):624-632.)
[5]Lei Y X，Karimi H R，Cen L H，et al.Processes soft modeling based on stacked autoencoders and wavelet extreme learning machine for aluminum plant-wide application［J］.Control Engineering Practice，2021，108:104706.
[6]Min M C，Chen X F，Xie Y F.Constrained voting extreme learning machine and its application［J］.Journal of Systems Engineering and Electronics，2021，32(1):209-219.
[7]Bostrom A，Bagnall A.Binary shapelet transform for multiclass time series classification［C］//International Conference on Big Data Analytics and Knowledge Discovery.Berlin:Springer，2015:257-269.
[8]Wan W X，Chen X F，Gui W H，et al.A novel shapelet transformation method for classification of multivariate time series with dynamic discriminative subsequence and application in anode current signals［J］.Journal of Central South University，2020，27(1):114-131.
[9]Kate R J.Using dynamic time warping distances as features for improved time series classification［J］.Data Mining and Knowledge Discovery，2016，30(2):283-312.
[10]Lines J，Bagnall A.Time series classification with ensembles of elastic distance measures［J］.Data Mining and Knowledge Discovery，2015，29(3):565-592.
[11]Baydogan M G，Runger G，Tuv E.A bag-of-features framework to classify time series［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2013，35(11):2796-2802.
[12]Deng H，Runger G，Tuv E，et al.A time series forest for classification and feature extraction［J］.Information Sciences，2013，239:142-153.
[13]Bagnall A，Lines J，Hills J，et al.Time-series classification with COTE:the collective of transformation-based ensembles［J］.IEEE Transactions on Knowledge and Data Engineering，2015，27(9):2522-2535.
[14]Goodfellow I，Bengio Y，Courville A.Deep learning［M］.Cambridge:MIT Press，2016.
[15]Krizhevsky A，Sutskever I，Hinton G E.ImageNet classification with deep convolutional neural networks［J］.Communications of the ACM，2017，60(6):84-90.
[16]Tan M，Le Q.EfficientNet:rethinking model scaling for convolutional neural networks［C］//International Conference on Machine Learning.New York:PMLR，2019:6105-6114.
[17]Liu A T，Yang S，Chi P H，et al.Mockingjay:unsupervised speech representation learning with deep bidirectional transformer encoders［C］//ICASSP 2020—2020 IEEE International Conference on Acoustics，Speech and Signal Processing (ICASSP).Barcelona:IEEE，2020:6419-6423.
[18]宋欣，李奇，解婉君，等.YOLOv3-ADS:一种基于YOLOv3的深度学习目标检测压缩模型［J］.东北大学学报(自然科学版)，2021，42(5):609-615.(Song Xin，Li Qi，Xie Wan-jun，et al.YOLOv3-ADS:a compression model for deep learning object detection based on YOLOv3［J］.Journal of Northeastern University(Natural Science)，2021，42(5):609-615.)
[19]Dempster A，Petitjean F，Webb G I.ROCKET:exceptionally fast and accurate time series classification using random convolutional kernels［J］.Data Mining and Knowledge Discovery，2020，34(5):1454-1495.
[20]Qiao M，Yan S，Tang X，et al.Deep convolutional and LSTM recurrent neural networks for rolling bearing fault diagnosis under strong noises and variable loads［J］.IEEE Access，2020，8:66257-66269.
[21]Vaswani A，Shazeer N，Parmar N，et al.Attention is all you need［C］//Advances in Neural Information Processing Systems.New York:Curran Associates，2017:5998-6008.
[22]Lai S，Liu K，He S，et al.How to generate a good word embedding［J］.IEEE Intelligent Systems，2016，31(6):5-14.
[23]Xiong R，Yang Y，He D，et al.On layer normalization in the transformer architecture［C］//International Conference on Machine Learning.New York:PMLR，2020:10524-10533.
[24]Touvron H，Cord M，Douze M，et al.Training data-efficient image transformers & distillation through attention［C］//International Conference on Machine Learning.New York:PMLR，2021:10347-10357.
[25]Odena A，Dumoulin V，Olah C.Deconvolution and checkerboard artifacts［J］.Distill，2016，1(10):e3.
[26]Noh H，Hong S，Han B.Learning deconvolution network for semantic segmentation［C］//Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE Computer Society，2015:1520-1528.
[27]Bagnall A，Dau H A，Lines J，et al.The UEA multivariate time series classification archive，2018［J］.arXiv preprint arXiv:1811.00075，2018.
[28]Karim F，Majumdar S，Darabi H，et al.Multivariate LSTM-FCNs for time series classification［J］.Neural Networks，2019，116:237-245.
[29]Yan J， Mu L，Wang L，et al.Temporal convolutional networks for the advance prediction of ENSO［J］.Scientific Reports，2020，10(1):1-15.
[30]曹德芳，刘柏池.SVM财务欺诈识别模型［J］.东北大学学报(自然科学版)，2019，40(2):295-299，304.(Cao De-fang，Liu Bai-chi.SVM model for financial fraud detection［J］.Journal of Northeastern University(Natural Science)，2019，40(2):295-299，304.)［31］Chen T，Guestrin C.Xgboost:a scalable tree boosting system［C］//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:Assoc Computing Machinery，2016:785-794.