Heart Anomaly Detection Algorithm Based on Multimodal Feature Engineering and TSNet

doi:10.12068/j.issn.1005-3026.2024.10.004

Abstract

Abstract:

Electrocardiogram （ECG） and Phonocardiogram （PCG） are commonly used diagrams in heart diseases diagnosis. While， using them alone for heart disease diagnosis is not effective. Based on multimodal feature engineering， after segmentation and normalization preprocess of the dataset， Gramiam angle fields （GAF） are used for time?series data reconstruction to form an image model. Additionally， a two?stream self?fusion network （TSNet） suitable for this image model is proposed， which replaces the bottom?layer convolution operations with a two?stream self?fusion （TS） module to better integrate the heterogeneous information of ECG and PCG. Tested on the PhysioNet Challenge 2016 a dataset， the proposed algorithm achieves best values of accuracy， F1 score， precision， and recall at 95.3%， 95.4%， 96.2%， and 99.4%， respectively. Compared to other multimodal convolutional neural network algorithms for ECG and PCG， it shows higher accuracy.

Key words: electrocardiogram （ECG）, phonocardiogram （PCG）, multimodal feature engineering, Gramian angle fields （GAF）, two?stream self?fusion network （TSNet）

CLC Number:

TP 391.7

Ji-hong LIU, Wei XUE, Chao XU. Heart Anomaly Detection Algorithm Based on Multimodal Feature Engineering and TSNet[J]. Journal of Northeastern University(Natural Science), 2024, 45(10): 1394-1400.

Figures/Tables 13

Fig.1 Framework of the algorithm

Fig.2 Schematic diagram of electrocardiosignal

Fig.3 Schematic diagram of heart sound signal

Fig.4 Two?dimensional spectrum of electrocardiosignal

Fig.5 Two?dimensional spectrum of heart sound signal

Fig.6 TS module

Table 1 Experimental parameters

模型配置	实验参数
骨干网络	ResNet-18
图像尺寸/像素	200×200
迭代次数	50
批量大小/个	128
通道数/个	10，20，64

Table 2 Evaluation indicators and expressions

评估指标	表达式
准确率	$T P + T N T P + T N + F P + F N × 100 %$
召回率	$T P T P + F N × 100 %$
精确率	$T P T P + F P × 100 %$
F1分数	$2 × 精确率 × 召回率精确率 + 召回率$

Table 2 Evaluation indicators and expressions

评估指标	表达式
准确率	$T P + T N T P + T N + F P + F N × 100 %$
召回率	$T P T P + F N × 100 %$
精确率	$T P T P + F P × 100 %$
F1分数	$2 × 精确率 × 召回率精确率 + 召回率$

Table 3 Comparative experiment

模型	准确率	召回率	精确率	F1分数
GAF+ResNet	92.6	99.9	96.3	92.8
MTF+ResNet	90.9	99.7	100.0	90.9
RP+ResNet	89.6	99.8	88.4	90.1

Fig.7 Loss function curves of comparative experiment

Table 4 Results of multiple indicator evaluation （mean ± standard deviation）

模型	准确率	召回率	精确率	F1分数
ResNet	90.36±2.27	96.16±2.55	93.04±1.71	90.44±2.42
TSNet	92.21±0.95	97.84±1.90	94.58±1.28	92.24±1.04
SEnet	90.97±2.78	96.38±3.45	92.40±2.38	91.08±2.87
CBAM	90.75±2.38	94.50±3.49	92.90±2.04	90.74±2.49
TS+CBAM	89.76±1.72	94.36±3.37	92.90±1.67	89.96±1.67

Fig.8 Network loss function curves

Fig.9 Diagrams of network comparisons experiments

References 23

1	Sookram C， Munodawafa D， Phori P M，et al.Who’s supported interventions on salt intake reduction in the sub‑Saharan Africa region［J］.Cardiovascular Diagnosis and Therapy，2015，5（3）：186-190.
2	Hettiarachchi R， Haputhanthri U， Herath K，et al.A novel transfer learning‑based approach for screening pre‑existing heart diseases using synchronized ECG signals and heart sounds［C］//2021 IEEE International Symposium on Circuits and Systems （ISCAS）.Daegu，2021：1-5.
3	Rautaharju P M.Eyewitness to history：landmarks in the development of computerized electrocardiography［J］.Journal of Electrocardiology，2016，49（1）：1-6.
4	Leatham A.Auscultation of the heart and phonocardiography［J］.Postgraduate Medical Journal，1971，47（550）：582.
5	Martins M， Gomes P， Oliveira C，et al.Design and evaluation of a diaphragm for electrocardiography in electronic stethoscopes［J］.IEEE Transactions on Biomedical Engineering，2020，67（2）：391-398.
6	Chakir F， Jilbab A， Nacir C，et al.Recognition of cardiac abnormalities from synchronized ECG and PCG signals［J］.Physical and Engineering Sciences in Medicine，2020，43（2）：673-677.
7	Li J H， Ke L， Du Q，et al.Multi‑modal cardiac function signals classification algorithm based on improved D-S evidence theory［J］.Biomedical Signal Processing and Control，2022，71：103078.
8	Singh S A， Singh S A， Devi N D，et al.Heart abnormality classification using PCG and ECG recordings［J］.Computación y Sistemas，2021，25（2）：381-391.
9	Huang J S， Chen B Q， Yao B，et al.ECG arrhythmia classification using STFT‑based spectrogram and convolutional neural network［J］.IEEE Access，2019，7：92871-92880.
10	Al Rahhal M M， Bazi Y， Al Zuair M，et al.Convolutional neural networks for electrocardiogram classification［J］.Journal of Medical and Biological Engineering，2018，38（6）：1014-1025.
11	Potes C， Parvaneh S， Rahman A，et al.Ensemble of feature‑based and deep learning‑based classifiers for detection of abnormal heart sounds［J］.Computing in Cardiology，2016，43：621-624.
12	TSchannen M， Kramer T， Marti G，et al.Heart sound classification using deep structured features［C］//2016 Computing in Cardiology Conference （CinC）.Vancouver：IEEE，2016：565-568.
13	Wibawa M S， Maysanjaya I M D， Novianti N K D P，et al.Abnormal heart rhythm detection based on spectrogram of heart sound using convolutional neural network［C］//The 6th International Conference on Cyber and IT Service Management （CITSM 2018）.Parapat：IEEE，2018：1-4.
14	Wu J M T， Tsai M H， Huang Y Z，et al.Applying an ensemble convolutional neural network with Savitzky‑Golay filter to construct a phonocardiogram prediction model［J］.Applied Soft Computing，2019，78：29-40.
15	Noman F， Ting C M， Salleh S H，et al.Short‑segment heart sound classification using an ensemble of deep convolutional neural networks［C］//International Conference on Acoustics，Speech and Signal Processing （ICASSP）.Brighton，2019：1318-1322.
16	Li H， Wang X P， Liu C C，et al.Integrating multi‑domain deep features of electrocardiogram and phonocardiogram for coronary artery disease detection［J］.Computers in Biology and Medicine，2021，138：104914.
17	Wang Z G， Oates T.Imaging time‑series to improve classification and imputation［EB/OL］.（2015-06-01）［2023-03-21］..
18	Clifford G D， Liu C Y， Moody B，et al.Classification of normal/abnormal heart sound recordings：the PhysioNet/computing in cardiology challenge 2016［C］//2016 Computing in Cardiology Conference （CinC）.Vancouver：IEEE，2016：609-612.
19	Eckmann J P， Kamphorst S O， Ruelle D.Recurrence plots of dynamical systems［J］.World Scientific Series on Nonlinear Science Series A，1995，16：441-445.
20	Liu L， Wang Z G.Encoding temporal Markov dynamics in graph for time series visualization［EB/OL］.（2016-10-24）［2023-03-21］. .
21	He K M， Zhang X Y， Ren S Q，et al.Deep residual learning for image recognition［C］//2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.Las Vegas，2016：770-778.
22	Hu J， Shen L， Sun G.Squeeze‑and‑excitation networks［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City，2018：7132-7141.
23	Woo S， Park J， Lee J Y，et al.CBAM：convolutional block attention module［C］//Computer Vision-ECCV2018.Munich，2018：3-19.