Surface Defect Detection of Riveting Holes Based on Improved YOLOv8

doi:10.12068/j.issn.1005-3026.2024.11.010

Abstract

Abstract:

The surface quality of riveting holes on aircraft skin， tail and other components is crucial to the overall assembly performance of an aircraft. Currently， most riveting hole defect detection relies on the traditional manual methods， which are prone to missing detection. An improved detection method based on YOLOv8 for surface defect detection of riveting holes was proposed. The conventional convolution was replaced by deformable convolution to solve the problem of the fixed receptive field shape in feature extraction. The SimAM attention mechanism was embedded in order to enhance the recognition ability of the network under low contrast between the background and targets. The CIoU loss function was replaced by the WIoU bounding box regression loss function to reduce the impact of low?quality images during model training and improve the robustness and generalization ability of the model. To verify the performance of the improved model， 6061 aluminum alloy plates with riveting holes were used as a substitute for aircraft skin in the detection process. Experimental results demonstrated that the improved model achieved mAP_0.5 and accuracy of 0.918 and 0.920 on the riveting hole test set， which represents an improvement of 24.1% and 25.3% compared to the original model.

Key words: YOLOv8, riveting hole, defect detection, deformable convolution, attention mechanism

CLC Number:

TP 391

Bo HAO, Xin-yan XU, Yu-xin ZHAO, Jun-wei YAN. Surface Defect Detection of Riveting Holes Based on Improved YOLOv8[J]. Journal of Northeastern University(Natural Science), 2024, 45(11): 1595-1603.

Figures/Tables 16

Table 1 Parameters of the CCD camera

型号	MV-EM510M
分辨率	2 456×2 058
外形尺寸/mm	50×50×48
最大帧率/（帧 $? s - 1$ ）	15
曝光方式	帧曝光
功耗/W	2.5

Table 1 Parameters of the CCD camera

型号	MV-EM510M
分辨率	2 456×2 058
外形尺寸/mm	50×50×48
最大帧率/（帧 $? s - 1$ ）	15
曝光方式	帧曝光
功耗/W	2.5

Table 2 Parameters of the telecentric lens

型号	BT-R23C144
放大倍率	0.061
视场尺寸/mm	144.3×108.2
光圈	8
远心度/（°）	<0.05
景深/mm	90
畸变/%	<0.08

Fig.1 Image acquisition equipment and riveting holes

Fig.2 Network structure of the improved YOLOv8

Fig.3 Calculation flowchart of deformable convolution

Fig.4 Comparison of attention mechanisms

Fig.5 Types and quantities of the data set samples

Fig.6 Target distribution and size of the data set

Fig.7 Loss value comparison of different models

Fig.8 mAP value comparison of different models

Fig.9 Confusion matrix of the two models

Table 3 Test accuracy parameters of the two models

模型	缺陷	P	R	F1值	AP_0.5
本文模型	裂纹	0.919	0.938	0.928	0.912
	凹陷	0.900	0.948	0.923	0.963
	毛刺	0.856	0.922	0.888	0.929
	划痕	0.890	0.759	0.819	0.869
YOLOv8 模型	裂纹	0.791	0.812	0.801	0.723
	凹陷	0.736	0.667	0.700	0.719
	毛刺	0.870	0.629	0.730	0.812
	划痕	0.575	0.487	0.527	0.453

Table 4 Results of ablation experiments

组别	模型	P	R	mAP_0.5	mAP_0.5∶0.95	帧率/（帧· $s - 1$ ）
1	YOLOv8	0.743	0.649	0.677	0.367	243.90
2	YOLOv8+WIoU	0.791	0.650	0.754	0.458	204.08
3	YOLOv8+WIoU+SimAM	0.790	0.681	0.806	0.476	178.57
4	YOLOv8+WIoU+SimAM+DCN	0.891	0.892	0.918	0.524	147.06

Table 4 Results of ablation experiments

组别	模型	P	R	mAP_0.5	mAP_0.5∶0.95	帧率/（帧· $s - 1$ ）
1	YOLOv8	0.743	0.649	0.677	0.367	243.90
2	YOLOv8+WIoU	0.791	0.650	0.754	0.458	204.08
3	YOLOv8+WIoU+SimAM	0.790	0.681	0.806	0.476	178.57
4	YOLOv8+WIoU+SimAM+DCN	0.891	0.892	0.918	0.524	147.06

Fig.10 PR curves

Fig.11 Comparison of heat maps between the improved model and the original model

Fig.12 Actual detection effects of the improved model and the original model

References 15

1	Wang J， Zhu C R， Yang Y P，et al.Effect of riveting displacement on the mechanical behavior of CFRP bolted joints with elliptical‑head non‑lug self‑locking rivet nut［J］.International Journal of Advanced Manufacturing Technology，2023，125（5/6）：2161-2182.
2	吕帅帅，杨宇，王彬文，等.基于改进Mask-RCNN的飞行器结构裂纹自动检测方法［J］.振动、测试与诊断，2021，41（3）：487-494，620.
	Shuai‑shuai Lyu， Yang Yu， Wang Bin‑wen，et al.An automatic crack detection method for structure test based on improved Mask-RCNN［J］.Journal of Vibration，Measurement & Diagnosis，2021，41（3）：487-494，620.
3	Liu Y X， Wu D B， Liang J W，et al.Aeroengine blade surface defect detection system based on improved faster RCNN［J］.International Journal of Intelligent Systems，2023：1992415.
4	潘睿志，林涛，李超，等.基于深度学习的多尺寸汽车轮辋焊缝检测与定位系统研究［J］.光学精密工程，2023，31（8）：1174-1187.
	Pan Rui‑zhi， Lin Tao， Li Chao，et al.Research on multi size automobile rim weld detection and positioning system based on depth learning［J］.Optics and Precision Engineering，2023，31（8）：1174-1187.
5	Aboah A， Wang B， Bagci U，et al.Real‑time multi‑class helmet violation detection using few‑shot data sampling technique and YOLOv8［C］//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）.Vancouver，2023：5350-5358.
6	Feng C J， Zhong Y J， Gao Y，et al.TOOD：task‑aligned one‑stage object detection［C］//2021 IEEE/CVF International Conference on Computer Vision （ICCV）.Montreal，2021：3490-3499.
7	Li X， Wang W H， Wu L J，et al.Generalized focal loss：learning qualified and distributed bounding boxes for dense object detection［J］.ArXiv，2022：2006.04388.
8	Dai J F， Qi H Z， Xiong Y W，et al.Deformable convolutional networks［C］//2017 IEEE International Conference on Computer Vision （ICCV）.Venice，2017：764-773.
9	Zhu X Z， Hu H， Lin S，et al.Deformable ConvNets V2：more deformable，better results［C］//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.Long Beach，2019：9300-9308.
10	Hu J， Shen L， Albanie S，et al.Squeeze‑and‑excitation networks［J］.IEEE Transaction on Pattern Analysis and Machine Intelligence，2020，42（8）：2011-2023.
11	Woo S， Park J， Lee J Y，et al.CBAM：convolutional block attention module［C］//European Conference on Computer Vision.Cham：Springer，2018：3-19.
12	Park J， Woo S， Lee J Y，et al.BAM：bottleneck attention module［J］.ArXiv，2022：1807.06514.
13	Yang L X， Zhang R Y， Li L D，et al.SimAM：a simple，parameter‑free attention module for convolutional neural networks［C］//International Conference on Machine Learning （ICML）.San Diego，2021：1-12.
14	Tong Z J， Chen Y H， Xu Z W，et al.Wise-IoU：bounding box regression loss with dynamic focusing mechanism［J］.ArXiv，2023：2301.10051.
15	Selvaraju R R， Cogswell M， Das A，et al.Grad-CAM：visual explanations from deep networks via gradient‑based localization［J］.International Journal of Computer Vision，2020，128（2）：336-359.

[1]	Yan LIU, Qi-jie BU, Hong-chen ZHAO, Xin GUO. Operating Performance Assessment of Flotation Process Based on Multi-source Heterogeneous Information [J]. Journal of Northeastern University(Natural Science), 2024, 45(9): 1217-1226.
[2]	Wei-wei LIU, Jia-he QIU, Guang-da HU, Ze-yuan LIU. Surface Damage Detection Method for Retired Shaft Parts Based on Improved YOLOv5 [J]. Journal of Northeastern University(Natural Science), 2024, 45(7): 1002-1010.
[3]	An-lin TIAN, Wei-min LEI, Peng ZHANG, Wei ZHANG. A Multi-scale Edge Detection Method Based on Encoder-Decoder [J]. Journal of Northeastern University(Natural Science), 2024, 45(7): 936-943.
[4]	Li-xin GUO, Su-tao BI, Ming-yang ZHAO. State Detection Algorithm of Manipulator Based on Improved YOLOv4 Lightweight Network [J]. Journal of Northeastern University(Natural Science), 2024, 45(6): 769-775.
[5]	Yuan MA, Li-huang SHE, Jia-wei LI, Xi-rong BAO. Adaptive Graph Convolutional 3D Point Cloud Recognition Algorithm Based on Attention Mechanism [J]. Journal of Northeastern University(Natural Science), 2024, 45(6): 786-792.
[6]	Hu FENG, Ke-chen SONG, Wen-qi CUI, Yun-hui YAN. Few-Shot Semantic Segmentation of Strip Steel Surface Defects Based on Meta-Learning [J]. Journal of Northeastern University(Natural Science), 2024, 45(3): 354-360.
[7]	Zhi-jin ZHANG, He LI, Yu-shi HUANG, Wen-xue WANG. Application of Deep Residual Shrinkage Network in Rolling Bearing Fault Diagnosis [J]. Journal of Northeastern University(Natural Science), 2024, 45(11): 1587-1594.
[8]	Meng-yuan LIU, Zhao-xia WU, Jin-yang WANG, Guang-lei XIA. Air Permeability Prediction of Sinter Layer Based on TST-LSTM Model [J]. Journal of Northeastern University(Natural Science), 2024, 45(10): 1379-1385.
[9]	Ying SUN, Ya-ru ZHOU, Xue-ying ZHANG. Speech Emotion Recognition Fusing Functional Paralanguage Proportion Coefficient [J]. Journal of Northeastern University(Natural Science), 2024, 45(1): 40-48.
[10]	JIANG Yang， LIU Cheng， DING Qi-chuan， WANG Li. Segmentation of COVID-19 CT Images Based on Dual Attention Mechanism [J]. Journal of Northeastern University(Natural Science), 2023, 44(9): 1259-1268.
[11]	ZHOU Song， GAO Tian-han. EEG Recognition Method for Epileptic Patients Based on RNN Model with Attention Mechanism [J]. Journal of Northeastern University(Natural Science), 2023, 44(8): 1098-1103.
[12]	DING Qi-chuan， WANG Li， LIU Cheng. Classification of Pulmonary Nodule by Combining Long-Distance Channel Attention and Pathological Feature [J]. Journal of Northeastern University(Natural Science), 2023, 44(4): 476-485.
[13]	CHEN Cheng， SHI Pei-xin， WANG Zhan-sheng， JIA Peng-jiao. Shield Load Prediction Method Based on Deep Learning with Multiattention Mechanism [J]. Journal of Northeastern University(Natural Science), 2023, 44(11): 1631-1638.
[14]	LIN Qing-yang， CHEN Xiao-fang， XIE Yong-fang. An Superheat Identification Method in Aluminium Electrolysis Based on Residual Convolutional Self-Attention Neural Network [J]. Journal of Northeastern University(Natural Science), 2023, 44(1): 8-17.
[15]	LIU Yang， YAN Dong-mei， MENG Fan-wei. Improved Two-Branch Person Re-identification Algorithm Based on Transformer [J]. Journal of Northeastern University(Natural Science), 2023, 44(1): 26-32.