Few-Shot Semantic Segmentation of Strip Steel Surface Defects Based on Meta-Learning

doi:10.12068/j.issn.1005-3026.2024.03.007

Abstract

Abstract:

Due to the limited availability of strip surface defect samples， the application of deep neural networks in strip surface detection is constrained. To solve this practical issue， a meta‐learning‐based few‐shot semantic segmentation method is proposed. A multi‐scale decoder and attention mechanism are used in the proposed method. The multi‐scale decoder can aggregate the defect information at different scales and improve segmentation accuracy of the proposed network. The attention mechanism can effectively improve the expression of defect features and suppress the interference of defect background information. In addition， a novel few‐shot steel strip surface defect semantic segmentation dataset is constructed including nine classes of strip steel surface defects. Comparison experiments on the proposed dataset show that the proposed method is superior to similar few‐shot segmentation methods such as PFENet， SCLNet， and HSNet in terms of evaluation index mean intersection over union and foreground‐background intersection over union.

Key words: strip steel surface defect detection, meta‐learning, few‐shot semantic segmentation, attention mechanism, multi‐scale decoder

CLC Number:

TP 391.4

Hu FENG, Ke-chen SONG, Wen-qi CUI, Yun-hui YAN. Few-Shot Semantic Segmentation of Strip Steel Surface Defects Based on Meta-Learning[J]. Journal of Northeastern University(Natural Science), 2024, 45(3): 354-360.

Figures/Tables 10

References 20

1	Song G R， Song K C， Yan Y H.EDRNet：encoder‐decoder residual network for salient object detection of strip steel surface defects［J］.IEEE Transactions on Instrumentation and Measurement，2020，69（12）：9709-9719.
2	Song K C， Yan Y H.A noise robust method based on completed local binary patterns for hot‐rolled steel strip surface defects［J］.Applied Surface Science，2013，285：858-864.
3	Zhou D H， Wei M H， Si X S.A survey on anomaly detection，life prediction and maintenance decision for industrial processes［J］.Acta Automatica Sinica，2013，39（6）：711-722.
4	Luo Q W， Fang X， Su J，et al.Automated visual defect classification for flat steel surface：a survey［J］.IEEE Transactions on Instrumentation and Measurement，2020，69（12）：9329-9349.
5	周鹏，徐科，杨朝霖.基于SIFT的中厚板表面缺陷识别方法［J］.清华大学学报（自然科学版），2018，58（10）：881-887.
	Zhou Peng， Xu Ke， Yang Chao‑lin.Surface defect recognition for moderately thick plates based on a SIFT operator［J］.Journal of Tsinghua University （Science and Technology），2018，58（10）：881-887.
6	Chu M X， Gong R F， Gao S，et al.Steel surface defects recognition based on multi‑type statistical features and enhanced twin support vector machine［J］.Chemometrics and Intelligent Laboratory Systems，2017，171：140-150.
7	张德富，宋克臣，牛孟辉，等.基于一维卷积的生产线冷态重轨表面缺陷快速检测［J］.东北大学学报（自然科学版），2021，42（2）：276-281.
	Zhang De‐fu， Song Ke‐chen， Niu Meng‐hui， al el.Rapid detection of cold heavy rail surface defects of production line based on one‐dimensional convolution network［J］.Journal of Northeastern University （Natural Science），2021，42（2）：276-281.
8	何彧，宋克臣，张德富，等.融合多层级特征的弱监督钢板表面缺陷检测方法［J］.东北大学学报（自然科学版），2021，42（5）：687-692.
	He Yu， Song Ke‑chen， Zhang De‑fu，et al.Weakly‑supervised steel plate surface defect detection algorithm by integrating multiple level features［J］.Journal of Northeastern University （Natural Science），2021，42（5）：687-692.
9	Bao Y Q， Song K C， Liu J，et al.Triplet‐graph reasoning network for few‐shot metal generic surface defect segmentation［J］.IEEE Transactions on Instrumentation and Measurement，2021，70：1-11.
10	Cha Y J， Choi W， Suh G，et al.Autonomous structural visual inspection using region‐based deep learning for detecting multiple damage types［J］.Computer Aided Civil and Infrastructure Engineering，2018，33（9）：731-747.
11	Long J， Shelhamer E， Darrell T.Fully convolutional networks for semantic segmentation［C］//Conference on Computer Vision and Pattern Recognition （CVPR）.Boston，2015：3431-3440.
12	Zhang B F， Xiao J M， Qin T.Self‐guided and cross‐guided learning for few‐shot segmentation［C］//Conference on Computer Vision and Pattern Recognition （CVPR）.Kuala Lumpur，2021：8312-8321.
13	Tian Z T， Zhao H S， Shu M，et al.Prior guided feature enrichment network for few‐shot segmentation［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44（2）：1050-1065.
14	Min J H， Kang D， Cho M.Hypercorrelation squeeze for few‐shot segmentation［C］//International Conference on Computer Vision （ICCV）.Montreal，2021：6941-6952.
15	Dong N Q， Xing E P.Few‐shot semantic segmentation with prototype learning［C］//British Machine Vision Conference （BMVC）.Newcastle，2018：1-13.
16	Feng X L， Gao X W， Luo L.X‐SDD：a new benchmark for hot rolled steel strip surface defects detection［J］.Symmetry，2021，13（4）：706-723.
17	Lyu X， Duan F， Jiang J，et al.Deep metallic surface defect detection：The new benchmark and detection network［J］.Sensors，2020，20（6）.1562-1578.
18	Russakovsky O， Deng J， Su H，et al.ImageNet large scale visual recognition challenge［J］.International Journal of Computer Vision，2015，115（3）：211-252.
19	He K M， Zhang X Y， Ren S Q，et al.Deep residual learning for image recognition［C］//Conference on Computer Vision and Pattern Recognition （CVPR）.Las Vegas，2016：770-778.
20	Woo S， Park J， Lee J Y，et al.CBAM：convolutional block attention module［C］//European Conference on Computer Vision （ECCV）.Munich，2018：3-19.

集合	缺陷
Fold-1	红铁皮	铁皮灰	板系氧化皮
Fold-2	轧辊印	水污	夹杂
Fold-3	油污	孔洞	划痕

集合	缺陷
Fold-1	红铁皮	铁皮灰	板系氧化皮
Fold-2	轧辊印	水污	夹杂
Fold-3	油污	孔洞	划痕

方法	1个支持集样本					5个支持集样本
	平均交并比/%				前景-背景交并比/%	平均交并比/%				前景-背景交并比/%
	Fold-1	Fold-2	Fold-3	均值	前景-背景交并比/%	Fold-1	Fold-2	Fold-3	均值	前景-背景交并比/%
SCLNet	53.3	38.0	46.4	45.9	75.9	55.6	39.2	47.5	47.4	76.5
PFENet	66.1	44.2	55.7	55.3	72.9	67.2	45.9	56.1	56.4	74.9
HSNet	61.4	43.1	58.7	54.4	73.3	63.8	46.3	58.9	56.3	74.4
本文方法	68.1	43.7	60.0	57.3	73.9	68.3	43.4	58.0	56.6	74.8

方法	1个支持集样本					5个支持集样本
	平均交并比/%				前景-背景交并比/%	平均交并比/%				前景-背景交并比/%
	Fold-1	Fold-2	Fold-3	均值	前景-背景交并比/%	Fold-1	Fold-2	Fold-3	均值	前景-背景交并比/%
SCLNet	53.3	38.0	46.4	45.9	75.9	55.6	39.2	47.5	47.4	76.5
PFENet	66.1	44.2	55.7	55.3	72.9	67.2	45.9	56.1	56.4	74.9
HSNet	61.4	43.1	58.7	54.4	73.3	63.8	46.3	58.9	56.3	74.4
本文方法	68.1	43.7	60.0	57.3	73.9	68.3	43.4	58.0	56.6	74.8

方法	1个支持集样本		5个支持集样本
方法	平均交并比/%	前景-背景交并比/%	平均交并比/%	前景-背景交并比/%
基准	51.8	71.3	53.8	72.0
基准+多尺度解码器	56.2	73.3	56.5	73.7
基准+注意力机制	53.9	72.0	54.8	72.8
基准+多尺度解码器+注意力机制	57.3	73.9	56.6	73.8