Defect identification algorithm for weld X-ray imagesbased on the CCBFE-RCNN model
-
摘要:
针对X射线图像人工评定过程中存在劳动强度大、检测效率低等问题,提出一种基于Cascade-RCNN网络改进的多尺度目标检测网络CCBFE-RCNN模型,采用合并卷积层结构和FPN特征金字塔网络提升模型特征提取的尺度范围,增强了模型对于多尺度特征的提取能力;使用BFE特征批量消除网络,随机消除特征图区域,避免多次训练过程中的过拟合问题并强化了特征区域表达,同时对损失函数进行改进,对模型没有准确识别出含缺陷图像加大惩罚. 通过构建并扩充熔焊焊缝X射线图像数据集,对模型进行测试. 结果表明,CCBFE-RCNN缺陷检测模型全类别召回率均值、全类别精确率均值为93.09%和91.92%,与Cascade-RCNN网络模型相比平均召回率提升5.16%,平均精确率提升5.27%. 并使用工业缺陷检测数据集对CCBFE-RCNN模型进行测试,验证了模型的泛化能力,可为焊缝缺陷智能化识别提供算法支撑.
Abstract:In view of the problems of high labor intensity and low detection efficiency in the manual evaluation process of X-ray images, a multi-scale object detection network CCBFE-RCNN model based on the improved Cascade-RCNN network is proposed. The combined convolutional layer structure and FPN feature pyramid network are used to enhance the scale range of model feature extraction, and the model's ability to extract multi-scale features is enhanced; Use BFE features to batch eliminate networks, randomly eliminate feature map regions, avoid overfitting problems during multiple training processes, and enhance feature region expression. At the same time, improve the loss function to increase penalties for models that do not accurately identify images containing defects. The model was tested by constructing and expanding a dataset of X-ray images of fusion welds The results show that the CCBFE-RCNN defect detection model has an average recall rate of 93.09% and an average precision rate of 91.92% across all categories, which is 5.16% higher than the average recall rate and 5.27% higher than the average precision rate of the Cascade-RCNN network model. The CCBFE-RCNN model is tested using an industrial defect detection dataset to verify its generalization ability, which can provide algorithm support for intelligent recognition of weld defects.
-
Keywords:
- defect identification /
- concatnated convolution /
- loss function /
- recall /
- precision
-
-
表 1 Resnet101网络
Table 1 Resnet101 network
卷积层 Resnet101 conv1 conv,7 × 7,64,stride 2
max pool,3 × 3, stride 2conv2 $ \left[ {\begin{array}{*{20}{l}} {{\text{conv}},1 \times 1,64} \\ {{\text{conv}},3 \times 3,64} \\ {{\text{conv}},1 \times 1,256} \end{array}} \right] \times 3 $ conv3 $ \left[ {\begin{array}{*{20}{l}} {{\text{conv}},1 \times 1,128} \\ {{\text{conv}},3 \times 3,128} \\ {{\text{conv}},1 \times 1,512} \end{array}} \right] \times 4 $ conv4 $ \left[ {\begin{array}{*{20}{l}} {{\text{conv}},1 \times 1,256} \\ {{\text{conv}},3 \times 3,256} \\ {{\text{conv}},1 \times 1,1024} \end{array}} \right] \times 23 $ conv5 $ \left[ {\begin{array}{*{20}{l}} {{\text{conv}},1 \times 1,512} \\ {{\text{conv}},3 \times 3,512} \\ {{\text{conv}},1 \times 1,2048} \end{array}} \right] \times 3 $ 表 2 指标计算公式
Table 2 Calculation formula
评价指标 公式 召回率R $ \dfrac{{TP}}{{TP + FN}} $ 精确率P $ \dfrac{{TP}}{{TP + FP}} $ 全类别召回率均值RmAR@0.5 $\dfrac{ {\displaystyle\sum\limits_{j = 1}^c {\sum\limits_{i = 1}^x {R_i} } } }{ {cx} }$ 全类别精确率均值PmAP@0.5 $\dfrac{ {\displaystyle\sum\limits_{j = 1}^c {\displaystyle\sum\limits_{i = 1}^x {P_i} } } }{ {cx} }$ 表 3 不同模型测试结果
Table 3 Results of different models
模型 夹渣 未焊透 气孔 全类别召回率
均值
RmAR@0.5(%)全类别精确率
均值
PmAP@0.5(%)频率
f/(帧·s−1)召回率R(%) 精确率P(%) 召回率R(%) 精确率P(%) 召回率R(%) 精确率P(%) YOLOV5-Tiny 90.23 89.60 83.68 88.71 89.65 86.32 87.85 88.21 61 SSD 85.98 86.33 80.24 85.13 85.07 86.68 83.76 86.05 56 Cascade-RCNN 89.20 87.11 85.26 87.92 89.33 84.92 87.93 86.65 27 SRYOLOV3 88.27 86.99 84.47 87.82 88.67 86.81 87.14 87.21 51 TLMDDNet 92.13 90.45 88.11 90.18 92.67 90.73 90.97 90.45 19 Swin-Cascade-RCNN 93.15 88.42 86.79 88.48 90.26 88.98 90.07 88.99 29 CCBFE-RCNN 93.69 90.63 90.40 93.01 95.20 92.13 93.09 91.92 22 表 4 不同数据集测试结果
Table 4 Test results for different datasets
数据集 全类别召回率均值
RmAR@0.5(%)全类别精确率均值
PmAP@0.5(%)DAGM 92.56 91.84 NEU 95.68 93.58 AITEX 89.34 87.59 -
[1] Li Y W. Research progress of aluminum alloy welding technology[J]. The International Journal of Advanced Manufacturing Technology, 2020, 109(1): 1207 − 1218.
[2] Chen S B. On intelligentized welding manufacturing. In: Robotic welding[C]//Intelligence and Automation: RWIA’2014. Springer International Publishing, 2015: 3 − 34.
[3] Ahmed, Mohamed M Z. Friction stir welding of aluminum in the aerospace industry: the current progress and state-of-the-art review[J]. Materials, 2023, 16(8): 2971. doi: 10.3390/ma16082971
[4] Mohandas T. Welding technologies in aerospace applications[J]. Aerospace Materials and Material Technologies, 2017, 2(1): 65 − 83.
[5] Wang G, Liao T W. Automatic identification of different types of welding defects in radiographic images[J]. Ndt & E International, 2002, 35(8): 519 − 528.
[6] Madhvacharyula, Anirudh Sampath, Pavan, et al. In situ detection of welding defects: a review[J]. Welding in the World, 2022, 66(4): 611 − 628. doi: 10.1007/s40194-021-01229-6
[7] Andrew G, Vignesh S, Sooyoung L, et al. Applications of deep learning for fault detection in industrial cold forging[J]. International Journal of Production Reseach, 2021, 59(16): 4826 − 4835.
[8] Lenka L, Jan S, Renata W. The learning path to neural network industrial application in distributed envisonment[J]. Processes, 2021, 9(12): 2247. doi: 10.3390/pr9122247
[9] Dutta, Sourav. An overview on the evolution and adoption of deep learning applications used in the industry[J]. Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery, 2018, 8(4): 1257. doi: 10.1002/widm.1257
[10] 王睿, 胡云雷, 刘卫朋, 等. 基于边缘AI的焊缝X射线图像缺陷检测[J]. 焊接学报, 2022, 43(1): 79 − 84. Wang Rui, Hu Yunlei, Liu Weipeng, et al. Defect detection of weld X-ray image based on edge AI[J]. Transactions of the China Welding Institution, 2022, 43(1): 79 − 84.
[11] 刘欢, 刘骁佳, 王宇斐, 等. 基于复合卷积层神经网络结构的焊缝缺陷分类技术[J]. 航空学报, 2022, 43(S1): 165 − 172. Liu Huan, Liu Xiaojia, Wang Yufei, et al. Weld defect classification technology based on compound convolution neural network structure[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(S1): 165 − 172.
[12] 任文坚, 王永红, 李春凯, 等. 液体火箭发动机推力室熔焊缝数字化胶片图像缺陷识别方法研究[J]. 电焊机, 2023, 53(1): 9 − 14. Ren Wenjian, Wang Yonghong, Li Chunkai, et al. Research on automatic detection method of DR digital imaging for brazing seam defects of thrust chamber body[J]. Electric Welding Machine, 2023, 53(1): 9 − 14.
[13] 程松, 杨洪刚, 徐学谦, 等. 基于YOLOv5的改进轻量型X射线铝合金焊缝缺陷检测算法[J]. 中国激光, 2022, 49(21): 136 − 144. Cheng Song, Yang Honggang, Xu Xueqian, et al. Improved lightweight X-ray aluminum alloy weld defects detection algorithm basedon YOLOv5[J]. Chinese Journal of Lasers, 2022, 49(21): 136 − 144.
[14] 杨会敏, 李春棚, 袁新安, 等. 基于SSD的交流电磁场缺陷智能识别方法[J]. 无损检测, 2022, 44(8): 25 − 30. Yang Huimin, Li Chunpeng, Yuan Xin’an, et al. Defects intelligent recognition method of ACFM based on SSD[J]. Nondestructive Testing, 2022, 44(8): 25 − 30.
[15] Zhang Z, Zhang Y, Wen Y, et al. Intelligent defect detection method for additive manufactured lattice structures based on a modified YOLOv3 model[J]. Journal of Nondestructive Evaluation, 2022, 41(1): 1 − 14. doi: 10.1007/s10921-021-00836-z
[16] Wei X, Wei D, Suo D, et al. Multi-target defect identification for railway track line based on image processing and improved YOLOv3 model[J]. IEEE Access, 2020, 8(1): 973 − 988.
[17] Dong C, Zhang K, Xie Z, et al. An improved cascade RCNN detection method for key components and defects of transmission lines[J]. IET Generation, Transmission Distribution, 2023, 17(19): 4277 − 4292. doi: 10.1049/gtd2.12948
[18] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, realtime object detection[J]. Computer Vision and Pattern Recognition, 2016, 91(1): 779 − 788.
[19] Bhavya Sree B, Yashwanth Bharadwaj V, Neelima N. An inter-comparative survey on state-of-the-art detectors—R-CNN, YOLO and SSD[C]//Intelligent Manufacturing and Energy Sustainability: Proceedings of ICIMES 2020. Springer Singapore, 2021: 475 − 483.
[20] Cao C Q. An improved faster R-CNN for small object detection[J]. Ieee Access, 2019, 7(1): 106838 − 106846.
[21] Bharati P, Pramanik A. Deep learning techniques—R-CNN to mask R-CNN: a survey[J]. Computational Intelligence in Pattern Recognition:Proceedings of CIPR 2019, 2020(1): 657 − 668.
[22] Cai Z W, Vasconcelos N. Cascade R-CNN: delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recongnition, 2018: 6165 − 6162.
[23] Sun C Y, Hong X J, Shi S, et al. Cascade faster R-CNN detection for vulnerable plaques in OCT images[J]. IEEE Access, 2021(9): 24697 − 24704.
[24] Cai Z W, Vasconcelos Nuno. Cascade R-CNN: high quality object detection and instance segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(5): 1483 − 1498.
[25] Feng T, Liu J G, Xia F, et al. A double-branch surface detection system for armatures in vibration motors with miniature volume based on ResNet-101 and FPN[J]. Sensors, 2020, 20(8): 1 − 16. doi: 10.1109/JSEN.2020.2978309
[26] Alshammari A, Chabaan R C. Sppn-Rn101: spatial pyramid pooling network with resnet 101-Based foreign object debris detection in airports[J]. Mathematics, 2023, 11(4): 841. doi: 10.3390/math11040841
[27] Ren K, Huang L, Fan C Q, et al. Real-time traffic sign detection network using DS-DetNet and lite fusion FPN[J]. Journal of Real-Time Image Processing, 2021, 18(6): 1 − 11.
[28] Li Y C, Zhou S L, Chen H, et al. Attention-based fusion factor in FPN for object detection[J]. Applied Intelligence, 2022, 1(1): 1 − 10.
[29] Chen J Y, Liu S P, Zhao L. Small object detection combining attention mechanism and a novel FPN[J]. Journal of Intelligent Fuzzy Systems, 2022, 42(4): 3691 − 3703.
[30] Dai Z Z, Chen M Q, Gu X D, et al. Batch drop block network for person re-identification and beyond [C]// Proceedings of the IEEE Conference on Computer Vision, 2019: 3690 − 3700.
[31] Ye J. Single valued neutrosophic cross-entropy for multicriteria decision making problems[J]. Applied Mathematical Modelling, 2014, 38(3): 1170 − 1175. doi: 10.1016/j.apm.2013.07.020
[32] Chen Y P, Xu K, Zhou P, et al. Improved cross entropy loss for noisy labels in vision leaf disease classification[J]. IET Image Processing, 2022, 16(6): 1511 − 1519. doi: 10.1049/ipr2.12402
[33] 程松, 戴金涛, 杨洪刚, 等. 基于改进型YOLOv4的焊缝图像检测与识别[J]. 激光与光电子学进展, 2022, 59(16): 105 − 111. Cheng Song, Dai Jintao, Yang Honggang, et al. Weld Image detection and recognition based on improved YOLOv4[J]. Laser & Optoelectronics Progress, 2022, 59(16): 105 − 111.