Research on weld surface defect detection method based on RGB-D feature fusion
-
摘要:
金属表面焊缝缺陷的准确检测是确保工件安全使用的前提，由于缺陷与母材颜色相近、图像不清晰等情况，使用常规的2DRGB视觉难以完全检测出所有的缺陷类别，需要添加深度信息来辅助检测. 试验提出一种基于RGB-D数据特征融合的焊缝表面缺陷检测方法，在YOLOv8网络模型的基础上，利用改进的对称主干网络结构提取RGB和深度特征的有效特征层，引入RGB-D数据特征融合模块，实现了RGB特征和深度特性在空间与通道位置的融合，在YOLOv8模型中加入CIoU-NMS(complete intersection over union-non max suppression)非极大值抑制模块，提高了检验框的准确度. 针对随机包含有烧穿、飞溅、焊瘤和气孔4个类别焊缝缺陷的图像进行了试验，结果表明，改进的YOLOv8比YOLOv8漏检率下降了17.84%，误检率下降了19.46%，证明了所述方法的有效性与准确性.
Abstract:The correct detection of surface defects of metal welds is the prerequisite to ensure the safe use of workpieces. Because the defects are similar in color to the base material and the images are not clear, it is difficult to completely detect all defect categories using conventional 2D RGB vision, and depth information needs to be added to assist detection. Therefore, a weld surface defect detection method based on RGB-D data feature fusion is proposed. Based on YOLOv8 network model, this method extracts effective feature layers of RGB and depth features by using improved symmetric backbone network structure, and introduces RGB-D data feature fusion module to realize the fusion of RGB and depth features in space and channel position. The CIOU-NMS non-maximum suppression module was added to YOLOv8 model to improve the accuracy of the check box. In this paper, the experiments were carried out in four categories: pore, weld beading, splash and burnthrough. The results showed that the missed detection rate of the improved YOLOv8 was reduced by 17.84% and the false detection rate by 19.46% compared with YOLOv8. The effectiveness and accuracy of the proposed method are proved.
-
Keywords:
- target detection /
- weld defect detection /
- feature fusion /
- YOLOv8
-
-
![]()
图 9 融合过程特征
Figure 9. Fusion process feature. (a) first fusion; (b) second fusion; (c) third fusion
![]()
图 10 焊缝表面缺陷检测结果
Figure 10. Example of weld surface defect detection results. (a) pore; (b) weld beading; (c) splash; (d) burnthrough
表 1 不同焊缝表面缺陷检测方法性能对比
Table 1 Performance comparison of different detection methods for weld surface defects
算法 输入数据 主干网络 准确率P(%) 召回率R(%) 平均精度均值mAP(%) YOLOv8 RGB — 72.2 53.0 60.2 改进YOLOv8 RGB-D 改进主干网络 39.3 21.6 28.7 改进YOLOv8 RGB-D Fusion + 改进主干网络 76.4 56.1 63.8 
下载: 导出CSV
表 2 工程应用中不同焊缝表面缺陷检测方法测试结果对比
Table 2 Comparison of test results of different weld surface defect detection methods in engineering applications
算法 输入数据 主干网络 漏检率
$\varepsilon_O $(%)误检率
$\varepsilon_N $(%)YOLOv8 RGB — 22.16 20 改进YOLOv8 RGB-D 改进主干网络 4.32 0.54
下载: 导出CSV
-
[1] Viola P, Jones M J. Robust real-time face detection[J]. International Journal of Computer Vision, 2004, 57(2): 137 − 154. doi: 10.1023/B:VISI.0000013087.49260.fb
[2] 胡丹, 吕波, 王静静, 等. 焊缝表面缺陷激光视觉传感HOG-SVM的检测方法[J]. 焊接学报, 2023, 44(1): 56 − 62. doi: 10.12073/j.hjxb.20211231001 Hu Dan, Lü Bo, Wang Jingjing, et al. Weld surface defect detection method based on laser vision sensor HOG-SVM[J]. Transactions of the Chinese Welding Institution, 2023, 44(1): 56 − 62. doi: 10.12073/j.hjxb.20211231001
[3] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014: 580 − 587.
[4] Girshick R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision, Santiago, Chile, USA, 2015: 1440 − 1448.
[5] Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137 − 1149. doi: 10.1109/TPAMI.2016.2577031
[6] He K, Gkioxari G, Dollár P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386 − 397. doi: 10.1109/TPAMI.2018.2844175
[7] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognitio, Las Vegas, NV, USA, 2016: 779 − 788.
[8] 吴天成, 王晓荃, 蔡艺军, 等. 基于特征融合的轻量级SSD目标检测方法[J]. 液晶与显示, 2021, 36(10): 1437 − 1444. doi: 10.37188/CJLCD.2021-0007 Wu Tiancheng, Wang Xiaoquan, Cai Yijun, et al. Lightweight SSD target detection method based on feature fusion[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(10): 1437 − 1444. doi: 10.37188/CJLCD.2021-0007
[9] Bocllkovskiy A, Wang C Y, Liao H. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. arXiv preprint arXiv: 2004.10934. Available at: https://arxiv.org/abs/2004.10934.
[10] Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7 trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 7464 − 7475.
[11] Zuo Y, Wang J, Song J. Application of YOLO object detection network in weld surface defect detection[C]//2021 IEEE 11th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems, Jiaxing, China. 2021: 704-710.
[12] 王睿, 胡云雷, 刘卫朋, 等. 基于边缘AI的焊缝X射线图像缺陷检测[J]. 焊接学报, 2022, 43(1): 79 − 84. doi: 10.12073/j.hjxb.20210516001 Wang Rui, Hu Yunlei, Liu Weipeng, et al. Weld X-ray image defect detection based on edge AI[J]. Transactions of the China Welding Institution, 2022, 43(1): 79 − 84. doi: 10.12073/j.hjxb.20210516001
[13] Pablo R, Manuel R. Weld bead detection based on 3D geometric features and machine learning approaches[J]. IEEE Access, 2019, 7: 14714 − 14727. doi: 10.1109/ACCESS.2019.2891367
[14] 张文铠, 刘佳. 基于改进YOLOv8s的钢材表面缺陷检测[J]. 北京信息科技大学学报(自然科学版), 2023, 38(6): 33 − 40. Zhang Wenkai, Liu Jia. Steel surface defect detection based on improved YOLOv8s[J]. Journal of Beijing University of Information Science and Technology (Natural Science Edition), 2023, 38(6): 33 − 40.
[15] Li X, Lü C, Wang W, et al. Generalized focal loss: towards efficient representation learning for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(3): 3139 − 3153.
[16] Oro D, Fernández C, Martorell X, et al. Work-efficient parallel non-maximum suppression kernels[J]. The Computer Journal, 2022, 65(4): 773 − 787.
[17] Lin T, Maire M, Belongie S, et al. Microsoft COCO: common objects in context[C]//European Conference on Computer Vision, Cham, Switzerland, 2014: 740 − 755.
计量
- 文章访问数: 216
- HTML全文浏览量: 83
- PDF下载量: 58
