高级检索

基于视觉注意VGGNet的中厚板低碳钢多层多道熔池分类模型

周浩, 陈善本

周浩, 陈善本. 基于视觉注意VGGNet的中厚板低碳钢多层多道熔池分类模型[J]. 焊接学报, 2024, 45(11): 71-76. DOI: 10.12073/j.hjxb.20240710001
引用本文: 周浩, 陈善本. 基于视觉注意VGGNet的中厚板低碳钢多层多道熔池分类模型[J]. 焊接学报, 2024, 45(11): 71-76. DOI: 10.12073/j.hjxb.20240710001
ZHOU Hao, CHEN Shanben. A MLMP welding pool classification model for medium-thick low-carbon steel plates based on a VGGNet with a visual attention mechanism[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(11): 71-76. DOI: 10.12073/j.hjxb.20240710001
Citation: ZHOU Hao, CHEN Shanben. A MLMP welding pool classification model for medium-thick low-carbon steel plates based on a VGGNet with a visual attention mechanism[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(11): 71-76. DOI: 10.12073/j.hjxb.20240710001

基于视觉注意VGGNet的中厚板低碳钢多层多道熔池分类模型

基金项目: 国家自然科学基金资助项目(61873164)
详细信息
    作者简介:

    周浩,博士;主要研究方向为图像处理模式识别,焊接过程控制;Email: image_processing@sjtu.edu.cn

    通讯作者:

    陈善本,博士,教授,博士研究生导师; Email: sbchen@sjtu.edu.cn.

  • 中图分类号: TG 409

A MLMP welding pool classification model for medium-thick low-carbon steel plates based on a VGGNet with a visual attention mechanism

  • 摘要:

    重大装备制造中厚板机器人多层多道焊(multi-layer and multi-pass welding,MLMPW)一直是热点和难点,而实现机器人MLMPW的核心是对其熔池的获取、监控并分类.为了提高MLMPW的自动化和智能化,有必要开发一个熔池图像在线分类系统.针对焊接过程中的熔池图像提出了一种新的MLMPW熔池分类方法——基于视觉注意的(SENet)VGGNet熔池分类方法.为了提高效率和精度,引入迁移学习中的预训练模型到网络训练过程中.因为针对中厚板多层多道熔池研究较少,导致熔池公开数据集较少,为了应对这一问题,需要对数据集进行增广. 结果表明,提出的模型可快速有效的对七类MLMPW熔池进行准确分类,预测精度可达到98.39%.

    Abstract:

    The multi-layer and multi-pass welding (MLMPW) of medium-thick plates in large equipment manufacturing has always been a challenging and important research topic. The core of achieving robotic MLMPW lies in acquiring, monitoring, and classifying the weld pool. To enhance the automation and intelligence of MLMPW, it is necessary to develop an online classification system for weld pool images. A novel MLMPW weld pool classification method is proposed for weld pool images during the welding process, utilizing a VGGNet with a visual attention mechanism (SENet). To improve efficiency and accuracy, pre-trained models from transfer learning are introduced into the network training process. Due to the lack of research on the medium-thick plate multi-layer and multi-pass molten pool, there are few open datasets of molten pool. In order to deal with this problem, data augmentation is necessary to applied to the dataset. The results indicate that the proposed model can quickly and effectively classify seven types of MLMPW weld pools with an accuracy of 98.39%.

  • 图  1   MLMPW熔池视觉传感系统

    Figure  1.   MLMPW Molten Pool Vision Sensing System

    图  2   HDR相机位置关系

    Figure  2.   Positional relationship of HDR camera

    图  3   MLMPW排序

    Figure  3.   MLMPW sequence

    图  4   MLMPW 7种类型熔池

    Figure  4.   MLMPW 7 types of molten pool. (a) T1: P1; (b) T2: P2; (c) T3: P4; (d) T4: P-7,11,16; (e) T5: P-3, 5, 6, 8, 9; (f) T6: P-10, 14, 15; (g) T7: P-12, 13

    图  5   熔池数据增广. 第1幅为原图,第2-6幅为翻转和顺时针旋转三次90°的图像,第7-9幅图像为随机旋转、缩放和平移组合的图像.

    Figure  5.   Melt pool data augmentation. The first image is the original image, the second to sixth images are flipped and rotated three times 90° clockwise, and the seventh to ninth images are images with random rotation, scaling, and translation.

    图  6   MLMPW熔池分类总体工作流

    Figure  6.   Overall workflow of MLMPW melton pool classification

    图  7   VGG16分类架构

    Figure  7.   VGG16 classification architecture

    图  8   SENet模型

    Figure  8.   SENet Model

    图  9   训练和测试曲线

    Figure  9.   Training and testing curves. (a) accuracy curve; (b) loss curve

    图  10   混淆矩阵

    Figure  10.   Confusion Matrix. (a) M1; (b) M2; (c) M3; (d) M4

    表  1   MLMPW参数

    Table  1   MLMPW parameters

    焊接类型 焊接速度v/(mm·min−1) 打底焊电流I1/ A 填充焊电流 I2/(A) 焊枪倾斜角θ/ (°) 气流量 q/(L·min−1) 母材尺寸L × W × H/ mm
    CMT 240 170 ~ 185 210 ~ 280 −15 ~ 15 20 400 × 150 × 30
    下载: 导出CSV

    表  2   超参数设置

    Table  2   hyperparameter configuration

    优化算法 Momentum 学习率 损失函数 最大epochs ε Batch size
    SGD 0.9 0.001 cross entropy 50(eps) 8(幅图)
    下载: 导出CSV

    表  3   混淆矩阵

    Table  3   Confusion Matrix

    Confusion matrix Actual value
    Predicted
    value
    TP FP
    FN TN
    下载: 导出CSV

    表  4   熔池分类评价值(%)

    Table  4   Molten pool classification evaluation value

    模型 Type1 Type2 Type3 Type4 Type5 Type6 Type7
    Pt_se_vgg16 Pr 100.00 98.69 97.58 97.02 98.14 97.07 96.46
    Re 99.78 99.04 98.54 96.51 97.19 95.76 98.54
    F1 99.89 98.87 98.05 96.76 97.66 96.41 97.49
    se_vgg16 Pr 99.87 97.88 98.44 96.31 97.94 95.01 96.65
    Re 99.74 98.54 97.35 95.92 96.31 96.19 98.12
    F1 99.81 98.21 97.90 96.12 97.12 95.60 97.38
    Pt_vgg16 Pr 99.96 97.86 97.90 96.93 97.97 96.53 97.29
    Re 99.96 99.04 97.51 96.66 97.46 95.85 98.18
    F1 99.96 98.45 97.70 96.80 97.71 96.19 97.73
    vgg16 Pr 99.87 95.23 95.18 90.87 93.92 91.90 91.00
    Re 99.57 97.09 93.56 91.66 93.84 87.15 94.89
    F1 99.72 96.15 94.36 91.26 93.88 89.46 92.90
    下载: 导出CSV
  • [1]

    Zhang H, Lu H, Cai C, et al. Robot path planning in multi-pass weaving welding for thick plates [C]//Robotic Welding, Intelligence and Automation: RWIA’2010. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011: 351 − 359.

    [2]

    Xue K, Wang Z, Shen J, et al. Robotic seam tracking system based on vision sensing and human-machine interaction for multi-pass MAG welding[J]. Journal of Manufacturing Processes, 2021, 63: 48 − 59. doi: 10.1016/j.jmapro.2020.02.026

    [3]

    Yang C, Zhong J, Chen Y, et al. The realization of no back chipping for thick plate welding[J]. The International Journal of Advanced Manufacturing Technology, 2014, 74: 79 − 88. doi: 10.1007/s00170-014-5927-z

    [4] 徐东辉, 孟范鹏, 孙鹏, 等. 基于深度学习的 GMAW 焊接缺陷在线监测[J]. 焊接学报, 2024, 45(3): 114 − 119.

    Xu D, Meng F, Sun P, et al. Online monitoring of GMAW welding defect based on deep learning[J]. Transactions of the China Welding Institution, 2024, 45(3): 114 − 119.

    [5] 高昶霖, 宋燕利, 左洪洲, 等. 基于动态权重的自适应 PSO-BP 神经网络焊接缺陷成因诊断[J]. 焊接学报, 2022, 43(1): 98 − 106.

    Gao C, Song Y, Zuo H, et al. Cause diagnosis of welding defects based on adaptive PSO-BP neural network with dynamic weighting[J]. Transactions of The China Welding Institution, 2022, 43(1): 98 − 106.

    [6]

    Zhou H, He Y, Chen H, et al. MLD classification model of visual features of multi-layer and multi-pass molten pool during robotic MAG welding of medium-thick steel plates[C]//International Conference on Robotic Welding, Intelligence and Automation. Singapore: Springer Nature Singapore, 2020: 56 − 81.

    [7]

    Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, 2018: 7132 − 7141.

    [8]

    Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [J]. arXiv preprint arXiv: 14091556, 2014.

图(10)  /  表(4)
计量
  • 文章访问数:  82
  • HTML全文浏览量:  10
  • PDF下载量:  39
  • 被引次数: 0
出版历程
  • 收稿日期:  2024-07-09
  • 网络出版日期:  2024-11-14
  • 刊出日期:  2024-11-24

目录

    /

    返回文章
    返回