Advanced Search
HONG Yuxiang, YING Qiluo, LIN Kai, WANG Kaiming, WANG Yaoqi. Arc welding molten pool image recognition based on attention mechanism and transfer learning[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2025, 46(4): 94-102. DOI: 10.12073/j.hjxb.20240112003
Citation: HONG Yuxiang, YING Qiluo, LIN Kai, WANG Kaiming, WANG Yaoqi. Arc welding molten pool image recognition based on attention mechanism and transfer learning[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2025, 46(4): 94-102. DOI: 10.12073/j.hjxb.20240112003

Arc welding molten pool image recognition based on attention mechanism and transfer learning

More Information
  • Received Date: January 11, 2024
  • Available Online: March 25, 2025
  • Due to the influence of complex time-varying interference and variations in process conditions during the welding process, the boundary characteristics of the molten pool are easy to be blurred, and the scale information is complex and changeable, which poses significant challenges to the accuracy recognition and the robust segmentation of the molten pool. In this paper, a molten pool image recognition method combining attention mechanism and transfer learning is proposed. Firstly, the residual block(RB) is added to the UNet down-sampling process to extract multi-scale low-level features, and the coordinate attention block(CAB) is introduced in the down-sampling and up-sampling processes to improve the feature weight of the effective region. Secondly, the pre-trained deep convolutional neural network in Pascal VOC2012 is transferred to the UNet network to realize feature transfer and parameter sharing, so as to alleviate the over-reliance of training effect on datasets. The TL-RCUNet network proposed in this paper has achieved good recognition results on the untrained MAG and TIG cross-process datasets. The mean intersection over union(MIoU) reaches 96.21% and 79.55%, respectively, which is about 15% and 25% higher than the classical semantic segmentation network. The model provides a feasible solution to the problem that existing semantic segmentation methods rely on a large number of training samples and pixel-level annotations based on expert experience.

  • [1]
    HONG Y X, JIANG Y X, YANG M X, et al. Intelligent seam tracking in foils joining based on spatial-temporal deep learning from molten pool serial images[J]. Robotics and Computer-Integrated Manufacturing, 2025, 91: 102840. doi: 10.1016/j.rcim.2024.102840
    [2]
    HONG Y X, YANG M X, JIANG Y X, et al. Real-time quality monitoring of ultrathin sheets edge welding based on microvision sensing and SOCIFS-SVM[J]. IEEE Transactions on Industrial Informatics, 2022, 19(4): 5506 − 5516.
    [3]
    张志芬, 陈善本, 张裕明, 等. 焊接智能化监测技术研究现状与展望[J]. 焊接学报, 2024, 45(11): 10 − 20,70.

    ZHANG Zhifen, CHEN Shanben, ZHANG Yuming, et al. Research progress and prospect of welding intelligent monitoring technology[J]. Transactions of the China Welding Institution, 2024, 45(11): 10 − 20,70.
    [4]
    徐远钊, 罗玖田, 方乃文, 等. 基于MS-FCM算法的船体板熔池图像处理技术[J]. 焊接学报, 2024, 45(3): 82 − 90. doi: 10.12073/j.hjxb.20231010001

    XU Yuanzhao, LUO Jiutian, FANG Naiwen, et al. Image processing technology for ship plate melt pool based on MS-FCM algorithm[J]. Transactions of the China Welding Institution, 2024, 45(3): 82 − 90. doi: 10.12073/j.hjxb.20231010001
    [5]
    HONG Y X, YANG M X, CHANG B H, et al. Filter-PCA-based process monitoring and defect identification during climbing helium arc welding process using DE-SVM[J]. IEEE Transactions on Industrial Electronics, 2022, 70(7): 7353 − 7362.
    [6]
    李巍, 李太江, 杨略, 等. 改进的U-Net算法在管道内焊缝缺陷图像分割中的应用[J]. 焊接, 2024(11): 73 − 80.

    LI Wei, LI Taijiang, YANG Lue, et al. Application of improved U-Net algorithm in image segmentation of pipeline inner weld defect[J]. Welding & Joining, 2024(11): 73 − 80.
    [7]
    YANG D Y, DAI P H, CUI S W, et al. Real-time recognition of molten pools based on improved DeepLabV3 + in keyhole tungsten inert gas welding applications[J]. Electronics, 2024, 13(2): 283. doi: 10.3390/electronics13020283
    [8]
    QI J Y , WU Y F. Strip steel surface defect detection algorithm based on improved Faster R-CNN[J]. China Welding, 2024, 33(2): 11 − 22.
    [9]
    马晓锋, 夏攀, 刘海生, 等. 全位置焊接熔池的深度学习检测方法[J]. 机械工程学报, 2023, 59(12): 272 − 283. doi: 10.3901/JME.2023.12.272

    MA Xiaofeng, XIA Pan, LIU Haisheng, et al. Depth learning detection method for all-position weld pool[J]. Journal of Mechanical Engineering, 2023, 59(12): 272 − 283. doi: 10.3901/JME.2023.12.272
    [10]
    LI T P, CAO Y, ZHANG Y M. Analysis of weld pool region constituents in GMAW for dynamic reconstruction through characteristic enhancement and LSTM U-Net networks[J]. Journal of Manufacturing Processes, 2024, 127: 573 − 588. doi: 10.1016/j.jmapro.2024.07.084
    [11]
    BACIOIU D, MELTON G, PAPAELIAS M, et al. Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning[J]. NDT & E International, 2019, 107: 102139.
    [12]
    RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, October 5-9, 2015, Munich, Germany. Berlin: Springer International Publishing, 2015: 234 – 241.
    [13]
    HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 20-25, 2021. Nashville, Tennessee. New York, IEEE: 2021: 13708-13717.
    [14]
    DHEERAJ D K, FANG C, ZHENG Y, et al. Semi-supervised transfer learning-based automatic weld defect detection and visual inspection[J]. Engineering Structures, 2023, 292: 116580. doi: 10.1016/j.engstruct.2023.116580
    [15]
    BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481 − 2495. doi: 10.1109/TPAMI.2016.2644615
    [16]
    ZHAO H S, SHI J P, QI X J, et al. Pyramid Scene Parsing Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, Hawaii. New York, IEEE: 2017: 6230 − 6239.
    [17]
    GONG L X, ZHANG Y Q, ZHANG Y K, et al. Erroneous pixel prediction for semantic image segmentation[J]. Computational Visual Media, 2022, 8(1): 165 − 175.
  • Related Articles

    [1]ZHANG Yonglin, CHENG Huichao, YANG Haifeng, LIU Ning, LIANG Yuandong. Impact toughness influence law and weldability analysis of 800 MPa grade steel for photovoltaic brackets[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION. DOI: 10.12073/j.hjxb.20240428001
    [2]HUANG Yong, GUO Wei, WANG Yanlei. Effects of introductions of oxygen and nitrogen elements on impact toughness of gas pool coupled activating TIG weld metal[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2022, 43(5): 83-89. DOI: 10.12073/j.hjxb.20210919001
    [3]YAN Han, ZHAO Di, QI Tongfu, LENG Xuesong, FU Kuijun, HU Fengya. Effect of element Nb on microstructures and impact toughness of CGHAZ in TiNbV micro-alloyed steels[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2020, 41(12): 33-37. DOI: 10.12073/j.hjxb.20200906001
    [4]WANG Dongpo, LIU Kaiyue, DENG Caiyan, GONG Baoming, WU Shipin, XIAO Na. Effects of PWHT on the impact toughness and fracture toughness of the weld metal under restraint welding[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2020, 41(8): 63-67, 78. DOI: 10.12073/j.hjxb.20190914001
    [5]CAO Rui, YANG Zhaoqing, LI Jinmei, LEI Wanqing, ZHANG Jianxiao, CHEN Jianhong. Influence of fraction of coarse-grained heat affected zone on impact toughness for 09MnNiDR welded joint[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2020, 41(5): 7-13. DOI: 10.12073/j.hjxb.20190818003
    [6]LIU Zhengjun, QIN Hua, SU Yunhai, LIU Changjun, LU Yanpeng. Microstructure and low temperature impact toughness of vibration assisted welded BWELDY960Q steel[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2015, 36(8): 93-96.
    [7]DU Bing, SUN Fenglian, XU Yujun, LI Xiaoyu, LÜ Xiaochun, QIN Jian. Effect of welding methods on impact toughness of ultra-low carbon martensitic stainless steel welding wire deposited metal[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2014, 35(8): 1-4.
    [8]HU Jie, JIANG Zhizhong, HUANG Jihua, CHEN Shuhai, ZHAO Xingke, ZHANG Hua. Effects of heat treatment processes on microstructure and impact toughness of weld metal of vacuum electron beam welding on CLAM steel[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2012, (11): 67-71.
    [9]LIANG Guoli, YANG Shanwu, WU Huibin, LIU Xueli. Impact toughness of simulated CGHAZ with high heat input for adding trace Zr oil tank steel[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2011, (11): 85-88.
    [10]XUE Gang, ZHAO Fuchen, JING Yanhong, NIU Jicheng, ZHANG Yonghui, GAI Dengyu. Effect of carbon on impact toughness of metal deposited with high strength austenite electrodes[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2009, (8): 89-92.
  • Cited by

    Periodical cited type(6)

    1. 张心保,吴开明,柯睿,胡丞杨,Amir A.Shirzadil,Serhii Yershov,黄日清. 00Cr12Ni双相组织不锈钢焊接接头区域特征与性能. 焊接学报. 2023(02): 22-31+130 . 本站查看
    2. 孙语鸿,刘威,李于朋,宫文彪. 409L铁素体不锈钢搅拌摩擦焊接接头的组织及力学性能. 材料热处理学报. 2023(05): 210-216 .
    3. 董志海,李逸文,Aleksandr Babkin,常云龙. 铁素体不锈钢焊缝晶粒细化技术的研究现状. 材料导报. 2022(21): 188-197 .
    4. 李苏珊,李连胜,马青军,赵明剑,赵清宇. 热输入对T4003/Q450NQR1异种钢焊接接头组织及性能的影响. 焊接. 2020(01): 60-64+68 .
    5. 马世辉,李积元. 搅拌摩擦焊接钢接头组织与变形分析. 煤矿机械. 2020(08): 68-69 .
    6. 向登勇,肖琳嵘,邵丹丹,赵明剑,邸赫. 热输入对T4003不锈钢MAG焊接头组织及性能的影响. 机械制造文摘(焊接分册). 2019(06): 31-36 .

    Other cited types(5)

Catalog

    Article views (105) PDF downloads (67) Cited by(11)

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return