Arc welding molten pool image recognition based on attention mechanism and transfer learning

HONG Yuxiang; YING Qiluo; LIN Kai; WANG Kaiming; WANG Yaoqi

doi:10.12073/j.hjxb.20240112003

TRANSACTIONS OF THE CHINA WELDING INSTITUTION > 2025 > 46(4): 94-102. > DOI: 10.12073/j.hjxb.20240112003

HONG Yuxiang, YING Qiluo, LIN Kai, WANG Kaiming, WANG Yaoqi. Arc welding molten pool image recognition based on attention mechanism and transfer learning[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2025, 46(4): 94-102. DOI: 10.12073/j.hjxb.20240112003

Citation:

PDF (17702 KB)

Arc welding molten pool image recognition based on attention mechanism and transfer learning

1.
College of Mechanical and Electrical Engineering, China Jiliang University, Hangzhou, 310018, China
2.
Key Laboratory of Intelligent Manufacturing Quality Big Data Tracing and Analysis of Zhejiang Province, China Jiliang University, Hangzhou, 310018, China
3.
College of Mechanical and Vehicle Engineering, Changsha University of Science & Technology, Changsha, 410114, China

More Information

Received Date: January 11, 2024
Available Online: March 25, 2025

Graphical Abstract

Abstract

Abstract

Due to the influence of complex time-varying interference and variations in process conditions during the welding process, the boundary characteristics of the molten pool are easy to be blurred, and the scale information is complex and changeable, which poses significant challenges to the accuracy recognition and the robust segmentation of the molten pool. In this paper, a molten pool image recognition method combining attention mechanism and transfer learning is proposed. Firstly, the residual block(RB) is added to the UNet down-sampling process to extract multi-scale low-level features, and the coordinate attention block(CAB) is introduced in the down-sampling and up-sampling processes to improve the feature weight of the effective region. Secondly, the pre-trained deep convolutional neural network in Pascal VOC2012 is transferred to the UNet network to realize feature transfer and parameter sharing, so as to alleviate the over-reliance of training effect on datasets. The TL-RCUNet network proposed in this paper has achieved good recognition results on the untrained MAG and TIG cross-process datasets. The mean intersection over union(MIoU) reaches 96.21% and 79.55%, respectively, which is about 15% and 25% higher than the classical semantic segmentation network. The model provides a feasible solution to the problem that existing semantic segmentation methods rely on a large number of training samples and pixel-level annotations based on expert experience.
- semantic segmentation,
- molten pool image,
- transfer learning,
- attention mechanism,
- residual network

FullText(HTML)

References (17)

References

[1]	HONG Y X, JIANG Y X, YANG M X, et al. Intelligent seam tracking in foils joining based on spatial-temporal deep learning from molten pool serial images[J]. Robotics and Computer-Integrated Manufacturing, 2025, 91: 102840. doi: 10.1016/j.rcim.2024.102840
[2]	HONG Y X, YANG M X, JIANG Y X, et al. Real-time quality monitoring of ultrathin sheets edge welding based on microvision sensing and SOCIFS-SVM[J]. IEEE Transactions on Industrial Informatics, 2022, 19(4): 5506 − 5516.
[3]	张志芬, 陈善本, 张裕明, 等. 焊接智能化监测技术研究现状与展望[J]. 焊接学报, 2024, 45(11): 10 − 20,70. ZHANG Zhifen, CHEN Shanben, ZHANG Yuming, et al. Research progress and prospect of welding intelligent monitoring technology[J]. Transactions of the China Welding Institution, 2024, 45(11): 10 − 20,70.
[4]	徐远钊, 罗玖田, 方乃文, 等. 基于MS-FCM算法的船体板熔池图像处理技术[J]. 焊接学报, 2024, 45(3): 82 − 90. doi: 10.12073/j.hjxb.20231010001 XU Yuanzhao, LUO Jiutian, FANG Naiwen, et al. Image processing technology for ship plate melt pool based on MS-FCM algorithm[J]. Transactions of the China Welding Institution, 2024, 45(3): 82 − 90. doi: 10.12073/j.hjxb.20231010001
[5]	HONG Y X, YANG M X, CHANG B H, et al. Filter-PCA-based process monitoring and defect identification during climbing helium arc welding process using DE-SVM[J]. IEEE Transactions on Industrial Electronics, 2022, 70(7): 7353 − 7362.
[6]	李巍, 李太江, 杨略, 等. 改进的U-Net算法在管道内焊缝缺陷图像分割中的应用[J]. 焊接, 2024(11): 73 − 80. LI Wei, LI Taijiang, YANG Lue, et al. Application of improved U-Net algorithm in image segmentation of pipeline inner weld defect[J]. Welding & Joining, 2024(11): 73 − 80.
[7]	YANG D Y, DAI P H, CUI S W, et al. Real-time recognition of molten pools based on improved DeepLabV3 + in keyhole tungsten inert gas welding applications[J]. Electronics, 2024, 13(2): 283. doi: 10.3390/electronics13020283
[8]	QI J Y , WU Y F. Strip steel surface defect detection algorithm based on improved Faster R-CNN[J]. China Welding, 2024, 33(2): 11 − 22.
[9]	马晓锋, 夏攀, 刘海生, 等. 全位置焊接熔池的深度学习检测方法[J]. 机械工程学报, 2023, 59(12): 272 − 283. doi: 10.3901/JME.2023.12.272 MA Xiaofeng, XIA Pan, LIU Haisheng, et al. Depth learning detection method for all-position weld pool[J]. Journal of Mechanical Engineering, 2023, 59(12): 272 − 283. doi: 10.3901/JME.2023.12.272
[10]	LI T P, CAO Y, ZHANG Y M. Analysis of weld pool region constituents in GMAW for dynamic reconstruction through characteristic enhancement and LSTM U-Net networks[J]. Journal of Manufacturing Processes, 2024, 127: 573 − 588. doi: 10.1016/j.jmapro.2024.07.084
[11]	BACIOIU D, MELTON G, PAPAELIAS M, et al. Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning[J]. NDT & E International, 2019, 107: 102139.
[12]	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, October 5-9, 2015, Munich, Germany. Berlin: Springer International Publishing, 2015: 234 – 241.
[13]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 20-25, 2021. Nashville, Tennessee. New York, IEEE: 2021: 13708-13717.
[14]	DHEERAJ D K, FANG C, ZHENG Y, et al. Semi-supervised transfer learning-based automatic weld defect detection and visual inspection[J]. Engineering Structures, 2023, 292: 116580. doi: 10.1016/j.engstruct.2023.116580
[15]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481 − 2495. doi: 10.1109/TPAMI.2016.2644615
[16]	ZHAO H S, SHI J P, QI X J, et al. Pyramid Scene Parsing Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, Hawaii. New York, IEEE: 2017: 6230 − 6239.
[17]	GONG L X, ZHANG Y Q, ZHANG Y K, et al. Erroneous pixel prediction for semantic image segmentation[J]. Computational Visual Media, 2022, 8(1): 165 − 175.

[1]	LIU Lijun, FENG Mengkui, WANG Xiaolu, CUI Yuanbiao, JIA Zhixin, LI Jiqiang. Microstructure analysis of laser cladding strengthening layer on H13 die steel surface assisted by ultrasonic[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2021, 42(6): 85-90, 96. DOI: 10.12073/j.hjxb.20200503001
[2]	YIN Yan, KANG Ping, LU Chao, ZHANG Yuan, ZHANG Ruihua. Microstructure and microhardness analysis of laser welded dissimilar steel[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2020, 41(7): 71-77. DOI: 10.12073/j.hjxb.20191227002
[3]	YIN Yan, PAN Cunliang, ZHAO Chao, ZHANG Ruihua, QU Yuebo. Formation mechanism of microstructure of laser cladding high chromium Fe-based alloy and its effect on microhardness[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2019, 40(7): 114-120. DOI: 10.12073/j.hjxb.2019400192
[4]	NIE Xin1, ZHU Weiqiang1, CAI Hongfeng2, CHEN Huanhuan1. Optimization of resistance spot welding parameters based on load of welding spot[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2018, 39(5): 114-120. DOI: 10.12073/j.hjxb.2018390135
[5]	YANG Shuo, CHANG Baohua, XING Bin, DU Dong. Influences of forced cooling on the microstructure and microhardness in laser metal deposition of IC10 super alloy[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2018, 39(3): 31-35. DOI: 10.12073/j.hjxb.2018390063
[6]	WU Xiangyang, ZHANG Zhiyi, QI Weichuang, TIAN Renyong, SHI Chunyuan. Optimization of narrow groove plasma-MAG hybrid welding process parameters[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2017, 38(5): 116-119. DOI: 10.12073/j.hjxb.20170526
[7]	JIANG Wei, WANG Lifeng. Stress analysis and structure optimization of copper cylinders in 3D packaging[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2017, 38(3): 112-116.
[8]	HONG Bo, LI Zhenkai, LIU Jian, DAI Jiangping. Method of optimizing magnetic-control arc welding sensor parameters based on Kriging surrogate model[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2017, 38(2): 1-4,9.
[9]	YIN Yan, LI Zilin, XU Guangwei, ZHANG Ruihua, QU Yuebo. Microhardness and microstructure of laser cladding layer on 3Cr13 kitchen knife by disc laser coaxial powder[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2016, 37(10): 85-88.
[10]	QI Yongai, ZHAO Jianfeng, XIE Deqiao, LI Yue. Fining grain of FGH95 nickel-based superalloy laser cladding layer by ultrasonic impact treatment[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2015, 36(3): 59-62.

Cited By

Cited by

Periodical cited type(7)

1.	林江红，杜茂华，潘传旭. 基于数学统计综合分析方法的H13模具钢表面激光熔覆工艺参数优化. 中国激光. 2025(08): 119-132 .
2.	叶超，侯亮，陈云，徐杨，刘文志，王振忠. 316L不锈钢激光熔覆宏微观特征优化. 焊接学报. 2023(03): 8-16+129 . 本站查看
3.	胡言峰，杜彦斌，许磊，周志杰，舒林森. 响应面法与粒子群算法集成的激光熔覆工艺参数优化方法. 机械科学与技术. 2023(04): 629-637 .
4.	赵菲，刘子敬，张杰，吴志生. 超细VC对激光熔覆H13合金显微组织和耐磨性的影响. 表面技术. 2022(02): 232-240 .
5.	胡中潮，唐永东，高忠玉，陈湖演，蔡健文，邓天德. H13模具钢铝合金工件生产应用现状综述. 模具制造. 2022(02): 81-85 .
6.	董月，舒林森. 离焦量对MoS_2改性Fe-Cr-Mo-Si合金涂层组织与性能的影响. 金属热处理. 2022(05): 246-251 .
7.	崔少伟，王树奇，姜伟，刘喜艳. 不同熔覆参数下的AlFeCrCoNiTi高熵合金涂层的高温摩擦磨损性能. 钢铁钒钛. 2021(03): 155-161+192 .

Other cited types(5)

Get Citation

PDF

XML

Article views (77) PDF downloads (54) Cited by(12)

Turn off MathJax

Article Contents

Abstract

References

Arc welding molten pool image recognition based on attention mechanism and transfer learning