基于改进CeiT的GTAW焊接熔透状态识别方法

王颖; 高胜

doi:10.12073/j.hjxb.20230327002

基于改进CeiT的GTAW焊接熔透状态识别方法

王颖,
高胜

东北石油大学 , 大庆 , 163318

基金项目: 国家自然科学基金资助项目(61702093)；国家重点研发计划项目(2018YFE0196000)；黑龙江省自然科学基金(F2018003)；黑龙江省博士后专项(LBH-Q20077)；东北石油大学青年科学基金项目(2020QNL-10).

详细信息

作者简介:
王颖，硕士，副教授；主要研究方向为智能焊接、先进机器人工程、深度学习、人工智能等；Email: nepu_wy@163.com

中图分类号: TG 409
计量
- 文章访问数: 176
- HTML全文浏览量: 19
- PDF下载量: 44
出版历程
- 收稿日期: 2023-03-26
- 网络出版日期: 2024-02-05
- 刊出日期: 2024-04-24

Identification method of GTAW welding penetration state based on improved CeiT

Northeast Petroleum University, Daqing, 163318, China

摘要

摘要:
针对熔池信息与背景相似度高、噪声多、预测实时性差、识别精度低等问题，提出了基于改进CeiT网络模型的GTAW焊接熔透状态识别方法.首先通过MobileNetV3对Image-to-Tokens模块进行轻量化改进，提升预测的实时性能；其次设计DGCA模块改进LeFF模块来增强特征间的远程依赖关系、丰富类标记中所包含的分类信息；最后将LeFF模块中的底层特征和高层语义特征进行融合，提高模型对熔池特征的表示能力.仿真结果表明，与MobileNetV3，ResNet50，ShuffleNetV2，DeiT和CeiT模型相比，所提出的模型获得了更高的准确率和较快的检测速度.
- 图像处理 /
- 熔池 /
- 熔透状态 /
- CeiT网络 /
- 熔化极气体保护焊
Abstract:
Aiming at the problems of high similarity between melt pool information and background, much noise, poor real-time prediction and low recognition accuracy, a GTAW welding fusion state recognition method based on improved CeiT network model is proposed. First, the Image-to-Tokens module is lightened and improved by MobileNetV3 to enhance the real-time performance of prediction; second, the DGCA module is designed to improve the LeFF module to enhance the remote dependencies among features and enrich the categorical information contained in the class labels; and lastly, the fusion of the underlying features and the high-level semantic features in the LeFF module improves the model's ability to represent the features of the melt pool. Simulation experiments show that the proposed model obtains higher accuracy and faster detection speed compared with MobileNetV3, ResNet50, ShuffleNetV2, DeiT, and CeiT models.
- image processing /
- melt pool /
- penetration state /
- CeiT network /
- gas tungsten arc welding

HTML全文

图 1 CeiT网络结构

Figure 1. CeiT network structure

下载: 全尺寸图片幻灯片

图 2 改进前后的bneck结构

Figure 2. Improved bneck structure before and after. (a) bneck original structure; (b) FMCbneck structure

下载: 全尺寸图片幻灯片

图 3 Focus切片采样原理

Figure 3. Focus slice sampling principle

下载: 全尺寸图片幻灯片

图 4 原始LeFF结构图

Figure 4. Original LeFF structure diagram

下载: 全尺寸图片幻灯片

图 5 改进的LeFF结构图

Figure 5. Improved LeFF structure diagram

下载: 全尺寸图片幻灯片

图 6 DGCA模块

Figure 6. DGCA Module

下载: 全尺寸图片幻灯片

图 7 CA注意力模块结构图

Figure 7. CA attention module structure diagram

下载: 全尺寸图片幻灯片

图 8 图像采集平台

Figure 8. Image acquisition platform

下载: 全尺寸图片幻灯片

图 9 三种焊接熔池

Figure 9. Three types of welding pools. (a) not melted through; (b) melt through; (c) burn through

下载: 全尺寸图片幻灯片

图 10 ROI提取结果

Figure 10. ROI extraction results. (a) original image; (b) image after ROI extraction

下载: 全尺寸图片幻灯片

图 11 样本数据增强效果

Figure 11. Sample data enhancement effect. (a) fusion state; (b) burn-through condition

下载: 全尺寸图片幻灯片

图 12 学习率对比结果

Figure 12. Learning rate comparison results. (a) validation set loss value curve; (b) validation set accuracy curve

下载: 全尺寸图片幻灯片

图 13 组合改进策略的模型迭代曲线

Figure 13. Model iteration curves for combined improvement strategies. (a) Iteration curves of loss values on the training set for different improvement schemes; (b) Iteration curves of the accuracy of different improvement schemes on the validation set

下载: 全尺寸图片幻灯片

图 14 各对比模型在验证集上准确率的迭代曲线

Figure 14. Iteration curves of the accuracy of each comparison model on the validation set

下载: 全尺寸图片幻灯片

表 1 MobileNetV3网络参数

Table 1 MobileNetV3 network parameters

序号号	输入层	操作	步长	exp size	#out	SE	激活函数
1	224² × 3	Conv2d	2	—	16	—	h-swish
2	112² × 16	FMCbneck,3 × 3	1	16	16	—	ReLU
3	112² × 16	bneck,3 × 3	2	64	24	—	ReLU
4	56² × 24	FMCbneck,3 × 3	1	72	24	—	ReLU
5	56² × 24	bneck,3 × 3	2	72	40	—	ReLU
6	28² × 40	FMCbneck,3 × 3	1	120	40	—	ReLU
7	28² × 40	FMCbneck,3 × 3	1	120	40	—	ReLU
8	28² × 40	bneck,3 × 3	2	240	80	—	h-swish
9	14² × 80	FMCbneck,5 × 5	1	200	80	√	h-swish
10	14² × 80	FMCbneck,5 × 5	1	184	80	√	h-swish
11	14² × 80	FMCbneck,5 × 5	1	184	80	√	h-swish
12	14² × 80	FMCbneck,5 × 5	1	480	112	√	h-swish

下载: 导出CSV

表 2 扩充后各数据集数量(张)

Table 2 Number of each data set after expansion

数据集	未熔透	熔透	烧穿	总和
训练集	5899	6053	5263	17215
验证集	738	757	658	2153
测试集	738	757	658	2153
总和	7375	7567	6579	21521

下载: 导出CSV

表 3 未数据增强试验结果

Table 3 No data enhancement test results

熔透类别	未熔透	正常熔透	烧穿
未熔透	82.5	12.5	5.0
正常熔透	3.0	96.0	1.0
烧穿	15.0	5.0	80
注：12.5表示未熔透状态的数据被预测为正常熔透的概率.

下载: 导出CSV

表 4 数据增强试验结果

Table 4 Data enhancement test results

熔透类别	未熔透	正常熔透	烧穿
未熔透	96.5	2.0	1.5
正常熔透	2.5	96.5	1.0
烧穿	1.0	2.0	97

下载: 导出CSV

表 5 消融试验结果

Table 5 Ablation test results

方案	I2T模块改进		LeFF模块改进		精确率			召回率			F1分数			准确率			训练时间 t/h
方案	M3	BNSA	DG	FI	C0	C1	C2	C0	C1	C2	C0	C1	C2	C0	C1	C2	训练时间 t/h
1	—	—	—	—	85.1	86.9	85.3	80.6	81.3	80.0	84.6	87.3	84.5	87.5	87.9	86.8	4.07
2	Π	—	—	—	81.2	83.7	81.9	74.7	80.7	79.2	81.1	83.8	80.9	82.1	84.3	83.5	3.23
3	Π	Π	—	—	91.2	92.3	92.0	87.8	88.8	88.6	92.5	93.3	92.9	90.3	93.8	91.4	3.58
4	—	—	Π	—	90.6	91.6	91.4	87.4	89.0	86.7	90.6	92.4	91.0	90.8	92.4	91.3	4.94
5	—	—	Π	Π	91.3	92.5	92.1	87.7	89.5	87.6	91.7	91.6	92.2	91.7	92.6	92.4	5.16
6	Π	Π	Π	Π	94.6	95.4	94.9	91.0	92.7	90.7	94.3	95.5	94.3	95.9	96.0	95.8	3.76

下载: 导出CSV

表 6 不同模型试验结果

Table 6 Test results of different models

模型	精确率	召回率	F1 分数	准确率	训练时间t/h	模型内存占用量 δ/MB	识别单个样本平均时间t'/s
MobileNetV3	82.2	80.1	81.3	85.1	3.24	17.6	0.16
ResNet50	90.4	89.3	90.0	94.3	6.18	179	0.78
ShuffleNetV2	85.1	83.5	84.8	90.2	3.04	9.85	0.11
DeiT	75.5	73.3	74.2	80.3	3.27	35.8	0.35
tttCeiT	82.9	80.7	81.0	87.8	4.07	44.8	0.47
Improved-CeiT	92.7	90.2	91.6	95.4	3.76	27.7	0.24

下载: 导出CSV

参考文献(22)

[1]	Wang Y M, Han J, Lu J, et al. TIG stainless steel molten pool contour detection and weld width prediction based on res-seg[J]. Metals-Open Access Metallurgy Journal, 2020, 10(11): 1495.
[2]	Jiao W, Wang Q, Cheng Y, et al. Prediction of weld penetration using dynamic weld pool arc images[J]. Welding Journal, 2020, 99(11): 295s − 302s. doi: 10.29391/2020.99.027
[3]	Kim H, Nam K, Oh S, et al. Deep-learning-based real-time monitoring of full-penetration laser keyhole welding by using the synchronized coaxial observation method[J]. Journal of Manufacturing Processes, 2021, 68(8): 1018 − 1030.
[4]	Wu J Z, Shi J W, Gao Y F, et al. Penetration recognition in GTAW welding based on time and spectrum images of arc sound using deep learning method[J]. Metals, 2022, 12(9): 1549. doi: 10.3390/met12091549
[5]	Ma G H, Yu L S, Yuan H T, et al. A vision-based method for lap weld defects monitoring of galvanized steel sheets using convolutional neural network[J]. Journal of Manufacturing Processes, 2021, 64: 130 − 139. doi: 10.1016/j.jmapro.2020.12.067
[6]	Chang B X, Huang J Y. Discrimination of molten pool penetration based on genetic algorithm optimization of BP neural network[J]. Journal of Physics Conference Series, 2020, 1437: 012110. doi: 10.1088/1742-6596/1437/1/012110
[7]	陈宸, 周方正, 李成龙, 等. 融合空间和通道特征的等离子弧焊熔池熔透状态预测方法[J]. 焊接学报, 2023, 44(4): 30 − 38. doi: 10.12073/j.hjxb.20220516001 Chen Chen, Zhou Fangzheng, Li Chenglong, et al. Prediction method of plasma arc welding molten pool melting state based on spatial and channel characteristics[J]. Transactions of the China Welding Institution, 2023, 44(4): 30 − 38. doi: 10.12073/j.hjxb.20220516001
[8]	王颖, 高胜, 吴立明. 基于胶囊网络的TIG熔透预测[J]. 焊接, 2023(4): 15 − 20,28. doi: 10.12073/j.hj.20220425003 Wang Ying, Gao Sheng, Wu Liming. TIG penetration prediction based on capsule network[J]. Welding & Joining, 2023(4): 15 − 20,28. doi: 10.12073/j.hj.20220425003
[9]	Gao Y F, Wang Q S, Xiao J H, et al. Weld penetration identification with deep learning method based on auditory spectrum images of arc sounds[J]. Welding in the World, 2022, 66(12): 2509 − 2520. doi: 10.1007/s40194-022-01373-7
[10]	Yu R W, He H Y, Han J, et al. Monitoring of back bead penetration based on temperature sensing and deep learning[J]. Measurement, 2022, 188. DOI: 10.1016/J.MEASUREMENT.2021.110410.
[11]	段明瑞. 基于深度学习的乏燃料池不锈钢GTAW焊接质量在线监测[D]. 哈尔滨: 哈尔滨工业大学, 2021. Duan Mingrui. On-line monitoring of welding quality of stainless steel GTAW in spent fuel pool based on deep learning [D]. Harbin : Harbin Institute of Technology, 2021.
[12]	Liu S K , Wu D, Luo Z Y , et al. Measurement of pulsed laser welding penetration based on keyhole dynamics and deep learning approach[J]. Measurement, 2022, 199. DOI:10.1016/J. MEASUREMENT.2022.111579.
[13]	Nomura K, Fukushima K, Matsumura T, et al. Burn-through prediction and weld depth estimation by deep learning model monitoring the molten pool in gas metal arc welding with gap fluctuation[J]. Journal of Manufacturing Processes, 2020. DOI: 10.1016/j.jmapro.2020.10.019.
[14]	Xia C Y, Pan Z X, Fei Z Y, et al. Vision based defects detection for Keyhole TIG welding using deep learning with visual explanation[J]. Journal of Manufacturing Processes, 2020, 56(8): 845 − 855.
[15]	刘天元, 鲍劲松, 汪俊亮, 等. 融合时序信息的激光焊接熔透状态识别方法[J]. 中国激光, 2021, 48(6): 228 − 238. Liu Tianyuan, Bao Jinsong, Wang Junliang, et al. Laser welding penetration state recognition method fused with timing information[J]. Chinese Journal of Lasers, 2021, 48(6): 228 − 238.
[16]	卢振洋, 宫兆辉, 闫志鸿, 等. 基于深度学习的TIG焊背部熔池检测和熔宽提取[J]. 北京工业大学学报, 2020, 46(9): 988 − 996. doi: 10.11936/bjutxb2018110030 Lu Zhenyang, Gong Zhaohui, Yan Zhihong, et al. Deep learning based detection and width extraction of back molten pool in TIG welding[J]. Journal of Beijing University of Technology, 2020, 46(9): 988 − 996. doi: 10.11936/bjutxb2018110030
[17]	Wang Z M, Li L Y, Chen H Y, et al. Recognition of GTAW weld penetration based on the lightweight model and transfer learning[J]. Welding in the World, 2022, 67(1): 251 − 264.
[18]	Li C, Wang Q, Jiao W, et al. Deep learning-based detection of penetration from weld pool reflection images[J]. Welding Journal, 2020, 99(9): 239s − 245s. doi: 10.29391/2020.99.022
[19]	Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 10012 − 10022.
[20]	Liang S J, Yu M X, Lu W S, et al. A lightweight vision transformer with symmetric modules for vision tasks[J]. Intelligent Data Analysis, 2023, 27(6): 1741 − 1757. doi: 10.3233/IDA-227205
[21]	Yuan K, Guo S, Liu Z, et al. Incorporating convolution designs into visual transformers[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021: 559 − 568.
[22]	Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 1314-1324.

施引文献(19)

期刊类型引用(12)

1.	邹阳，魏巍，范悦，王泽震，王强，赵亮. 铝合金搅拌摩擦焊工艺研究进展. 热加工工艺. 2024(03): 7-13 . 百度学术
2.	陈平. 2系铝合金的搅拌摩擦焊接接头微观组织与力学性能研究. 北京印刷学院学报. 2024(03): 32-37 . 百度学术
3.	王浡婳，张立杰. 铝合金搅拌摩擦焊接头微观组织和力学性能分析. 精密成形工程. 2023(01): 94-100 . 百度学术
4.	许辉，刘宽，徐耀钟，于文凯，刘婷，王笑含，徐雪华. 2A14铝合金FSW焊缝背部线状缺陷返修工艺. 电焊机. 2023(03): 117-123 . 百度学术
5.	马领航，李波，赵彦广，宋建岭，高世康，李雨，许子彦，周利. 火箭贮箱焊接缺陷修复技术研究现状. 电焊机. 2023(03): 31-45 . 百度学术
6.	李晓华，齐影，郑瑞娟，武媛，祝弘滨，崔雷. QT400球墨铸铁摩擦塞焊接头的微观组织和力学性能研究. 热加工工艺. 2023(05): 141-144 . 百度学术
7.	鲁克锋，殷凤仕，王文宇，滕涛，樊世冲，刘亚凡，王鸿琪，朱建，任智强. 铝合金搅拌摩擦焊接头缺陷及焊件结构问题控制策略的研究进展. 表面技术. 2023(07): 55-79 . 百度学术
8.	丁清伟，汪春能，眭怀明，赵引红，陈冬梅. 铝合金机匣预埋管处渗漏机理分析及解决措施. 铸造技术. 2023(09): 873-876 . 百度学术
9.	李德福，王希靖. 6082铝合金摩擦塞补焊接头焊核区晶体特征. 兰州理工大学学报. 2022(03): 7-12 . 百度学术
10.	高彦军，刘西伟，刘旭升，邵震，崔雷. 2060-T8铝锂合金顶锻式摩擦塞补焊接头组织性能研究. 电焊机. 2022(07): 69-75+99 . 百度学术
11.	赵慧慧，高焓，胡蓝，董吉义，尹玉环，崔雷. 2219铝合金薄板拉拔式摩擦塞焊工艺及力学性能优化. 焊接. 2021(06): 48-55+64 . 百度学术
12.	胡永鹅，龙琼，韩兴科，何波，王尧，杨秀芳. 铝合金材料焊接方法研究进展. 贵州农机化. 2020(03): 15-19 . 百度学术