受限解空间下粗粒度正则化的熔池边缘自适应检测方法

刘天元; 鲍劲松; 汪俊亮; 郑小虎; 王佳铖

doi:10.12073/j.hjxb.20200815002

受限解空间下粗粒度正则化的熔池边缘自适应检测方法

东华大学，上海，201600

基金项目: 国家自然科学基金资助项目(51805079)；国家自然科学基金资助项目(51905091)；上海市自然科学基金资助项目(17ZR1400600)；中央高校基本科研业务费专项资金、东华大学研究生创新基金(CUSF-DH-D-2020053).

详细信息

作者简介:
刘天元，1992年出生，博士；主要研究方向为焊接自动化，图像处理，深度学习，人工智能；Email：tyliu@mail.dhu.edu.cn

通讯作者:
鲍劲松，教授，博士研究生导师；Email：bao@dhu.edu.cn.

中图分类号: TG 403
计量
- 文章访问数: 855
- HTML全文浏览量: 191
- PDF下载量: 24
出版历程
- 收稿日期: 2020-08-14
- 网络出版日期: 2020-12-27
- 刊出日期: 2021-02-01

Adaptive edge detection of molten pool based on coarse-grained regularization in restricted solution space

College of Mechanical Engineering, Donghua University, Shanghai 201600, China

摘要

摘要: 边缘检测是熔池图像处理的关键步骤. 鉴于熔池区域弧光变化剧烈，依赖人为设置阈值的边缘检测方法难以适应弧光的动态变化过程，文中提出了一种基于深度学习的熔池边缘提取模式. 首先对原始熔池图像进行像素级标注和数据增广以构建数据集；其次结合先验知识提出了一种受限解空间下的粗粒度正则化方法(coarse grained regularization method in restricted solution space, CGRRSS)以增强边缘特征；最后从定量和定性两方面将所提方法与传统方法进行了对比. 结果表明，所提方法对边缘点的召回率较高，所获熔池边缘更连续且对伪边缘具有更好的抑制作用. 单幅图像检测时间6.2 ms，可满足在线监测需求.
- 熔池图像 /
- 深度学习 /
- 边缘检测 /
- 正则化 /
- 粗粒度
Abstract: Edge detection is a key step in the image processing of the molten pool. In view of the sharp changes in the arc in the molten pool area, the edge detection method that relies on artificially setting the threshold is difficult to adapt to the dynamic change of the arc. This paper proposes a deep learning-based edge extraction mode of the molten pool. Firstly, pixel-level annotation and data augmentation are performed on the original molten pool image to build a dataset. Secondly, a coarse-grained regularization method in restricted solution space (CGRRSS) is proposed to enhance edge features. Finally, the proposed method is compared with the traditional methods in both quantitative and qualitative aspects. The results show that the proposed method has a higher recall of edge points, the obtained molten pool edge is more continuous and has a better suppression effect on false edges. The detection time of a single image is 6.2 ms, which can meet the needs of online monitoring.
- molten pool image /
- deep learning /
- edge detection /
- regularization /
- coarse-grained

HTML全文

图 1 基于深度学习的熔池图像边缘检测总体架构

Figure 1. Overall architecture of edge detection of the molten pool image based on deep learning

下载: 全尺寸图片幻灯片

图 2 焊接机器人系统

Figure 2. Welding robot system

下载: 全尺寸图片幻灯片

图 3 数据集构建过程

Figure 3. Construction process of the dataset

下载: 全尺寸图片幻灯片

图 4 CGRRSS方法流程图

Figure 4. Flowchart of the CGRRSS method

下载: 全尺寸图片幻灯片

图 5 训练过程图

Figure 5. Training process. (a) HED; (b) methods of this paper

下载: 全尺寸图片幻灯片

图 6 不同方法测试结果对比

Figure 6. Comparison of test results of different methods

下载: 全尺寸图片幻灯片

表 1 不同方法在测试集上的评估结果

Table 1 evaluation results of different methods on test set

方法	ACC (%)	PRE (%)	REC (%)	F1-score (%)	测试时间 t/ms
Sobel	98.7	5.1	9.4	6.5	12.3
Canny	98.9	8.1	10.5	7.1	7.8
Prewitt	97.1	6.8	38.7	11.1	7.1
形态学	95.7	5.6	49.5	9.7	10.1
HED	98.1	14.9	58.4	22.6	5.3
文中方法	95.9	9.3	78.4	16.3	6.2

下载: 导出CSV

参考文献(17)

[1]	侯震, 许燕玲, 黄色吉, 等. 视觉传感技术在机器人焊接中的应用[J]. 上海交通大学学报, 2016, 50(S1): 55 − 58. Hou Zhen, Xu Yanling, Huang Seji, et al. Application status of vision sensing used in robotic welding[J]. Journal of Shanghai Jiaotong University, 2016, 50(S1): 55 − 58.
[2]	Liu Y K, Zhang Y M, Kvidahl L. Skilled human welder intelligence modeling and control[J]. Welding Journal, 2014, 93: 46 − 52.
[3]	刘祖明. 受控脉冲穿孔等离子弧焊接背面小孔动态行为的视觉检测与控制[D]. 济南: 山东大学, 2013. Liu Zuming. Vision sensing and control of the backside keyhole dynamic behaviors in controlled-pulse keyholing plasma arc welding process[D]. Jinan: Shandong University, 2013.
[4]	陈子琴, 高向东, 王琳. 大功率盘形激光焊焊缝背面宽度预测[J]. 光学精密工程, 2017, 25(9): 2524 − 2531. doi: 10.3788/OPE.20172509.2524 Chen Ziqin, Gao Xiangdong, Wang Lin. Weld width prediction of weldment bottom surface in high-power disk laser welding[J]. Optics and Precision Engineering, 2017, 25(9): 2524 − 2531. doi: 10.3788/OPE.20172509.2524
[5]	黄军芬, 薛龙, 黄继强, 等. 基于视觉传感的GMAW熔透状态预测[J]. 机械工程学报, 2019, 55(17): 41 − 47. doi: 10.3901/JME.2019.17.041 Huang Junfen, Xue Long, Huang Jiqiang, et al. GMAW penetration state prediction based on visual sensing[J]. Journal of Mechanical Engineering, 2019, 55(17): 41 − 47. doi: 10.3901/JME.2019.17.041
[6]	方吉米, 王克鸿, 黄勇. 高速GMAW驼峰焊道形成过程熔池图像识别[J]. 焊接学报, 2019, 40(2): 42 − 46. Fang Jimi, Wang Kehong, Huang Yong. Weld pool image recognition of humping formation process in high speed GMAW[J]. Transactions of the China Welding Institution, 2019, 40(2): 42 − 46.
[7]	李静, 李芳, 朱伟, 等. 基于被动光的MAG管道打底焊焊缝边缘提取方法[J]. 焊接学报, 2011, 32(10): 69 − 72. Li Jing, Li Fang, Zhu Wei, et al. A new seam location extraction method for pipe-line backing welding of MAG based on passive optical vision sensor[J]. Transactions of the China Welding Institution, 2011, 32(10): 69 − 72.
[8]	董瑶, 李伟超, 刘今越, 等. 用于航空发动机叶片焊接修复的改进Canny算法[J]. 焊接学报, 2018, 39(1): 37 − 40. Dong Yao, Li Weichao, Liu Jinyue, et al. An improved Canny operator used for aero-engine's blade welding and repairing[J]. Transactions of the China Welding Institution, 2018, 39(1): 37 − 40.
[9]	桂楷欽. 焊缝缺陷射线数字成像参考图像研究[D]. 南昌: 南昌航空大学, 2019. Gui Kaiqin. Research on digital reference image of digital radiography for weld defect[D]. Nanchang: Nanchang Hangkong University, 2019.
[10]	刘晓刚, 闫红方, 张荣. 基于形态学多尺度多结构的熔池图像边缘检测[J]. 热加工工艺, 2019, 48(5): 216 − 219. Liu Xiaogang, Yan Hongfang, Zhang Rong. Edge detection of molten pool image based on morphology multi-scale and multi-structuring elements[J]. Hot Working Technology, 2019, 48(5): 216 − 219.
[11]	Xie S N, Tu Z W. Holistically-Nested edge detection[J]. International Journal of Computer Vision, 2017, 125(1−3): 3 − 18. doi: 10.1007/s11263-017-1004-z
[12]	Liu Y, Cheng M, Hu X, et al. Richer convolutional features for edge detection[C]//IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017: 5872−5881.
[13]	Yu Z, Feng C, Liu M, et al. CASENet: Deep category-aware semantic edge detection[C]//IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017: 1761−1770.
[14]	Bertasius G, Shi J, Torresan L. DeepEdge: A multi-scale bifurcated deep network for top-down contour detection[C]//IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 4380−4389.
[15]	Simard P Y, Steinkraus D, Platt J C. Best practices for convolutional neural networks applied to visual document analysis[C]//International Conference on Document Analysis and Recognition, Edinburgh, UK, 2003: 958 − 963.
[16]	Zhang H, Cisse M, Dauphin Y N, et al. Mixup: Beyond empirical risk minimization[C]//International Conference on Learning Representations, Vancouver, BC, Canada, 2018: 1−13.
[17]	Goodfellow I J, Pouget A J, Mirza M, et al. Generative adversarial nets[C]//International Conference on Neural Information Processing Systems, Montréal, Canada, 2014: 2672−2680.

施引文献(18)

期刊类型引用(12)

1.	周浩南，孙文磊，王伟，张志虎. 面向涂层裂纹的激光熔覆预测模型研究. 热加工工艺. 2024(14): 27-32 . 百度学术
2.	郑世茂，刘玉国，王豪，王新佩，陈洪堂. 长直焊缝自动焊接设备研究. 南方农机. 2023(10): 127-128+154 . 百度学术
3.	王颖，高胜，吴立明. 基于胶囊网络的TIG熔透预测. 焊接. 2023(04): 15-20+28 . 百度学术
4.	黄威威，游德勇，高向东，张艳喜，黄宇辉. 基于相关分析和神经网络的激光焊接稳态识别. 激光技术. 2022(03): 312-319 . 百度学术
5.	吴月玉，张弓，林群煦，侯至丞，杨文林. 机器人TIG焊接的焊缝形貌遗传神经网络预测. 制造业自动化. 2022(07): 86-90 . 百度学术
6.	刘秀航，黄宇辉，张艳喜，高向东. 基于BP神经网络补偿卡尔曼滤波的激光-MIG复合焊缝熔宽在线检测. 中国激光. 2022(16): 115-121 . 百度学术
7.	陶永，兰江波，任帆，王田苗，江山，高赫，温宇方. 基于自适应模糊神经网络的机器人焊接焊缝外形预测方法. 计算机集成制造系统. 2022(11): 3643-3651 . 百度学术
8.	刘天元，鲍劲松，汪俊亮，顾俊. 融合时序信息的激光焊接熔透状态识别方法. 中国激光. 2021(06): 228-238 . 百度学术
9.	吴月玉，张弓，林群煦，侯至丞，杨文林，刘胜祥，徐群华，张雨航. 焊接机器人特征参数预测方法的研究综述与展望. 机床与液压. 2021(15): 168-173+199 . 百度学术
10.	成慧翔，马艳娥，李新卫. 基于改进神经网络的激光焊接偏差智能识别研究. 激光杂志. 2021(12): 165-169 . 百度学术
11.	火巧英，闫海宁，涂本荣，陆安进. 焊接工艺参数对Q345NQR2耐候钢激光焊焊缝成形的影响. 焊接技术. 2020(08): 16-18+105-106 . 百度学术
12.	范鹏飞，张冠. 基于线性回归和神经网络的金属陶瓷激光熔覆层形貌预测. 表面技术. 2019(12): 353-359+368 . 百度学术