Citation: | MA Jiawei, SUN Jingbo, CHI Guanxin, ZHANG Guangjun, LI Xinlei. Weld identification and robot path planning algorithm based on stereo vision and YOLO deep learning framework[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(11): 45-49. DOI: 10.12073/j.hjxb.20240720001 |
In order to realize the teaching-free path planning of robot welding, an efficient and stable weld intelligent identification algorithm was developed by combining deep learning and point cloud processing technology. Firstly, an industrial-grade 3D camera with an eye-to-hand identification was used to obtain the two-dimensional image and 3D point cloud model around the weldment, and the pre-trained YOLOv8 target detection model was used to identify the ROI(Region of Interest) area where the weldment is located. The model identification accuracy is 99.5%, thereby realizing the rapid removal of background point clouds. Based on point cloud processing algorithms such as RANSAC plane fitting and Euclidean clustering, the three-dimensional point cloud in the ROI area was finely identified for the spatial position of the weld seam. Finally, the trajectory was converted into welding trajectories in the robotic user coordinate system with the coordinate transformation matrix. The verification experimental results show that the algorithm developed in this paper can realize the automatic identification of randomly placed weld seams and welding robot path planning, and the generated trajectory has a similar effect to the manual teaching trajectory, with a deviation of within 0.5 mm.
[1] |
吴明晖, 黄海军, 王先伟, 等. 基于改进蚁群算法的机器人焊接路径规划[J]. 焊接学报, 2018, 39(10): 113 − 118.
Wu Minghui, Huang Haijun, Wang Xianwei, et al. Robot welding path planning based on improved ant colony algorithm[J]. Transactions of the China Welding Institution, 2018, 39(10): 113 − 118.
|
[2] |
王斐, 梁宸, 韩晓光, 等. 基于焊件识别与位姿估计的焊接机器人视觉引导[J]. 控制与决策, 2020, 35(8): 1873 − 1878.
Wang Fei, Liang Chen, Han Xiaoguang, et al. Visual guidance of welding robot based on weldment recognition and pose estimation[J]. Control and Decision, 2020, 35(8): 1873 − 1878.
|
[3] |
王浩, 赵小辉, 徐龙哲, 等. 结构光视觉辅助焊接的轨迹识别与控制技术[J]. 焊接学报, 2023, 44(6): 50 − 57. doi: 10.12073/j.hjxb.20220715002
Wang Hao, Zhao Xiaohui, Xu Longzhe, et al. Research on trajectory recognition and control technology of structured light vision-assisted welding[J]. Transactions of the China Welding Institution, 2023, 44(6): 50 − 57. doi: 10.12073/j.hjxb.20220715002
|
[4] |
Zhou Peng, Peng Rui, Xu M, et al. Path planning with automatic seam extraction over point cloud models for robotic arc welding[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 5002 − 5009. doi: 10.1109/LRA.2021.3070828
|
[5] |
陈思豪, 余震, 王中任, 等. 基于结构光三维视觉角焊缝定位系统的研究[J]. 激光与红外, 2019, 49(3): 303 − 308.
Chen Sihao, Yu Zhen, Wang Zhongren, et al. Research on three-dimensional visual fillet weld positioning system based on structured light[J]. Laser & Infrared, 2019, 49(3): 303 − 308.
|
[6] |
方纬华. 基于3D视觉的机器人焊接智能轨迹规划关键技术研究[D]. 济南: 山东大学, 2023.
Fang Weihua. Research on key technologies of intelligent trajectory planning for robot welding based on 3D vision[D]. Jinan: Shandong University, 2023.
|
[7] |
Zou Yanbiao, Zhan Runqin. Research on 3D curved weld seam trajectory position and orientation detection method[J]. Optics and Lasers in Engineering, 2023, 162: 107435. doi: 10.1016/j.optlaseng.2022.107435
|
[8] |
Zhang Yuankai, Geng Yusen, Tian Xincheng, et al. Feature extraction and robot path planning method in 3D vision-guided welding for multi-blade wheel structures[J]. Optics and Lasers in Engineering, 2024, 176: 108066. doi: 10.1016/j.optlaseng.2024.108066
|
[9] |
Geng Yusen, Zhang Yuankai, Tian Xincheng, et al. A novel 3D vision-based robotic welding path extraction method for complex intersection curves[J]. Robotics and Computer-Integrated Manufacturing, 2024, 87: 102702. doi: 10.1016/j.rcim.2023.102702
|
[10] |
Ma Yunkai, Fan Junfeng, Yang Huizhen, et al. An efficient and robust complex weld seam feature point extraction method for seam tracking and posture adjustment[J]. IEEE Transactions on Industrial Informatics, 2023, 19(11): 10704 − 10715.
|
[1] | WANG Hui, XIANG Songming, RONG Youmin, HUANG Yu, XU Jiajun, WU Congyi. Non-teaching automatic welding system based on DXF file parsing and point cloud data processing[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(12): 28-35. DOI: 10.12073/j.hjxb.20230729001 |
[2] | LI Chunkai, PAN Yu, SHI Yu, WANG Wenkai, ZHAO Zhongbo. Feature information extraction of hot wire laser metal deposition process based on object and key point detection[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(11): 115-120. DOI: 10.12073/j.hjxb.20240711002 |
[3] | JIA Ruiyan, LI Haichao, WEI Fangkai, XU Yong, ZHOU Yufei. Magnetic pole weld identification and robot trajectory generation technology based on 3D point cloud[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(11): 50-54. DOI: 10.12073/j.hjxb.20240719002 |
[4] | HU Jiacheng, YU Shuangfei, ZHANG Tao, GUAN Yisheng, ZHU Haifei. Determination of welding torch space pose based on local point cloud of weld path points[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(4): 86-92. DOI: 10.12073/j.hjxb.20230406002 |
[5] | GUO Zhongfeng, LIU Junchi, YANG Junlin. Weld recognition based on key point detection method[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2024, 45(1): 88-93. DOI: 10.12073/j.hjxb.20230204001 |
[6] | LI Hexi, HAN Xinle, FANG Zaojun. A visual model of welding robot based on CNN deep learning[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2019, 40(2): 154-160. DOI: 10.12073/j.hjxb.2019400060 |
[7] | TU Zhiqiang, ZHANG Ke, YANG Chenglong, ZHU Xiaopeng, HUANG Jie. Improved ICP registration in 3-D model reconstruction[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2013, (1): 97-100. |
[8] | YU Xinghua, LIANG Zhimin, GAO Hongming, ZHANG Guangjun. Sparse point clouds fitting in 3D reconstruction for welding environment[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2007, (6): 109-112. |
[9] | WANG Hou-xiao, SUN Jun-sheng, WEI Yan-hong, ZHENG Yuan-yuan. Solution and application of energy distribution models based on point heat sources during gas metal arc welding process[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2006, (5): 27-30,34. |
[10] | FU Yu-wen, HU Sheng-sun, YI Xiao-lin, KONG Xiang-fen. Signal detection of plasma cloud and analysis of influencing factors[J]. TRANSACTIONS OF THE CHINA WELDING INSTITUTION, 2003, (5): 29-32. |