To address the problems that traditional detection methods were difficult to conduct three-dimensional quantitative analysis inside workpieces, a binocular vision detection method was proposed based on multi-threaded semi-global stereo matching(M-SGSM). Firstly, the Zhang's calibration method was used to obtain the parameters of the binocular endoscope, and image distortion correction and epipolar rectification were performed. Secondly, a multi-threaded regional overlap segmentation strategy was designed to optimize the stereo matching algorithm, improve the efficiency of disparity map calculation, and generate a 3D color point cloud. Finally, a self-calibration area measurement method combined with Euclidean distance calculation was proposed to realize three-dimensional quantitative analysis. Experimental results show that the optimized stereo matching algorithm improves computational efficiency by approximately 30%, the generated 3D point cloud has a clear structure, the Euclidean distance measurement error is less than 3%, and the area measurement error is less than 1.5%. This method provides an efficient and high-precision solution for three-dimensional internal detection of workpieces.
双目内窥镜作为一种新型的无损测量仪器,在高温高压、有毒有害等恶劣工况下的狭小空间中具有良好的性能,因此双目内窥镜已成为工业检测中非常重要的成像工具。在工业领域中,双目内窥镜可在复杂铸件、涡轮、齿轮箱、发动机等密闭器械在不拆解的情况下进行内部检测。我国高端内窥镜起步较晚,绝大多数内窥镜仅具备图传功能,不具备三维重建和三维测量等更优质的检测手段,目前该领域主要是欧美和日本处于领先地位。美国贝克休斯旗下的Everest Mentor Visual iQ™ HD Version型号工业内窥镜[6]可支持左右屏幕分别成像、三维测量以及应用场景三维点云重建等,同时利用人工智能和基于云端的数字化检测工具提高检测数据的准确性和可靠性;由德国雪力公司研发的FlexiVision 100型号工业内窥镜[7]可提供优质的全高清图像,清晰识别缺陷,检测物体表面异常;日本奥林巴斯推出的IPLEX NX型号工业内窥镜[8]采用了高分辨率CCD相机,结合降噪控制算法可提供高分辨率和高清晰度的图像,搭配超广角三维测量功能可实现对缺陷尺寸以及深度的精准测量。
在技术层面,作为双目视觉的核心,立体匹配生成视差图的质量与速度直接决定了检测的精度与效率[9],FIRMANSYAH等[10]提出一种半全局匹配(semi-global matcing,SGM)算法与绝对差值求和(sum of absolut differences,SAD)算法相结合的策略用于无人机俯拍农田,通过分析作物长势,利用现场可编辑门阵列(field-programmable gate array,FPGA)对结合算法进行加速,加速后算法的计算时间可缩短至0.77 s;SHAHBAZI等[11]面对无人机航拍摄影测量中存在的大视差搜索问题提出了基于内禀曲线的高密度立体图像匹配方法,该方法有效地缩小了匹配过程中的视差搜索范围,提高了匹配效率;张泽琳等[12]提出了一种基于改进SGM算法的废旧机械零件彩色三维重建与检测方法,优化了算法在图像光照失真区域以及复杂结构处的匹配效果,使得重建的彩色三维模型纹理更加清晰;WANG等[13]通过改进绝对差值与Census变换(absolute difference and Census transform,AD-Census)混合算法,引入平均窗口像素代替中央像素值,提出区域自适应窗口匹配技术,发现改进后的算法在视差图中生成的物体轮廓更加清晰。GUO等[14]开发了一种双目内窥镜散斑系统来克服窄基线和拍摄图像大畸变的问题,提高了三维重建与检测的精度。随着近年来人工智能的发展,研究者们也将深度学习训练的方法应用到立体匹配中。HAMID等[15]分析了近十年来深度学习方法在立体匹配任务中的应用,比较了不同方法的网络结构和处理速度,并预测了基于深度学习的立体匹配研究在未来的应用前景;YU等[16]提出了一种基于深度学习的双目内窥镜三维测量方法,克服了传统立体匹配算法在弱纹理区域中鲁棒性差的问题。TAHMASEBI等[17]提出了一种双成本体立体匹配网络,由上层分组相关代价体和下层范数相关代价体构成,并通过一个耦合模块融合上下代价体提取的几何信息,具有较强的泛化能力,单对图像推理时间可缩短至67 ms。
其中,zCP 为点P在相机坐标系下的Z轴坐标,xVP 、yVP 分别为点P在像素坐标系下X轴与Y轴的坐标,dx、dy分别为像素坐标系下X轴与Y轴方向上单个像素点的实际物理尺寸,为图像坐标系原点在像素坐标系下的坐标, R 为旋转矩阵, T 为平移矩阵, R、T 二者共同用于世界坐标系与相机坐标系间的转化。
使用OpenCV结合MATLAB中的Stereo Camera Calibration程序对所采集的棋盘格双目图像进行标定计算。本文双目内窥镜的最终标定误差与拍摄位姿如图5所示,标定重投影误差为0.11 pixel,满足实验误差要求,标定求得的内外参数和畸变系数如表1所示,其中为径向畸变系数,为切向畸变系数。
HEBaofeng, DINGSiyuan, WEICuie, et al. Review of Measurement Methods for Areal Surface Roughness[J]. Optics and Precision Engineering, 2019, 27(1): 78-93.
NAYiming, HUChao, QIUYeyu, et al. Three-dimensional Positioning Guidance of Automobile Doors Based on Machine Vision[J]. China Mechanical Engineering, 2024, 35(9): 1677-1687.
YIHuaian, ZHAOXinjia, TANGLe, et al. Vision Mearsurement Method for Ground Surface Roughness Based on Color Image Singular Value Entropy Index [J]. China Mechanical Engineering, 2021, 32(13): 1577-1583.
CAOXuming, FENGHao, CAOLigang, et al. Online Quality Inspection System for Bolt Parts Parameters Based on Machine Vision [J]. Control Engineering of China, 2021, 28(6): 1187-1194.
MADazhi, YUBinchao, ZHANGYanze, et al. Measurement System for Large-scale High Reflective Component Based on Binocular Vision[J]. Journal of Applied Optics, 2021, 42(4): 577-585.
ZHOUFuqiang, WANGYexin, CHAIXinghua, et al. Review on Precise Measurement Technology Based on Mirror Binocular Vision [J]. Acta Optica Sinica, 2018, 38(8): 59-67.
[16]
FIRMANSYAHI, YAMAGUCHIY, TMARU-YAMA, et al. FPGA-Based Stereo Matching for Crop Height Measurement Using Monocular Camera [J]. Microprocess and Microsystems, 2024, 108: 105063.
[17]
SHAHBAZIM, SOHNG, THEOJ. High-density Stereo Image Matching Using Intrinsic Curves [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146: 373-388.
ZHANGZelin, CAOXing, WANGLei, et al. Color 3D Reconstruction Method of Used Machinery Parts Based on Improved SGM [J]. Laser & Optoelectronics Progress, 2024, 61(12): 114-124.
[20]
WANGYina, GUMengjiao, ZHUYufeng, et al. Improvement of AD-Census Algorithm Based on Stereo Vision [J]. Sensors, 2022, 22(18): 6933.
[21]
GUOJunguang, GUFeifei, YEYuping, et al. An Accurate Speckle 3D Reconstruction System Based on Binocular Endoscope[C]∥2021 IEEE International Conference on Real-time Computing and Robotics (RCAR). Xining, 2021: 703-708.
[22]
HAMIDM S, MANAPN A, HAMZAHR A, et al. Stereo Matching Algorithm Based on Deep Learning: a Survey [J]. Journal of King Saud University: Computer and Information Sciences, 2022, 5(34): 1663-1673.
[23]
YUHao, ZHOUChangjiang, ZHANGWei . et al. A Three-dimensional Measurement Method for Binocular Endoscopes Based on Deep Learning[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(4):653-660.
YANLong, ZHAOXingfang, DUHuiqiu. Research on 3D Measuring Based Binocular Vision[C]∥2014 IEEE International Conference on Control Science and Systems Engineering. Yantai, 2014: 18-22.
[26]
HUANGWei, MIAOHuisi, JIAOShuming, et al. A Planar Constraint Optimization Method to Improve Camera Calibration for Imperfect Planar Targets[J]. Optics and Lasers in Engineering, 2024,180: 108273 .
[27]
ZHANGXiaowen, TiegangLYU, DANWang, et al.High-precision Binocular Camera Calibration Method Based on a 3D Calibration Object[J]. Applied Optics, 2024, 63(10): 2667-2682.
[28]
ZHANGZhengyou. Flexible Camera Calibration by Viewing a Plane from Unknown Orientations[C] ∥Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, 1999: 666-673.
[29]
HIRSCHMULLERH. Stereo Processing by Semiglobal Matching and Mutual Information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328-341.
[30]
MOSTEGELC, RUMPLERM, FRAUN-DORFERF, et al. Using Self-contradiction to Learn Confidence Measures in Stereo Vision[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, 2016: 4067-4076.
[31]
MIANGOLEHS M H, DILLES, MAI L, et al. Boosting Monocular Depth Estimation Models to High-resolution via Content-adaptive Multi-resolution Merging[C]∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, 2021: 9680-9689.