Research on surface measurement method based on multi-view stereo vision and neutral network calibration
-
摘要: 针对三维表面形貌的非接触式光学测量是计算机多目立体视觉技术的一项重要应用,但目前还存在相机个数受限、特征点匹配算法复杂与纵向测量精度不够等难题。开发了一种基于多目立体视觉和神经网络标定的表面形貌测量方法,其中包括:使用神经网络完成多目标定与三维重构,在表面投射激光点阵作为图像识别与匹配的特征点,应用蚁群粒子跟踪测速技术进行多相机间相同特征点的匹配。经实验测试,相较于传统基于小孔成像模型进行标定与基于核线约束或互相关算法进行匹配的立体视觉测量系统,所提出的方法可适配具有大光学畸变的场景,能有效提高测量的空间分辨率,深度方向的测量误差在1.0%~2.0%的水平。Abstract: Non-contact optical measurement of three-dimensional surface topography is an important application of the computer multi-view stereo vision technology. However, there are still some technical problems needed to be solved such as limited number of cameras, complex image matching algorithms and insufficient longitudinal measurement accuracy. The article develops an optical method for surface topography measurement based on multi-view stereo vision and neural network calibration. This algorithm includes: applying neural networks for the calibration of the multi-view cameras and the three-dimensional reconstruction of feature points, projecting a laser speckle pattern on the surface as the feature points for image recognition and matching, using the particle tracking velocity technology to match the same single feature point in the multi-view images. As proved by experiments, compared with the traditional stereo vision measurement system, which is calibrated based on the pinhole camera model and matched by the epipolar constraint or cross-correlation algorithm, the method proposed in this paper can be adapted to scenes with large optical distortion, besides, the spatial resolution of the surface topography measurement is effectively improved and the measurement error in the depth direction is at the level of 1.0% – 2.0%.
-
表 1 实验模型参数
Table 1. Parameters of the experimental model
实验
模型波形函数 波幅
A/mm波长
${\lambda }$/mm模型尺寸
/(mm×mm)模型A ${\textit{z} } = 20{\rm{sin} }\left[ {\dfrac{ {\text{π} } }{ {60} }\left( {x - 10} \right)} \right]$ 20.0 120 140×100 模型B $\begin{array}{l} x = 0.006{t^2} + 0.03t\\{\textit{z} }=0.18t\times \mathrm{s}\mathrm{i}\mathrm{n}\left(\dfrac{ { {\text{π} } }t }{25}\right) \end{array}$ 2.6~34.0 9~129 200×200 表 2 不同实验工况下的重投影误差
Table 2. Reprojection errors under different experimental conditions
相机个数 重投影误差/${\rm{mm }}$ K=2 $ 0.351 $ K=3 $ 0.039 $ K=6 $ 0.033 $ 表 3 不同实验工况下的实验误差E
Table 3. Experimental errors E under different experimental conditions
相机个数模型A 模型B K=2 $1.6{\text{%}}$ $5.2{\text{%}}$ K=3 $1.1{\text{%}}$ $2.8{\text{%}}$ K=6 $1.0{\text{%}}$ $1.9{\text{%}}$ -
[1] MARR D. Vision[M]. New York: W H Freeman and Company, 1982. [2] BARNARD S T,FISCHLER M A. Computational stereo[J]. ACM Computing Surveys,1982,14(4):553-572. doi: 10.1145/356893.356896 [3] ITO M,ISHII A. Three-view stereo analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,PAMI-8(4):524-532. doi: 10.1109/TPAMI.1986.4767817 [4] GURWITZ E, SADOT E. More on the benefit of a third eye machine stereo perception[C]//Proc of the International Conference on Pattern Recognition, 1986. [5] LOWE D G. Object recognition from local scale-invariant features[C]// Proc of the Proceedings of the Seventh IEEE International Conference on Computer Vision. 1999. doi: 10.1109/ICCV.1999.790410 [6] LOWE D G. Distinctive image features from scale-invariant keypoin-ts[J]. International Journal of Computer Vision,2004,60(2):91-110. doi:10.1023/B: VISI.0000029664.99615.94 [7] 赵钦君,赵东标,韦虎. Harris-SIFT算法及其在双目立体视觉中的应用[J]. 电子科技大学学报,2010,39(4):546-550. doi: 10.3969/j.issn.1001-0548.2010.04.015ZHAO Q J,ZHAO D B,WEI H. Harris-SIFT algorithm and its application in binocular stereo vision[J]. Journal of University of Electronic Science and Technology of China,2010,39(4):546-550. doi: 10.3969/j.issn.1001-0548.2010.04.015 [8] SOLAV D,MOERMAN K M,JAEGER A M,et al. MultiDIC: an open-source toolbox for multi-view 3D digital image correlation[J]. IEEE Access,2018,6:30520-30535. doi: 10.1109/ACCESS.2018.2843725 [9] 左承林,马军,岳廷瑞,等. 基于双目立体视觉的直升机旋翼桨叶位移变形测量方法[J]. 实验流体力学,2020,34(1):87-95. doi: 10.11729/syltlx20190071ZUO C L,MA J,YUE T R,et al. Displacement and deformation measurements of helicopter rotor blades based on binocular stereo vision[J]. Journal of Experiments in Fluid Mechanics,2020,34(1):87-95. doi: 10.11729/syltlx20190071 [10] PAN B,QIAN K M,XIE H M,et al. Two-dimensional digital image correlation for in-plane displacement and strain measurement: a revi-ew[J]. Measurement Science and Technology,2009,20(6):062001. doi: 10.1088/0957-0233/20/6/062001 [11] 丁菁汀. 立体视觉在实际应用中的若干问题研究[D]. 杭州: 浙江大学, 2012.DING J T. Research on several problems for the practical use of stereo vision[D]. Hangzhou: Zhejiang University, 2012. [12] 苏勇,高越,郜泽仁,等. 光绘: 自由开源的数字散斑图像生成和评价软件[J]. 实验力学,2021,36(1):17-28. doi: 10.7520/1001-4888-20-161SU Y,GAO Y,GAO Z R,et al. Glare: a free and open source software for generation and assessment of digital speckle pattern[J]. Journal of Experimental Mechanics,2021,36(1):17-28. doi: 10.7520/1001-4888-20-161 [13] 窦建宇,潘翀. 基于神经网络的体视PIV空间标定模型[J]. 航空学报,2021,42(4):365-373. doi: 10.7527/S1000-6893.2020.24720DOU J Y,PAN C. Spatial calibration model of stereo PIV based on neural network[J]. Acta Aeronautica et Astronautica Sinica,2021,42(4):365-373. doi: 10.7527/S1000-6893.2020.24720 [14] NIE M Y,PAN C,WANG J J,et al. A hybrid 3D particle matching algorithm based on ant colony optimization[J]. Experiments in Fluids,2021,62(4):1-17. doi: 10.1007/s00348-021-03160-4 [15] HASSAN Y A,CANAAN R E. Full-field bubbly flow velocity measurements using a multiframe particle tracking technique[J]. Experiments in Fluids,1991,12(1):49-60. doi: 10.1007/BF00226565 [16] UEMURA T,YAMAMOTO F,OHMI K. A high-speed algorithm of image analysis for real time measurement of a two-dimensional velocity distribution[J]. Flow Visualization ASME FED,1989,85:129-134.