WO2020063013A1 - 一种基于深度学习的条纹投影时间相位展开方法 - Google Patents

一种基于深度学习的条纹投影时间相位展开方法 Download PDF

Info

Publication number
WO2020063013A1
WO2020063013A1 PCT/CN2019/094884 CN2019094884W WO2020063013A1 WO 2020063013 A1 WO2020063013 A1 WO 2020063013A1 CN 2019094884 W CN2019094884 W CN 2019094884W WO 2020063013 A1 WO2020063013 A1 WO 2020063013A1
Authority
WO
WIPO (PCT)
Prior art keywords
phase
frequency
data
wrapped
neural network
Prior art date
Application number
PCT/CN2019/094884
Other languages
English (en)
French (fr)
Inventor
陈钱
左超
冯世杰
张玉珍
顾国华
Original Assignee
南京理工大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京理工大学 filed Critical 南京理工大学
Priority to US17/280,464 priority Critical patent/US11906286B2/en
Publication of WO2020063013A1 publication Critical patent/WO2020063013A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2518—Projection by scanning of the object
    • G01B11/2527—Projection by scanning of the object with phase change by in-plane movement of the patern
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions

Definitions

  • the invention belongs to the field of optical measurement technology, and particularly relates to a fringe projection time and phase expansion method based on deep learning.
  • fringe projection contouring based on the principle of structured light and triangulation is one of the most practical techniques. Because it has the advantages of non-contact, full field, high accuracy and high efficiency, mainstream fringe projection contours In general, three processes are required to achieve three-dimensional measurement, which are phase recovery, phase expansion, and phase-to-height mapping.
  • phase-shifting contouring In phase recovery technology, the two most commonly used methods are Fourier contouring and phase shifting contouring. Fourier contouring only needs a fringe pattern to extract the phase, but this method is affected by spectral aliasing, resulting in poor quality of the measurement results and cannot measure objects with complex shapes. Compared with Fourier contouring, phase-shifting contouring has the advantages of being insensitive to ambient light and capable of obtaining pixel-level phase measurements. It is suitable for measuring objects with complex surfaces. However, this method generally requires projecting multiple phase shift fringe patterns (at least three) to achieve phase extraction. With the rapid development of high-speed cameras and DLP projection technology, phase shift contouring can also be used to achieve fast three-dimensional measurement.
  • both Fourier contouring and phase-shifting contouring use the arctangent function to extract the phase, and the range of the arctangent function is [0,2 ⁇ ]. Therefore, these two methods can only obtain the wrapped phase map, where the phase of 2 ⁇ exists. Jump. Therefore, it is necessary to implement a phase unwrapping technique to make the wrapped phase map into an absolute phase map.
  • phase expansion is time domain phase expansion and space domain phase expansion.
  • phase unwrapping in the airspace requires only a wrapped phase map to achieve phase unwrapping, but complex objects or multiple isolated objects cannot be measured effectively, and phase unwrapping errors are prone to occur.
  • the phase expansion in the time domain can unfold the wrapped phase stably, but it is necessary to use multiple wrapped phase maps with different frequencies, which greatly affects the efficiency of phase expansion and reduces the speed of three-dimensional measurement.
  • phase expansion techniques in the time domain a multi-frequency algorithm based on time phase expansion, a heterodyne method based on time phase expansion, and a number theory method based on time phase expansion.
  • the multi-frequency algorithm based on time and phase expansion can achieve the best phase expansion results, and the multi-wavelength rule is the most sensitive to noise.
  • the principle of the multi-frequency algorithm based on time-phase expansion is to use a single-cycle low-frequency wrapping phase to expand the high-frequency wrapping phase map. Due to the influence of noise during the measurement process, usually the multi-frequency method can only expand the high-frequency wrapping phase map with a frequency of 20.
  • the higher-frequency phase map has higher accuracy, so in order to achieve high-precision 3D measurement, it is often necessary to project multiple sets of fringe maps with different frequencies. This further reduces the measurement efficiency of the fringe projection profilometry and thereby suppresses its ability to measure moving objects. Therefore, for the three-dimensional imaging technology based on fringe projection contouring, there is currently a lack of a method that has both measurement accuracy and measurement efficiency.
  • the purpose of the present invention is to provide a fringe projection time phase expansion method based on deep learning, which can realize high-precision three-dimensional measurement without increasing the number of projection patterns of a projector.
  • the technical solution to achieve the purpose of the present invention is: a fringe projection time and phase expansion method based on deep learning, firstly projecting four groups of three-step phase-shifted grating patterns to the measured object with frequencies of 1, 8, 32, and 64, and a camera The raster image is collected using a three-step phase shift method to obtain a wrapped phase image. Second, a multi-frequency algorithm based on time phase expansion is used to phase unwrap the wrapped phase image to obtain a periodic level image with a phase of 64. Then a residual volume is built.
  • Product neural network set its input data to be a wrapped phase diagram with frequencies of 1 and 64, and output data to be a periodic order diagram of phases with frequency of 64; finally make training and verification sets to train and verify the network, and test the network A set of periodic order diagrams of phases with an output frequency of 64.
  • the present invention has significant advantages: due to the influence of noise in the measurement process, the multi-frequency algorithm based on time-phase expansion cannot use a single-cycle low-frequency wrapping phase to better expand the high-frequency wrapping phase map with a frequency of 64. , The result has a lot of error points, the effect is very poor.
  • the invention adopts deep learning method to realize time phase expansion. Compared with the multi-frequency algorithm based on time phase expansion, the residual phase convolution neural network is used to realize phase expansion. A single-cycle low-frequency wrapping phase can be used to better expand the frequency to 64. High-frequency wrapped phase map, this method can get an absolute phase map with fewer error points and higher accuracy.
  • FIG. 1 is a schematic flowchart of a fringe projection time and phase expansion method based on deep learning.
  • FIG. 2 is a schematic diagram of the three-dimensional measurement system of the present invention.
  • FIG. 3 is a structural diagram of a residual convolutional neural network based on deep learning of the present invention.
  • Figure 4 shows the loss curve of the residual convolutional neural network after 500 rounds of training and verification.
  • FIG. 5 is a periodic order diagram of phases of a set of data in a test set, where (a) is a periodic order diagram of a phase of frequency 64 obtained by using a two-phase wrapped phase diagram based on a multi-frequency algorithm based on time-phase expansion, (b ) Is a cycle order diagram of the phase with a frequency of 64 that is predicted by the network using two sets of wrapped phase diagrams, and (c) is a multi-frequency algorithm based on time and phase expansion using four sets of phase diagrams with a frequency of 64 In the hierarchy diagram, (d) is the difference between the results of (a) and (c), and (e) is the difference between the results of (b) and (c).
  • Figure 6 is a three-dimensional measurement diagram of a set of data in the test set, where (a) is a three-dimensional measurement diagram obtained by using a multi-frequency algorithm based on time phase expansion using two sets of wrapped phase diagrams, and (b) is obtained by a network using two sets of wrapped phase diagrams (C) is a three-dimensional measurement diagram obtained by using four sets of wrapped phase diagrams based on a multiphase algorithm based on time and phase expansion.
  • the deep learning-based fringe projection time and phase expansion method of the present invention includes the following five steps:
  • Step 1 Project and collect four groups of three-step phase-shifted raster images, each group containing three raster images with the same frequency and different initial phases.
  • Any group of three-step phase-shifted raster images projected by the projector can be expressed as:
  • x p , y p is the pixel coordinates of the plane of the projector
  • W is the horizontal resolution of the projector
  • f is the frequency of the three-step phase-shifted raster image.
  • a DLP projector is used to project four groups of three-step phase-shifted raster images to the measured object.
  • the frequencies of the four groups of three-step phase-shifted raster images are 1, 8, 32, and 64, and the frequency of the three raster images of each group is the same.
  • the projected raster image is captured synchronously by the camera.
  • the light intensity map of the collected three-step phase-shifted raster image is expressed as:
  • I 1 (x, y) A (x, y) + B (x, y) cos [ ⁇ (x, y)]
  • I 2 (x, y) A (x, y) + B (x, y) cos [ ⁇ (x, y) + 2 ⁇ / 3]
  • I 3 (x, y) A (x, y) + B (x, y) cos [ ⁇ (x, y) + 4 ⁇ / 3]
  • I 1 (x, y), I 2 (x, y), I 3 (x, y) are the corresponding three-step phase-shifted raster image light intensity maps
  • (x, y) are the pixel coordinates of the camera plane
  • a (x, y) is the background light intensity
  • B (x, y) is the modulation degree of the fringe
  • ⁇ (x, y) is the phase to be determined. Since the frequency of each group of three-step phase-shifted raster images is different, ⁇ (x, y) of the light intensity map corresponding to the three-step phase-shifted raster images is different.
  • Step 2 Use the light intensity maps I 1 (x, y), I 2 (x, y), and I 3 (x, y) of the three-step phase-shifted raster image collected in step 1 .
  • Wrapped phase diagram The specific formula is as follows:
  • the core of the phase expansion technique in fringe projection is how to solve k (x, y).
  • Step 3 Use a multi-frequency phase-unwrapping method based on time-phase unwrapping to achieve phase unwrapping to obtain the periodic order graph k (x, y). Since four sets of three-step phase-shifted raster images with different frequencies are projected, according to step two, wrapped phase maps with frequencies of 1, 8, 32, and 64 can be obtained. Because the range of the absolute phase map with frequency 1 is also [0, 2 ⁇ ], the wrapped phase map with frequency 1 is also an absolute phase map.
  • an absolute phase map can be obtained through a multi-frequency algorithm based on time-phase expansion, that is, the absolute phase map with a frequency of 1 can be used to help expand the wrapped phase map with a frequency of 8 to an absolute phase and a frequency of 8
  • the absolute phase map assists in expanding the wrapped phase map with frequency 32 as the absolute phase, and so on, to obtain the absolute phase with frequency 64, as follows:
  • f h is the frequency of the high-frequency raster image
  • f l is the frequency of the low-frequency raster image (when using an absolute phase map with a frequency of 1 to expand the wrapped phase map with a frequency of 8, the high frequency means 8 and the low frequency means 1)
  • k h (x, y) is the periodic order of the phase of the high-frequency raster image
  • ⁇ h (x, y) and ⁇ l (x, y) are the high-frequency and low-frequency gratings, respectively
  • the absolute phase of the graph, Round is a rounding function.
  • an absolute phase map with a frequency of 1 can be directly used to assist in expanding a wrapped phase map with a frequency of 64 as an absolute phase.
  • multi-frequency algorithms based on time and phase expansion generally use multiple sets of wrapped phase maps of different frequencies to expand in order to obtain the absolute phase of high frequencies.
  • multi-frequency algorithms based on time and phase expansion will consume a lot of time and are not conducive to implementation Fast and high-precision 3D measurement based on fringe projection.
  • Step 4 Build a residual convolutional neural network for phase unwrapping.
  • M is determined according to needs, and the more data is generally better, such as 1100
  • the data is divided into a training set, a validation set, and a test set, and the residual convolutional nerve is trained using the training set data Network, use the validation set to verify the learning effect of the network.
  • Figure 3 shows the structure of the neural network. It can be seen from Fig.
  • the entire residual convolutional neural network is mainly composed of 6 modules, including a convolution layer, a pooling layer, a feature fusion layer, a residual block, an upsampling block, and a sub-pixel layer.
  • Conv Layer, Pooling Layer, and Feature Fusion Layer are very commonly used modules in traditional convolutional neural networks.
  • the convolution layer consists of multiple convolution kernels.
  • the number of convolution kernels is the number of channels of the convolution layer.
  • Each convolution kernel independently performs spatial convolution operations on the input data to generate an output tensor.
  • the role of the pooling layer is to compress the input tensor.
  • the input tensor reduces the input tensor, simplifies the computational complexity of the network, and prevents overfitting; on the other hand, it compresses the input tensor to extract the main features of the input tensor.
  • Commonly used pooling layers are Average Pooling Layer and Max Pooling Layer.
  • the input tensor is subjected to 1/2 downsampling, 1/4 downsampling, and 1/8 downsampling using the maximum pooling layer, respectively.
  • the function of the feature fusion layer is to fuse the input tensor of each path.
  • each residual block contains two convolutional layers and two activation functions (ReLU). Due to the processing of the maximum pooling layer, the tensor in each path has a problem of inconsistent size. Therefore, for different paths, different numbers of upsampling blocks are used to make the sizes of the tensor in each path consistent.
  • the upsampling block consists of a convolutional layer, an activation function (ReLU), and a sub-pixel layer (Sub-Pixel Layer).
  • the role of the sub-pixel layer is to use the input tensor-rich channel data to up-sample the tensor in the spatial dimension, so that the number of channels of the tensor becomes 1/4 of the original, and the horizontal and vertical dimensions of the tensor become the original 2 times (Document "Real-Time Single Image Video Super-Resolution Using Efficient Sub-Pixel Convolutional Neural Network", author Wenzhe Shi et al.).
  • the innovation of the present invention lies in how to use the existing modules to build a network model capable of phase expansion, as shown in FIG. 3.
  • the model of the network After the model of the network is set up, set the input data of the network to be the phase diagrams of the packages with frequencies 1 and 64 obtained in step 2, and set the output data of the network to be the periodic order of the phases of frequency 64 obtained in step 3.
  • the absolute phase is the sum of the periodic order that wraps the phase map and the phase, you only need to get the periodic order of the phase to get the absolute phase.
  • the data type of the periodic level diagram of the phase is an integer
  • the data type of the absolute phase diagram is a floating point number.
  • the input data is a package phase map with frequencies 1 and 64 obtained in step 2.
  • the output data is the periodic order of the phase with frequency 64 obtained in step 3. It is worth noting that the data in the training, validation, and test sets are not reused. This data needs to be preprocessed before training the residual convolutional neural network. Because the picture taken by the camera contains the background and the measured object, the background can be removed by the following formula:
  • M (x, y) is the actual degree of fringe modulation.
  • the pixels in the picture corresponding to the background M (x, y) are much smaller than the pixels of the measured object corresponding to M (x, y), so You can set a threshold (such as 0.005) to remove the background from the picture.
  • the background-removed data is used as the residual convolutional neural network data set for residual convolutional neural network learning.
  • Step 5 Use the trained and verified residual convolutional neural network to test the test set data to evaluate the accuracy of the residual convolutional neural network, and output a periodic level diagram of the phase with a frequency of 64.
  • the residual convolutional neural network predicts the corresponding output data according to the input data in the test set data, compares the real output data with the output data predicted by the network, and the comparison result is used to evaluate the accuracy of the network. Due to the influence of noise in the measurement process, the multi-frequency algorithm based on time-phase expansion cannot use a single-cycle low-frequency wrapping phase to better develop a high-frequency wrapping phase map with a frequency of 64. As a result, there are a large number of error points and the effect is very poor.
  • the invention adopts deep learning method to realize time phase expansion.
  • the residual phase convolution neural network is used to realize phase expansion.
  • a single-cycle low-frequency wrapping phase can be used to better expand the frequency to 64.
  • High-frequency wrapped phase map this method can get an absolute phase map with fewer error points and higher accuracy.
  • a camera model acA640-750um, Basler
  • a DLP projector model LightCrafter 4500PRO, TI
  • a computer was used to construct a set of fringe projections based on deep learning
  • the three-dimensional measurement device of the time-phase expansion method is shown in FIG. 2.
  • the device has a shooting speed of 25 frames per second when performing three-dimensional measurement of objects.
  • step 1 four groups of three-step phase-shifted raster images with different frequencies are projected and collected, and the frequencies of the four groups of grating patterns are 1, 8, 32, and 64, respectively.
  • step two four sets of wrapped phase diagrams with different frequencies can be obtained.
  • Step 3 can be used to obtain a periodic order diagram and an absolute phase diagram of a phase with a frequency of 64.
  • step four build a residual convolutional neural network as shown in Figure 3.
  • steps one, two, and three 1100 sets of data are projected and photographed, of which 800 sets are used as training sets, 200 sets are used as validation sets, and 100 sets are used as test sets. It is worth noting that the data in the training, validation, and test sets are not reused.
  • the data after performing the background removal operation is used as the data set of the neural network for network learning.
  • Set the network's loss function to mean square error (MSE), the optimizer to Adam, and the network's training cycle to 500 rounds.
  • Figure 4 shows the loss curve of the residual convolutional neural network after 500 rounds of training and verification.
  • Figure 4 shows that the network stops converging after 250 rounds.
  • the loss value of the final training set is about 0.0058, and the loss value of the final verification set is about 0.0087.
  • step five the test set data is tested using a trained and verified residual convolutional neural network to evaluate the accuracy of the network. A comparative experiment was performed on a set of data in the test set, and the results are shown in Figure 5.
  • Fig. 5 is a cycle order diagram of the phases of a set of data in the test set, and Fig.
  • FIG. 5 (a) shows a cycle order diagram of a phase with a frequency of 64 obtained by using the two-phase wrapped phase diagram based on the multi-frequency algorithm of time phase expansion
  • Figure 5 (b) shows the network using two sets of wrapped phase maps to predict the periodic order of the phase with a frequency of 64.
  • Figure 5 (c) shows the multi-frequency algorithm based on time and phase expansion using four sets of wrapped phase maps
  • the obtained periodic order diagram of the phase with a frequency of 64 is shown in FIG. 5 (d), which is a difference between the result of FIG. 5 (a) and the result of FIG. 5 (c), and the number of difference points is 8,909.
  • FIG. 5 (e) is a difference between the result of FIG. 5 (b) and the result of FIG.
  • FIG. 5 confirms that the method based on the deep learning proposed by the present invention can obtain a cycle level diagram of a phase with fewer error points and higher accuracy.
  • Fig. 6 is a three-dimensional measurement diagram of the measured object, of which Fig. 6 (a) is a three-dimensional measurement diagram obtained by a multi-frequency algorithm based on time-phase expansion using two sets of wrapping phase diagrams, and Fig. 6 (b) is a network using two sets of wrapping phases The three-dimensional measurement diagram obtained from the figure, and FIG.
  • FIG. 6 (c) is a three-dimensional measurement diagram obtained by using a multi-frequency algorithm based on time-phase expansion using four sets of wrapped phase diagrams. This result in FIG. 6 further proves that a high-precision 3D measurement can be achieved without increasing the number of projection patterns of the projector (measurement efficiency).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

一种基于深度学习的条纹投影时间相位展开方法，首先向被测对象投影四组三步相移光栅图案，频率分别为1、8、32和64，相机采集光栅图利用三步相移法得到包裹相位图；然后使用基于时间相位展开的多频算法对包裹相位图进行相位展开得到频率为64的相位的周期级次图，并搭建一个残差卷积神经网络，设置其输入数据是频率为1和64的包裹相位图，输出数据是频率为64的相位的周期级次图，最后制作训练集和验证集对网络进行训练和验证，网络检验测试集输出频率为64的相位的周期级次图。采用深度学习的方法使用频率为1的包裹相位图展开频率为64的包裹相位图，可以得到错误点更少，准确度更高的绝对相位图。

Description

一种基于深度学习的条纹投影时间相位展开方法 技术领域
本发明属于光学测量技术领域，具体涉及一种基于深度学习的条纹投影时间相位展开方法。
背景技术
近几十年，快速三维形貌测量技术被广泛的应用于各个领域，如智能监控，工业质量控制和三维人脸识别等。在众多三维形貌测量方法中，基于结构光和三角测量原理的条纹投影轮廓术是最实用的技术之一，由于它具有无接触、全场、高精度和高效等优点，主流的条纹投影轮廓术一般需经过三个流程实现三维测量，分别是相位恢复、相位展开和相位到高度的映射。
在相位恢复技术中，最常用的两种方法是傅里叶轮廓术和相移轮廓术。傅里叶轮廓术只需一张条纹图即可提取相位，但这种方法受到频谱混叠的影响，导致测量结果的质量很差，不能测量形貌复杂的物体。相比于傅里叶轮廓术，相移轮廓术具有对环境光不敏感、能够获得像素级相位测量的优点，它适合于测量具有复杂表面的物体。但是这个方法一般需要投影多幅相移条纹图(至少三幅)实现相位提取。随着高速相机和DLP投影技术的快速发展，使得相移轮廓术也可以用于实现快速三维测量。但是，傅里叶轮廓术和相移轮廓术都使用反正切函数提取相位，反正切函数的值域[0,2π]，因此这两种方法都只能得到包裹相位图，其中存在2π的相位跳变。因此，需要实施相位展开技术使包裹相位图变为绝对相位图。
目前主流的相位展开方法是时域相位展开与空域相位展开。一方面，空域相位展开只需一幅包裹相位图即可实现相位展开，但是不能有效测量复杂物体或者多个孤立物体，容易出现相位展开错误。另一方面，时域相位展开能够稳定地展开包裹相位，但是需要使用多幅不同频率的包裹相位图，这极大地影响相位展开的效率从而降低三维测量的速度。常用的时域相位展开技术有三个：基于时间相位展开的多频算法，基于时间相位展开的外差法和基于时间相位展开的数论法。其中，基于时间相位展开的多频算法能够实现最好的相位展开结果，而多波长法则对噪声最敏感(文献“Temporal phase unwrapping algorithms for fringe projection profilometry:A comparative review”，作者Chao Zuo等)。基于时间相位展开的 多频算法的原理是使用单周期的低频包裹相位展开高频包裹相位图，由于测量过程中的噪声影响，通常多频法只能展开频率为20的高频包裹相位图。而更高频率的相位图拥有更高的精度，因此为了实现高精度的三维测量往往需要投影多组不同频率的条纹图。这进一步降低了条纹投影轮廓术的测量效率从而抑制了其测量运动物体的能力。因此，针对基于条纹投影轮廓术的三维成像技术而言，目前尚缺乏一种测量精度与测量效率兼得的方法。
发明内容
本发明目的在于提供一种基于深度学习的条纹投影时间相位展开方法，在不增加投影仪投影图案数的前提下，实现高精度的三维测量。
实现本发明目的的技术解决方案为：一种基于深度学习的条纹投影时间相位展开方法，首先向被测对象投影四组三步相移光栅图案，频率分别为1,8,32和64，相机采集光栅图利用三步相移法得到包裹相位图；其次，使用基于时间相位展开的多频算法对包裹相位图进行相位展开得到频率为64的相位的周期级次图；然后搭建一个残差卷积神经网络，设置其输入数据是频率为1和64的包裹相位图，输出数据是频率为64的相位的周期级次图；最后制作训练集和验证集对网络进行训练和验证，网络检验测试集输出频率为64的相位的周期级次图。
本发明与现有技术相比，其显著优点：由于测量过程中的噪声影响，基于时间相位展开的多频算法不能使用单周期的低频包裹相位较好地展开频率为64的高频包裹相位图，其结果存在大量的错误点，效果很差。本发明采用深度学习的方法实现时间相位展开，与基于时间相位展开的多频算法相比，使用残差卷积神经网络实现相位展开，可以使用单周期的低频包裹相位更好地展开频率为64的高频包裹相位图，该方法可以得到错误点更少，准确度更高的绝对相位图。
附图说明
图1为一种基于深度学习的条纹投影时间相位展开方法的流程示意图。
图2为本发明的三维测量***的原理图。
图3为本发明的基于深度学习的残差卷积神经网络的结构图。
图4为残差卷积神经网络经过500轮训练和验证后的损失值曲线。
图5为测试集中一组数据的相位的周期级次图，其中(a)为基于时间相位展开的多频算法使用两组包裹相位图得到的频率为64的相位的周期级次图，(b)为网络使用两组包裹相位图预测输出的频率为64的相位的周期级次图，(c)为 基于时间相位展开的多频算法使用四组包裹相位图得到的频率为64的相位的周期级次图，(d)为(a)的结果与(c)的结果的差异，(e)为(b)的结果与(c)的结果的差异。
图6为测试集中一组数据的三维测量图，其中(a)为基于时间相位展开的多频算法使用两组包裹相位图得到的三维测量图，(b)为网络使用两组包裹相位图得到的三维测量图，(c)为基于时间相位展开的多频算法使用四组包裹相位图得到的三维测量图。
具体实施方式
本发明基于深度学习的条纹投影时间相位展开方法包括以下五个步骤：
步骤一：投影并采集四组三步相移光栅图像，每组包含三张频率相同初始相位不同的光栅图像，投影仪投影的任一组三步相移光栅图像可以被表示为：
Figure PCTCN2019094884-appb-000001
Figure PCTCN2019094884-appb-000002
Figure PCTCN2019094884-appb-000003
其中
Figure PCTCN2019094884-appb-000004
为投影仪投影的三步相移光栅图像，(x p,y p)为投影仪平面的像素坐标，W是投影仪的水平分辨率，f是三步相移光栅图像的频率。使用DLP投影仪向被测对象投影四组三步相移光栅图像，四组三步相移光栅图像的频率分别为1、8、32和64，每组三张的光栅图像频率相同。通过相机同步拍摄投影的光栅图像，采集的三步相移光栅图像的光强图被表示为：
I 1(x,y)＝A(x,y)+B(x,y)cos[Φ(x,y)]
I 2(x,y)＝A(x,y)+B(x,y)cos[Φ(x,y)+2π/3]
I 3(x,y)＝A(x,y)+B(x,y)cos[Φ(x,y)+4π/3]
其中I 1(x,y)，I 2(x,y)，I 3(x,y)为对应的三步相移光栅图像光强图，(x,y)为相机平面的像素坐标，A(x,y)为背景光强，B(x,y)为条纹的调制度，Φ(x,y)为待求相位。由于每组三步相移光栅图像的频率不同，因此对应三步相移光栅图像的光强图的Φ(x,y)不同。
步骤二：利用步骤一采集的三步相移光栅图像的光强图I 1(x,y)，I 2(x,y)，I 3(x,y)，通过三步相移法可以得到包裹相位图
Figure PCTCN2019094884-appb-000005
具体如下式：
Figure PCTCN2019094884-appb-000006
由于arctan函数的截断效应，求得的相位图
Figure PCTCN2019094884-appb-000007
为包裹相位，其值域为[0,2π]，其与Φ(x,y)的关系如下：
Figure PCTCN2019094884-appb-000008
其中k(x,y)为相位的周期级次，其值域为[0,N-1]范围内的整数，N为条纹的总周期数，N＝f。条纹投影中相位展开技术的核心在于如何求解k(x,y)。
步骤三：使用基于时间相位展开的多频算法(Multi-frequency phase unwrapping method)实现相位展开，得到周期级次图k(x,y)。由于投影四组不同频率的三步相移光栅图像，根据步骤二可以得到频率为1、8、32和64的包裹相位图。因为频率为1的绝对相位图的范围也是[0,2π]，所以频率为1的包裹相位图也是绝对相位图。利用不同频率的包裹相位图，通过基于时间相位展开的多频算法可以得到绝对相位图，即可以用频率为1的绝对相位图辅助展开频率为8的包裹相位图为绝对相位，用频率为8的绝对相位图辅助展开频率为32的包裹相位图为绝对相位，依次类推，可以得到频率为64的绝对相位，具体如下式：
Figure PCTCN2019094884-appb-000009
Figure PCTCN2019094884-appb-000010
其中f h是高频光栅图的频率，f l是低频光栅图的频率(用频率为1的绝对相位图辅助展开频率为8的包裹相位图时，高频是指8，低频是指1)，
Figure PCTCN2019094884-appb-000011
是高频光栅图的包裹相位，k h(x,y)是高频光栅图的相位的周期级次，Φ h(x,y)和Φ l(x,y)分别是高频和低频光栅图的绝对相位，Round是四舍五入函数。基于时间相位展开的多频算法的原理，理论上可以直接使用频率为1的绝对相位图辅助展开频率为64的包裹相位图为绝对相位。但实际***中存在着不可忽视的噪声，导致获得的频率为64的绝对相位会存在较多的错误，效果很差。因此，基于时间相位展开的多频算法一般会使用多组不同频率的包裹相位图依次展开，从而获取高频的绝对相位，显然基于时间相位展开的多频算法会消耗大量的时间，不利于实现基于条纹投影的快速高精度三维测量。
步骤四：搭建残差卷积神经网络用于实现相位展开。重复执行步骤一至步骤三获取M组数据(M根据需要确定，数据一般越多越好，如1100)，将数据分为训练集、验证集和测试集，使用训练集数据训练残差卷积神经网络，使用验证集验证网络的学***均池化层(AveragePooling Layer)和最大池化层(MaxPooling Layer)。在本网络中，使用最大池化层对输入张量分别进行1/2下采样、1/4下采样和1/8下采样。特征融合层的作用是对各个路径的输入张量进行融合。此外，在网络的各个路径中都使用了4个残差块(Residual Block)用于解决在深层网络中梯度消失的问题，防止过拟合，使网络的损失函数加速收敛(文献“Deep Residual Learning for Image Recognition”，作者Kaiming He等)。每个残差块都包含两个卷积层和两个激活函数(ReLU)。由于最大池化层的处理，导致各个路径中的张量存在尺寸不一致的问题，因此，针对不同路径，使用不同数目的上采样块(Upsampling Block)使各个路径中的张量的尺寸一致。上采样块由一个卷积层，一个激活函数(ReLU)和一个亚像素层(Sub-Pixel Layer)组成。亚像素层的作用是使用输入张量丰富的通道数据对张量在空间维度进行上采样，从而是张量的通道数变为原来的1/4，张量的水平和垂直尺寸变为原来的2倍(文献“Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”，作者Wenzhe Shi等)。
虽然用于搭建网络的各个模块都是已公开发表过的技术，但本发明的创新点在于如何使用现有模块搭建一个能够实现相位展开的网络模型，如图3所示。网络的模型搭建完毕后，设置网络的输入数据是步骤二中得到的频率为1和64的包裹相位图，设置网络的输出数据是步骤三中得到的频率为64的相位的周期级次，而不是直接设置频率为64的绝对相位作为网络的输出数据。因为绝对相位是包裹相位图和相位的周期级次的和，只需得到相位的周期级次即可得到绝对相位。此外，相位的周期级次图的数据类型是整数，而绝对相位图的数据类型是浮点数，使用相位的周期级次作为网络的输出数据会减小网络的复杂度，使网络的损失函数更快地收敛，从而有效提升网络的输出准确度。为了准备用于网络学习 的数据集，重复执行步骤一至步骤三，对多个被测对象投影并采集相同的四组三步相移光栅图像，经过投影、数据采集和数据处理后，获得M组数据(如1100组)，其中800组作为训练集，200组作为验证集，100组作为测试集。每组数据由输入数据和输出数据构成，输入数据是在步骤二中获得的频率为1和64的包裹相位图，输出数据是在步骤三中获得的频率为64的相位的周期级次。值得注意的是，训练集、验证集和测试集中的数据均不重复使用。在训练残差卷积神经网络之前，需要预处理这些数据。因为相机拍摄得到的图片中包含背景和被测物体，可以通过如下公式去除背景：
Figure PCTCN2019094884-appb-000012
其中，M(x,y)是实际的条纹调制度，图片中属于背景的像素点对应的M(x,y)比被测物体的像素点对应的M(x,y)小得多，因此可以设置阈值(如0.005)来去除图片中的背景。经过去背景操作的数据作为残差卷积神经网络的数据集用于残差卷积神经网络学习。设置残差卷积神经网络的损失函数为均方误差(MSE)，优化器为Adam，设置残差卷积神经网络的训练周期(训练网络时，网络的损失值停止收敛后停止训练，如500轮)，开始使用训练集数据训练残差卷积神经网络，使用验证集验证残差卷积神经网络的学习效果。
步骤五：使用经过训练和验证后的残差卷积神经网络检验测试集数据用于评价残差卷积神经网络的准确性，输出频率为64的相位的周期级次图。残差卷积神经网络根据测试集数据中的输入数据预测其对应的输出数据，将真实的输出数据与网络预测的输出数据进行比较，其比较结果用于评价网络的准确性。由于测量过程中的噪声影响，基于时间相位展开的多频算法不能使用单周期的低频包裹相位较好地展开频率为64的高频包裹相位图，其结果存在大量的错误点，效果很差。本发明采用深度学习的方法实现时间相位展开，与基于时间相位展开的多频算法相比，使用残差卷积神经网络实现相位展开，可以使用单周期的低频包裹相位更好地展开频率为64的高频包裹相位图，该方法可以得到错误点更少，准确度更高的绝对相位图。
实施例
为验证本发明所述方法的有效性，使用一台相机(型号acA640-750um,Basler)，一台DLP投影仪(型号LightCrafter 4500PRO,TI)以及一台计算机构建 了一套基于深度学习的条纹投影时间相位展开方法的三维测量装置，如图2所示。该套装置在进行物体的三维测量时的拍摄速度为25帧每秒。利用步骤一所述，投影并采集四组不同频率的三步相移光栅图像，四组光栅图案的频率分别为1,8,32和64。利用步骤二可以得到四组不同频率的包裹相位图。利用步骤三可以得到频率为64的相位的周期级次图和绝对相位图。利用步骤四，搭建如图3所示的残差卷积神经网络。利用步骤一、二和三，投影并拍摄1100组数据，其中800组作为训练集，200组作为验证集，100组作为测试集。值得注意的是，训练集、验证集和测试集中的数据均不重复使用。执行去背景操作后的数据作为神经网络的数据集用于网络学习。设置网络的损失函数为均方误差(MSE)，优化器为Adam，设置网络的训练周期为500轮。图4为残差卷积神经网络经过500轮的训练和验证后的损失值曲线。图4显示网络经过250轮后停止收敛，最终训练集的损失值约为0.0058，最终验证集的损失值约为0.0087。利用步骤五，使用经过训练和验证后的残差卷积神经网络检验测试集数据，用于评价网络的准确性。针对测试集中一组数据做了一个对比性的实验，结果如图5所示。图5为测试集中一组数据的相位的周期级次图，其中图5(a)显示了基于时间相位展开的多频算法使用两组包裹相位图得到的频率为64的相位的周期级次图，图5(b)显示了网络使用两组包裹相位图预测输出的频率为64的相位的周期级次图，图5(c)显示了基于时间相位展开的多频算法使用四组包裹相位图得到的频率为64的相位的周期级次图，图5(d)为图5(a)的结果与图5(c)的结果的差异，差异点的个数为8909个。图5(e)为图5(b)的结果与图5(c)的结果的差异，差异点的个数为381个。相比于基于时间相位展开的多频算法，图5证实了本发明提出的基于深度学习的方法可以得到错误点更少，准确度更高的相位的周期级次图。图6为被测对象的三维测量图，其中图6(a)为基于时间相位展开的多频算法使用两组包裹相位图得到的三维测量图，图6(b)为网络使用两组包裹相位图得到的三维测量图，图6(c)为基于时间相位展开的多频算法使用四组包裹相位图得到的三维测量图。图6这个结果进一步证明，在不增加投影仪投影图案数的前提下(测量效率)，实现高精度的三维测量。

Claims (6)

  1. 一种基于深度学习的条纹投影时间相位展开方法，其特征在于如下步骤：
    步骤一：向被测对象投影四组三步相移光栅图像，四组光栅图案的频率分别为1、8、32和64，通过相机同步拍摄投影的光栅图像，采集四组三步相移光栅图像的光强图；
    步骤二：利用步骤一采集的三步相移光栅图像的光强图，基于三步相移法，计算得到不同频率的包裹相位图；
    步骤三：使用基于时间相位展开的多频算法对得到的四组包裹相位图依次进行相位展开，最终得到频率为64的相位的周期级次图和绝对相位图；
    步骤四：搭建残差卷积神经网络用于实现相位展开，重复执行步骤一至三获取多组数据，将数据分为训练集、验证集和测试集，使用训练集数据训练残差卷积神经网络，使用验证集验证网络的学习效果；
    步骤五：使用经过训练和验证后的残差卷积神经网络检验测试集数据用于评价网络的准确性，输出频率为64的相位的周期级次图。
  2. 根据权利要求1所述的基于深度学习的条纹投影时间相位展开方法，其特征在于步骤一的向被测对象投影四组三步相移光栅图像，每组包含三张频率相同初始相位不同的光栅图像，投影仪投影的任一组三步相移光栅图像可以被表示为：
    Figure PCTCN2019094884-appb-100001
    Figure PCTCN2019094884-appb-100002
    Figure PCTCN2019094884-appb-100003
    其中
    Figure PCTCN2019094884-appb-100004
    为投影仪投影的三步相移光栅图像，(x p,y p)为投影仪平面的像素坐标，W是投影仪的水平分辨率，f是三步相移光栅图像的频率，使用DLP投影仪向被测对象投影四组三步相移光栅图像，四组三步相移光栅图像的频率分别为1、8、32和64，每组三张的光栅图像频率相同，通过相机同步拍摄投影的光栅图像，采集的三步相移光栅图像的光强图被表示为：
    I 1(x,y)＝A(x,y)+B(x,y)cos[Φ(x,y)]
    I 2(x,y)＝A(x,)+B(x,y)cos[Φ(x,y)+2π/3]
    I 3(x,y)＝A(x,y)+B(x,y)cos[Φ(x,y)+4π/3]
    其中I 1(x,y)，I 2(x,y)，I 3(x,y)为对应的三步相移光栅图像光强图，(x,y)为相机平面的像素坐标，A(x,y)为背景光强，B(x,y)为条纹的调制度，Φ(x,y)为待求相位。
  3. 根据权利要求1所述的基于深度学习的条纹投影时间相位展开方法，其特征在于步骤二中，包裹相位图
    Figure PCTCN2019094884-appb-100005
    由下式计算而得：
    Figure PCTCN2019094884-appb-100006
    由于arctan函数的截断效应，求得的相位图
    Figure PCTCN2019094884-appb-100007
    为包裹相位，其值域为[0,2π]，其与Φ(x,y)的关系如下：
    Figure PCTCN2019094884-appb-100008
    其中k(x,y)为相位的周期级次，其值域为[0,N-1]范围内的整数，N为条纹的总周期数，N＝f。
  4. 根据权利要求1所述的基于深度学习的条纹投影时间相位展开方法，其特征在于步骤三中，频率为1的绝对相位图的范围是[0,2π]，所以频率为1的包裹相位图是绝对相位图，利用不同频率的包裹相位图，通过基于时间相位展开的多频算法得到绝对相位图，即可以用频率为1的绝对相位图辅助展开频率为8的包裹相位图为绝对相位，用频率为8的绝对相位图辅助展开频率为32的包裹相位图为绝对相位，依次类推，得到频率为64的绝对相位，具体如下式：
    Figure PCTCN2019094884-appb-100009
    Figure PCTCN2019094884-appb-100010
    其中f h是高频光栅图的频率，f l是低频光栅图的频率，
    Figure PCTCN2019094884-appb-100011
    是高频光栅图的包裹相位，k h(x,y)是高频光栅图的相位的周期级次，Φ h(x,y)和Φ l(x,y)分别是高频和低频光栅图的绝对相位，Round是四舍五入函数。
  5. 根据权利要求1所述的基于深度学习的条纹投影时间相位展开方法，其特征在于步骤四中，首先搭建一个残差卷积神经网络，由6种模块组成，包括卷积层、池化层、特征融合层、残差块、上采样块和亚像素层；
    其次，网络的模型搭建完毕后，为准备用于网络学习的数据集，重复执行步骤一至步骤三，对多个被测对象投影并采集相同的四组三步相移光栅图像，经过投影、数据采集和数据处理后，获得多组数据，并将这些数据分别作为训练集、验证集、测试集，每组数据由输入数据和输出数据构成，输入数据是在步骤二中获得的频率为1和64的包裹相位图，输出数据是在步骤三中获得的频率为64的相位的周期级次，其中训练集、验证集和测试集中的数据均不重复使用；
    在训练残差卷积神经网络之前，对获取的数据进行预处理，即相机拍摄得到 的图片中包含背景和被测物体，通过如下公式去除背景：
    Figure PCTCN2019094884-appb-100012
    其中，M(x,y)是实际的条纹调制度，图片中属于背景的像素点对应的M(x,y)比被测物体的像素点对应的M(x,y)小得多，设置阈值来去除图片中的背景；经过去背景操作的数据作为残差卷积神经网络的数据集用于残差卷积神经网络学习，设置残差卷积神经网络的损失函数为均方误差，优化器为Adam，设置残差卷积神经网络的训练周期，然后使用训练集数据训练残差卷积神经网络，使用验证集验证残差卷积神经网络的学习效果。
  6. 根据权利要求1所述的基于深度学习的条纹投影时间相位展开方法，其特征在于步骤五中，残差卷积神经网络根据测试集数据中的输入数据预测其对应的输出数据，将真实的输出数据与网络预测的输出数据进行比较，其比较结果用于评价网络的准确性。
PCT/CN2019/094884 2018-09-29 2019-07-05 一种基于深度学习的条纹投影时间相位展开方法 WO2020063013A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/280,464 US11906286B2 (en) 2018-09-29 2019-07-05 Deep learning-based temporal phase unwrapping method for fringe projection profilometry

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811149287.9A CN109253708B (zh) 2018-09-29 2018-09-29 一种基于深度学习的条纹投影时间相位展开方法
CN201811149287.9 2018-09-29

Publications (1)

Publication Number Publication Date
WO2020063013A1 true WO2020063013A1 (zh) 2020-04-02

Family

ID=65044755

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/094884 WO2020063013A1 (zh) 2018-09-29 2019-07-05 一种基于深度学习的条纹投影时间相位展开方法

Country Status (3)

Country Link
US (1) US11906286B2 (zh)
CN (1) CN109253708B (zh)
WO (1) WO2020063013A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111651954A (zh) * 2020-06-10 2020-09-11 嘉兴市像景智能装备有限公司 基于深度学习对smt电子元件三维重建的方法
CN111736173A (zh) * 2020-05-24 2020-10-02 深圳奥比中光科技有限公司 一种基于tof的深度测量装置、方法及电子设备
CN112212805A (zh) * 2020-09-18 2021-01-12 南京理工大学 一种基于复合编码的高效立体相位展开方法
CN112212806A (zh) * 2020-09-18 2021-01-12 南京理工大学 一种基于相位信息导向的立体相位展开方法
CN113066164A (zh) * 2021-03-19 2021-07-02 河南工业大学 一种基于相移轮廓术的单条纹运动物体三维重建方法
CN113160180A (zh) * 2021-04-23 2021-07-23 深圳高性能医疗器械国家研究院有限公司 一种基于深度学习的磁共振图像相位复原方法
CN113239614A (zh) * 2021-04-22 2021-08-10 西北工业大学 一种大气湍流相位时空预估算法
EP4379661A1 (en) 2022-11-30 2024-06-05 Piotr Piechocki A method for dynamic 3d scanning of a spatial object and a dynamic 3d scanner

Families Citing this family (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109253708B (zh) * 2018-09-29 2020-09-11 南京理工大学 一种基于深度学习的条纹投影时间相位展开方法
CN109903377B (zh) * 2019-02-28 2022-08-09 四川川大智胜软件股份有限公司 一种无需相位展开的三维人脸建模方法及***
CN110081817B (zh) * 2019-04-24 2021-05-04 杭州光粒科技有限公司 消除背景光的方法、装置、计算机设备和存储介质
CN110163817B (zh) * 2019-04-28 2021-06-18 浙江工业大学 一种基于全卷积神经网络的相位主值提取方法
CN110210119A (zh) * 2019-05-30 2019-09-06 东南大学 一种基于深层卷积神经网络的高效率相位展开方法
CN110260795B (zh) * 2019-05-31 2020-12-29 广东工业大学 一种基于增量绝对光栅尺的绝对位移检测方法
CN110378367B (zh) * 2019-06-05 2021-04-30 深圳大学 用于相位展开的生成式对抗网络的获取方法、装置及***
CN110514143B (zh) * 2019-08-09 2021-05-07 南京理工大学 一种基于反射镜的条纹投影***标定方法
CN110500957B (zh) * 2019-09-10 2021-09-14 中国科学院苏州纳米技术与纳米仿生研究所 一种主动三维成像方法、装置、设备以及存储介质
CN110686652B (zh) * 2019-09-16 2021-07-06 武汉科技大学 一种基于深度学习和结构光相结合的深度测量方法
CN111043988B (zh) * 2019-12-10 2021-04-23 东南大学 一种基于图形学和深度学习的单张条纹投影测量方法
CN111006612B (zh) * 2019-12-20 2021-01-01 四川大学 三维成像方法、装置、计算机设备和可读存储介质
CN111189414B (zh) * 2020-01-09 2021-09-03 西安知象光电科技有限公司 一种实时单帧相位提取方法
CN111311606B (zh) * 2020-01-20 2023-05-05 四川大学 连续相位图获取方法、装置、计算机设备和可读存储介质
CN111402240A (zh) * 2020-03-19 2020-07-10 南京理工大学 基于深度学习的单帧彩色条纹投影的三维面型测量方法
CN111351450B (zh) * 2020-03-20 2021-09-28 南京理工大学 基于深度学习的单帧条纹图像三维测量方法
CN111523618A (zh) * 2020-06-18 2020-08-11 南京理工大学智能计算成像研究院有限公司 一种基于深度学习的相位展开方法
CN111829458B (zh) * 2020-07-20 2022-05-13 南京理工大学智能计算成像研究院有限公司 基于深度学习的伽马非线性误差矫正方法
CN111947600B (zh) * 2020-07-24 2022-05-20 南京理工大学 基于相位级次代价滤波的鲁棒立体相位展开方法
CN111928794B (zh) * 2020-08-04 2022-03-11 北京理工大学 基于深度学习的闭合条纹兼容单幅干涉图解相方法及装置
KR102512873B1 (ko) * 2020-11-06 2023-03-23 한국생산기술연구원 인공 지능을 이용한 모아레 간섭계 측정 시스템 및 모아레 간섭계 측정 방법
CN112747693B (zh) * 2020-12-25 2022-05-31 南京理工大学智能计算成像研究院有限公司 针对高反光物体的一种基于彩色图像检测的三维测量方法
CN112833818B (zh) * 2021-01-07 2022-11-15 南京理工大学智能计算成像研究院有限公司 一种单帧条纹投影三维面型测量方法
CN112325799A (zh) * 2021-01-07 2021-02-05 南京理工大学智能计算成像研究院有限公司 一种基于近红外光投影的高精度三维人脸测量方法
CN113238227B (zh) * 2021-05-10 2022-09-30 电子科技大学 一种结合深度学习的改进最小二乘相位解缠方法及***
CN113379818B (zh) * 2021-05-24 2022-06-07 四川大学 一种基于多尺度注意力机制网络的相位解析方法
CN113743579B (zh) * 2021-07-26 2024-03-01 东莞理工学院 基于深度学习的三波长相位解包方法、***、设备及介质
CN113983960B (zh) * 2021-10-12 2023-10-31 安徽农业大学 一种多频条纹投影非线性误差校正方法
CN114152217B (zh) * 2022-02-10 2022-04-12 南京南暄励和信息技术研发有限公司 基于监督学习的双目相位展开方法
CN115046717B (zh) * 2022-06-08 2023-05-12 安徽农业大学 一种利用傅里叶变换轮廓术优化的结构振型可视化方法、装置及***
CN115187649B (zh) * 2022-09-15 2022-12-30 中国科学技术大学 抗强环境光干扰的三维测量方法、***、设备及存储介质
CN116105632B (zh) * 2023-04-12 2023-06-23 四川大学 一种结构光三维成像的自监督相位展开方法及装置
CN117047286B (zh) * 2023-10-09 2024-01-16 东莞市富明钮扣有限公司 一种激光加工工件表面的方法、加工***、处理器和存储介质
CN117475172B (zh) * 2023-12-28 2024-03-26 湖北工业大学 一种基于深度学习的高噪声环境相位图解包裹方法和***
CN117689705B (zh) * 2024-01-31 2024-05-28 南昌虚拟现实研究院股份有限公司 一种基于深度学习条纹结构光深度重建方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101105393A (zh) * 2006-07-13 2008-01-16 周波 投射多频光栅的物体表面三维轮廓的视觉测量方法
US20100027739A1 (en) * 2007-10-30 2010-02-04 Massachusetts Institute Of Technology Phase-Contrast X-Ray Imaging
US8929644B2 (en) * 2013-01-02 2015-01-06 Iowa State University Research Foundation 3D shape measurement using dithering
CN106840036A (zh) * 2016-12-30 2017-06-13 江苏四点灵机器人有限公司 一种适用于快速三维形貌测量的二元结构光优化方法
CN108596008A (zh) * 2017-12-12 2018-09-28 南京理工大学 针对三维人脸测量的面部抖动补偿方法
CN109253708A (zh) * 2018-09-29 2019-01-22 南京理工大学 一种基于深度学习的条纹投影时间相位展开方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2353720A1 (en) * 2001-07-25 2003-01-25 Brendan J. Frey Method for unwrapping 2-dimensional phase signals
CN101329169B (zh) * 2008-07-28 2010-09-08 中国航空工业第一集团公司北京航空制造工程研究所 一种电子束焊接熔凝区形状因子的神经网络建模方法
CN102155924B (zh) * 2010-12-17 2012-07-04 南京航空航天大学 基于绝对相位恢复的四步相移方法
SG11201400794QA (en) * 2011-10-18 2014-06-27 Univ Nanyang Tech Apparatus and method for 3d surface measurement
AU2013260650B2 (en) * 2013-11-20 2015-07-16 Canon Kabushiki Kaisha Rotational phase unwrapping
CN103759673B (zh) * 2014-01-21 2016-07-06 南京理工大学 基于双频三灰阶正弦光栅条纹投影的时间相位去包裹方法
CN103791856B (zh) * 2014-01-21 2017-01-04 南京理工大学 基于四幅结构光图像的相位求解与去包裹方法
CN108510546B (zh) * 2017-02-28 2021-10-01 北京航空航天大学 一种适用于图谱及结构信息同步探测***的相机标定方法
CN108319693A (zh) * 2018-02-01 2018-07-24 张文淑 一种基于立体遥感数据库的地貌特征聚类分析方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101105393A (zh) * 2006-07-13 2008-01-16 周波 投射多频光栅的物体表面三维轮廓的视觉测量方法
US20100027739A1 (en) * 2007-10-30 2010-02-04 Massachusetts Institute Of Technology Phase-Contrast X-Ray Imaging
US8929644B2 (en) * 2013-01-02 2015-01-06 Iowa State University Research Foundation 3D shape measurement using dithering
CN106840036A (zh) * 2016-12-30 2017-06-13 江苏四点灵机器人有限公司 一种适用于快速三维形貌测量的二元结构光优化方法
CN108596008A (zh) * 2017-12-12 2018-09-28 南京理工大学 针对三维人脸测量的面部抖动补偿方法
CN109253708A (zh) * 2018-09-29 2019-01-22 南京理工大学 一种基于深度学习的条纹投影时间相位展开方法

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111736173A (zh) * 2020-05-24 2020-10-02 深圳奥比中光科技有限公司 一种基于tof的深度测量装置、方法及电子设备
CN111651954A (zh) * 2020-06-10 2020-09-11 嘉兴市像景智能装备有限公司 基于深度学习对smt电子元件三维重建的方法
CN111651954B (zh) * 2020-06-10 2023-08-18 嘉兴市像景智能装备有限公司 基于深度学习对smt电子元件三维重建的方法
CN112212805A (zh) * 2020-09-18 2021-01-12 南京理工大学 一种基于复合编码的高效立体相位展开方法
CN112212806A (zh) * 2020-09-18 2021-01-12 南京理工大学 一种基于相位信息导向的立体相位展开方法
CN112212805B (zh) * 2020-09-18 2022-09-13 南京理工大学 一种基于复合编码的高效立体相位展开方法
CN113066164A (zh) * 2021-03-19 2021-07-02 河南工业大学 一种基于相移轮廓术的单条纹运动物体三维重建方法
CN113239614A (zh) * 2021-04-22 2021-08-10 西北工业大学 一种大气湍流相位时空预估算法
CN113160180A (zh) * 2021-04-23 2021-07-23 深圳高性能医疗器械国家研究院有限公司 一种基于深度学习的磁共振图像相位复原方法
CN113160180B (zh) * 2021-04-23 2024-02-09 深圳高性能医疗器械国家研究院有限公司 一种基于深度学习的磁共振图像相位复原方法
EP4379661A1 (en) 2022-11-30 2024-06-05 Piotr Piechocki A method for dynamic 3d scanning of a spatial object and a dynamic 3d scanner

Also Published As

Publication number Publication date
US11906286B2 (en) 2024-02-20
CN109253708B (zh) 2020-09-11
US20210356258A1 (en) 2021-11-18
CN109253708A (zh) 2019-01-22

Similar Documents

Publication Publication Date Title
WO2020063013A1 (zh) 一种基于深度学习的条纹投影时间相位展开方法
WO2021184707A1 (zh) 基于深度学习的单帧彩色条纹投影的三维面型测量方法
Lin et al. Three-dimensional shape measurement technique for shiny surfaces by adaptive pixel-wise projection intensity adjustment
CN111351450B (zh) 基于深度学习的单帧条纹图像三维测量方法
CN114777677B (zh) 基于深度学习的单帧双频复用条纹投影三维面型测量方法
CN108955571B (zh) 双频外差与相移编码相结合的三维测量方法
CN110163817B (zh) 一种基于全卷积神经网络的相位主值提取方法
WO2021184686A1 (zh) 基于多尺度生成对抗神经网络的单帧条纹分析方法
CN111947599B (zh) 基于学习的条纹相位恢复和散斑相关的三维测量方法
Dai et al. A dual-frequency fringe projection three-dimensional shape measurement system using a DLP 3D projector
CN105300317B (zh) 基于正弦和三角波条纹投影的三维测量方法
CN102322822A (zh) 一种三频彩色条纹投影三维测量方法
CN108195313A (zh) 一种基于光强响应函数的高动态范围三维测量方法
US11512946B2 (en) Method and system for automatic focusing for high-resolution structured light 3D imaging
CN110006365B (zh) 基于二维查找表的相位展开方法、装置及电子设备
Lv et al. An improved phase-coding method for absolute phase retrieval based on the path-following algorithm
CN115063466A (zh) 一种基于结构光与深度学习的单帧三维测量方法
CN104154879A (zh) 一种非均匀条纹分段生成方法
Liu et al. A novel phase unwrapping method for binocular structured light 3D reconstruction based on deep learning
CN115205360A (zh) 复合条纹投影钢管的三维外轮廓在线测量与缺陷检测方法及应用
CN112212806B (zh) 一种基于相位信息导向的立体相位展开方法
CN111947600A (zh) 基于相位级次代价滤波的鲁棒立体相位展开方法
KR101001894B1 (ko) 컬러 투사 모아레 기법을 이용한 3차원 형상 측정 장치 및 방법
CN116645466A (zh) 一种三维重建方法及电子设备、存储介质
Zhou et al. Patch-based texture synthesis for image inpainting

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19864164

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19864164

Country of ref document: EP

Kind code of ref document: A1