CN106815864A - Depth information measuring method based on single frames modulation template - Google Patents

Abstract

The invention discloses a kind of depth information measuring method based on single frames modulation template, mainly solve the problems, such as prior art take it is long, cannot dynamically measure depth information of scene.Its implementation is：Design single frames modulation template P；P is projected into target, deformation pattern I is obtained；I is transmitted to computer, demodulation obtains demodulation cosine templates I '₁And demodulation De Bruijn sequence templates I '₂；I ' is calculated with Gabor filter₁Block phaseAccording to I '₂WithSolve phase unwrapping value Φ (x, y)；Match point (x of the pixel (x, y) in P is asked for according to Φ (x, y)_p,y_p)；Utilize (x, y) and (x_p,y_p) space geometry relation, calculate depth information value.The present invention dynamic access depth information of scene and can possess precision higher, can be used for industrial monitoring, man-machine interaction, virtual reality and 3D printing scene.

Description

Depth information measuring method based on single frames modulation template

Technical field

The invention belongs to field of measuring technique, a kind of measuring method of depth information is related generally to, can apply to industry Monitoring, medical science, man-machine interaction, virtual reality, reality enhancing and 3D printing scene.

Background technology

The signal that traditional Image-forming instrument is collected only is the two-dimensional signal in spatial domain, the depth information in three-dimensional scenic Cannot be collected.With the development and the lifting of people's living standard of computer science, two-dimensional signal can not meet The need for actual production is lived.How fast and efficiently to obtain three-dimensional depth information from scene and have become current research Focus.

It refers to the depth information that object is obtained in three-dimensional scenic that depth is obtained.The method that current depth is obtained is broadly divided into Active and two kinds of passive type.Passive type method is generally basede on one or more image to obtain the three-dimensional shape information of object, Such as stereo vision method, skiametry method, wherein most widely used is exactly stereo vision method, the method is more using multiple cameras Angle is shot to Same Scene, and depth information is obtained by the matching between multiple image.Proactive is then to utilize Optical projection device projects probing wave, is captured to sensor after being reflected through object, and is processed using computer, so as to obtain Depth information, common method has time-of-flight method, Structure light method, laser scanning method, and wherein structural light measurement method is used as one Contactless measurement is planted, the method with structure due to realizing that simple, with low cost, precision and real-time are preferable, anti- The features such as interference performance is strong, obtains a wide range of applications.

The structure optical mode plate that one or more have specific rule is typically projected measurement scene by structural light measurement method In, object under test is shot using image capture device, by the image for comparing the template of projection and collect, with reference to range of triangle Principle calculates the depth information of body surface.Structural light measurement method can be divided into time encoding according to the coded system of template Method and space encoding method, wherein：

The characteristics of space encoding is only to project a width template, and the coding information of each pixel can combine its neighbour in template Domain is obtained, and the depth that space encoding is applied to dynamic scene is obtained, but because space encoding may be subject to neighbour in decoding process The loss of domain information, body surface reflectivity be inconsistent and the problems such as object under test surface color, easily causes decoding error and leads Depth information mistake in computation is caused, therefore space encoding is relatively low relative to the resolution ratio of time encoding, precision.

Time encoding, common are binary-coding, n values coding, hybrid coding etc., and these methods have simple being easy in fact Now, the advantages of spatial resolution is high, certainty of measurement is high, but it projects several measurement modules due to needing, and time of measuring is more long, because And it is only applicable to the depth survey of static scene.

The content of the invention

Deficiency it is an object of the invention to be directed to above-mentioned time encoding technology, proposes a kind of based on single frames modulation template Depth information measuring method, to shorten time of measuring, realizes the dynamic measurement of depth information of scene.

Technical method of the invention is：Using modulation technique, template and De Bruijn sequences based on cosine coding are compiled The form assembly of code is for single frames composite shuttering and is projected；By matching the scene image of projection template and deformation, with reference to Principle of triangulation, obtains the depth information of scene, and implementation step includes as follows：

(1) design single frames modulation template P：

(1a) design is based on cosine templates I₁With based on De Bruijn sequence templates I₂；

(1b) design frequency is f_c1Carrier wave template I_c1It is f with frequency_c2Carrier wave template I_c2, and according to the two carrier waves Template I_c1、I_c2With above-mentioned cosine templates I₁, De Bruijn sequence templates I₂Obtain single frames modulation template P：

P=I₁·I_c1+I₂·I_c2；

(1c) calculates the intensity level I of each pixel (x, y) in single frames modulation template P_p(x, y) is：

I_p(x, y)=I₁(x,y)·I_c1(x,y)+I₂(x,y)·I_c2(x, y),

Wherein I₁(x,y)、I₂(x,y)、I_c1(x,y)、I_c2(x, y) is respectively cosine templates I₁, De Bruijn sequence templates I₂And carrier wave template I_c1、I_c2Intensity level, and I₁(x,y)、I₂Each pixel intensity is along y direction changes, I in (x, y)_c1 (x,y)、I_c2Each pixel intensity is along x direction changes in (x, y)；

(2) the projecting apparatus T in system and video camera V is placed vertically, and makes both optical axises parallel, will be single with projecting apparatus T Frame modulation template P is projected on target object O；And the target object is shot with video camera V, obtain by target object Deformation pattern I after O modulation；

(3) deformation pattern I is transmitted to computer C, is demodulated by computer C and obtain demodulation cosine templates I '₁And solution Adjust De Bruijn sequence templates I '₂；

(4) demodulation cosine templates I ' is calculated with Gabor filter₁In each pixel (x, y) place block phase

(5) according to demodulation De Bruijn sequence templates I '₂And block phaseSolve pixel in deformation pattern I Phase unwrapping value Φ (x, y) at (x, y) place；

(6) according to phase unwrapping value Φ (x, y) of pixel (x, y) in deformation pattern I, pixel (x, y) is asked in list Match point (x in frame modulation template P_p,y_p), wherein x, y are the ranks coordinate value of pixel in deformation pattern I, x_p、y_pIt is list The ranks coordinate value of pixel in frame modulation template P；

(7) according to principle of triangulation, using pixel (x, y) and match point (x_p,y_p) space geometry relation, calculate The depth information value of object under test.

The present invention has advantages below compared with prior art：

First, the present invention uses single frames structure optical mode plate, it is only necessary to the depth information of scene is obtained by projecting a width template, Implementation process is simplified, the requirement to system equipment is reduced；

Second, the present invention is modulated so that phase on the basis of striped template with reference to De Bruijn sequence templates The De accurate phase cyclings of Bruijn retrievals can be utilized during unwrapping, the degree of accuracy of unwrapping is improved, it is to avoid Ambiguous generation, increased the precision of object depth information acquisition；

3rd, the present invention only needs projection single frames modulation template measurement depth information of scene, is accelerated by using GPU, can be real The dynamic measurement of existing depth information of scene.

Brief description of the drawings

Fig. 1 is the system framework figure that the present invention is used；

Fig. 2 realizes general flow chart for of the invention；

Fig. 3 is the sub-process figure of generating structure optical mode plate in the present invention；

Fig. 4 is the sub-process figure of demodulation single frames modulation template in the present invention.

Specific embodiment

The present invention is the improvement to traditional structure light method, it is not necessary to increase extra measuring apparatus and measuring process, with It is lower the present invention is described in further detail referring to the drawings.

Reference picture 1, the Depth Information Acquistion system that the present invention is used includes：Target object O, video camera V, throwing to be measured Shadow instrument T and computer C.On projecting apparatus T projections single frames modulation template P to target object O, video camera V shoots to the target, Obtain the deformation pattern I after target object O is modulated.

Reference picture 2, depth information measuring method of the present invention based on single frames modulation template, implementation step is as follows：

Step 1, design single frames modulation template P.

Reference picture 3, this step is implemented as follows：

(1a) design is based on cosine templates I₁With based on De Bruijn sequence templates I₂；

(1b) design frequency is f_c1Carrier wave template I_c1It is f with frequency_c2Carrier wave template I_c2, and according to the two carrier waves Template I_c1、I_c2With above-mentioned cosine templates I₁, De Bruijn sequence templates I₂, obtain single frames modulation template P：

P=I₁·I_c1+I₂·I_c2；

The intensity level of each pixel (x, y) is in (1c) calculating single frames modulation template P：

I_p(x, y)=I₁(x,y)·I_c1(x,y)+I₂(x,y)·I_c2(x, y),

Wherein I₁(x,y)、I₂(x, y) is respectively cosine templates I₁, De Bruijn sequence templates I₂Intensity level, I_c1(x, y)、I_c2(x, y) is respectively carrier wave template I_c1、I_c2Intensity level, and I₁(x,y)、I₂Intensity level is along y direction changes in (x, y), I_c1(x,y)、I_c2Intensity level is along x direction changes in (x, y).

Step 2, acquisition deformation pattern I.

Projecting apparatus T in system and video camera V is placed vertically, and makes both optical axises parallel, will be described with projecting apparatus T Single frames composite shuttering P is projected on target object O；And the target object is shot with video camera V, obtain by object Deformation pattern I after body O modulation.

Step 3, deformation pattern I is transmitted to computer C, demodulated by computer C and obtain demodulation cosine templates I '₁And Demodulation De Bruijn sequence templates I '₂。

Reference picture 4, the specific implementation of this step is as follows：

(3a) carries out bandpass filtering to the deformation pattern I for obtaining, and extracts the modulation image for carrying cosine templatesIts The intensity level at pixel (x, y) place

Wherein I '₁(x, y) is the intensity level of cosine templates,It is the carrier frequency disturbed by scene；

(3b) carries out square operation to the result of above-mentioned (3a), and the intensity level obtained on each pixel (x, y) is：

(3c) carries out extracting operation again after carrying out LPF to the result of (3b), demodulation is obtained from deformation pattern I remaining String template I '₁；

(3d) carries out bandpass filtering to deformation pattern I, extracts the modulation image for carrying De Bruijn sequence templates Its intensity level at pixel (x, y) place

Wherein I '₂(x, y) is the intensity level of De Bruijn sequence templates,It is the carrier frequency disturbed by scene；

(3e) carries out square operation to above-mentioned (3d) result, and the intensity level obtained on each pixel (x, y) is：

(3f) carries out extracting operation again after carrying out LPF to the result of (3e), obtains demodulating De from deformation pattern I Bruijn sequence templates I '₂。

Step 4, demodulation cosine templates I ' is calculated with Gabor filter₁In each pixel (x, y) place block phase

(4a) will demodulate cosine templates I '₁Convolution is carried out with Gaussian kernel, convolution results z (x, y) are obtained：

Z (x, y)=I '₁(x, y) * g (x, y, λ),

Wherein, * represents convolution symbol, and g (x, y, λ) is Gaussian kernel, and λ is the wavelength of Gaussian filter, I '₁(x, y) is solution Adjust cosine templates I '₁Intensity level；

(4b) blocks phase according to convolution results z (x, y), calculating

Wherein, Im (z (x, y)) is the complex number part of z (x, y), and Re (z (x, y)) is the real part of z (x, y), and calculating is obtained Block phase be limited in [- π, π) in.

Step 5, according to demodulation De Bruijn sequence templates I '₂And block phaseSolve picture in deformation pattern I Phase unwrapping value Φ (x, y) at vegetarian refreshments (x, y) place.

(5a) is to demodulation cosine templates I '₁Carry out continuum division：

Define zoning function

WhereinAny pixel point blocks phase value,To block phase value on i-th direction neighborhood of the pixel, Wherein i=1,2 ..., 8,

Define τ₁It is the first neighborhood thresholding, τ₂It is the second neighborhood thresholding,

The pixel and its neighborhood that following condition will be met are divided into the same area：

WhereinIt is the phase value that blocks on n-th direction neighborhood, n=3,7；To block phase on m-th neighborhood Value, m=1,2,4,5,6,8；

(5b) on the basis of continuum divides, to demodulation De Bruijn sequence templates I '₂Enter line period division, ask Demodulation De Bruijn sequence template code values are solved, phase cycling n is blocked in acquisition；

(5c) calculates phase unwrapping value Φ (x, y) of pixel (x, y) in deformation pattern I：

WhereinIt is the phase value that blocks of pixel (x, y), n is to block phase cycling.

Step 6, phase unwrapping value Φ (x, y) according to pixel (x, y) in deformation pattern I, ask for pixel (x, y) and exist Match point (x in single frames modulation template P_p,y_p), wherein x, y are the ranks coordinate value of pixel in deformation pattern I, x_p、y_pFor The ranks coordinate value of pixel in single frames modulation template P.

Step 7, according to principle of triangulation, using pixel (x, y) and match point (x_p,y_p) space geometry relation, meter Calculate the depth information value of object under test.

(7a) sets up world coordinate system by origin of the photocentre of video camera V, and wherein the x-axis of world coordinate system is along video camera V The horizontal direction of imaging plane, the y-axis of world coordinate system along video camera V imaging planes vertical direction, the z of world coordinate system Axle along video camera V optical axis direction, and and imaging plane perpendicular；

(7b) calculates pixel (x, y) and match point (x in deformation pattern I_p,y_p) between parallax Δ_y：

Wherein T is the periodic quantity of striped, and Φ (x, y) is phase unwrapping value, y_pIt is the y directions coordinate value of match point；

(7c) calculates the corresponding depth information value d (x, y) of pixel (x, y)：

Wherein Δ_yIt is parallax, f is expressed as the focal length of video camera, and b is the parallax range between video camera and projecting apparatus.

Above description is only one of the invention specific example, does not constitute any limitation of the invention.Obviously for For one of skill in the art, after present invention and principle has been understood, all may be without departing substantially from the principle of the invention, structure In the case of, various amendments and the change in form and details are carried out, but these are based on the amendment and change of inventive concept Still within claims of the invention.

Claims (6)

1. a kind of depth information measuring method based on single frames modulation template, including：

(1) design single frames modulation template P：

(1a) design is based on cosine templates I₁With based on De Bruijn sequence templates I₂；

(1b) design frequency is f_c1Carrier wave template I_c1It is f with frequency_c2Carrier wave template I_c2, and according to the two carrier wave templates I_c1、I_c2With above-mentioned cosine templates I₁, De Bruijn sequence templates I₂Obtain single frames modulation template P：

P=I₁·I_c1+I₂·I_c2；

(1c) calculates the intensity level I of each pixel (x, y) in single frames modulation template P_p(x, y) is：

I_p(x, y)=I₁(x,y)·I_c1(x,y)+I₂(x,y)·I_c2(x, y),

Wherein I₁(x,y)、I₂(x,y)、I_c1(x,y)、I_c2(x, y) is respectively cosine templates I₁, De Bruijn sequence templates I₂With And carrier wave template I_c1、I_c2Intensity level, and I₁(x,y)、I₂Each pixel intensity is along y direction changes, I in (x, y)_c1(x, y)、I_c2Each pixel intensity is along x direction changes in (x, y)；

(2) the projecting apparatus T in system and video camera V is placed vertically, and makes both optical axises parallel, adjusted single frames with projecting apparatus T Pallet P is projected on target object O；And the target object is shot with video camera V, obtain and adjusted by target object O Deformation pattern I after system；

(3) deformation pattern I is transmitted to computer C, is demodulated by computer C and obtain demodulation cosine templates I '₁And demodulation De Bruijn sequence templates I '₂；

(4) demodulation cosine templates I ' is calculated with Gabor filter₁In each pixel (x, y) place block phase

(5) according to demodulation De Bruijn sequence templates I '₂And block phaseSolve pixel in deformation pattern I (x, Y) phase unwrapping value Φ (x, y) at place；

(6) according to phase unwrapping value Φ (x, y) of pixel (x, y) in deformation pattern I, ask for pixel (x, y) and adjusted in single frames Match point (x in pallet P_p,y_p), wherein x, y are the ranks coordinate value of pixel in deformation pattern I, x_p、y_pFor single frames is adjusted The ranks coordinate value of pixel in pallet P；

(7) according to principle of triangulation, using pixel (x, y) and match point (x_p,y_p) space geometry relation, calculate it is to be measured The depth information value of object.

2. the method according to right 1, demodulation cosine templates I ' is obtained wherein in step (3) by computer C₁, by following step Suddenly carry out：

(3.1) bandpass filtering is carried out to the deformation pattern I for obtaining, extracts the modulation image for carrying cosine templatesIt is in picture The intensity level at vegetarian refreshments (x, y) place

$I_1^b(x,y)=I_1^{\′}(x,y)\·cos\left(2{\mathrm{\πf}}_{c_1}^{\′}x\right),$

Wherein I '₁(x, y) is the intensity level of cosine templates,It is the carrier frequency disturbed by scene；

(3.2) result to above-mentioned (3.1) carries out square operation, and the intensity level obtained on each pixel (x, y) is：

${\left(I_1^b\right(x,y\left)\right)}^2={\left(I_1^{\′}\right(x,y\left)\right)}^2\·\frac{1+cos(2\mathrm{\π}\·2f_{c_1}^{\′}x)}{2};$

(3.3) extracting operation is carried out again after carrying out LPF to the result of (3.2), obtains demodulating cosine from deformation pattern I Template I '₁。

3. the method according to right 1, wherein step (3) obtain demodulation De Bruijn sequence templates I ' by computer C₂, Carry out as follows：

(3.4) bandpass filtering is carried out to the deformation pattern I for obtaining, extracts the modulation image for carrying De Bruijn sequence templatesIts intensity level at pixel (x, y) place

$I_2^b(x,y)=I_2^{\′}(x,y)\·cos\left(2{\mathrm{\πf}}_{c_2}^{\′}x\right),$

Wherein I '₂(x, y) is the intensity level of De Bruijn sequence templates,It is the carrier frequency disturbed by scene；

(3.5) result to above-mentioned (3.4) carries out square operation, and the intensity level obtained on each pixel (x, y) is

${\left(I_2^b\right(x,y\left)\right)}^2={\left(I_2^{\′}\right(x,y\left)\right)}^2\·\frac{1+cos(2\mathrm{\π}\·2f_{c_2}^{\′}x)}{2};$

(3.6) extracting operation is carried out again after carrying out LPF to the result of (3.5), obtains demodulating De from deformation pattern I Bruijn sequence templates I '₂。

4. the method according to right 1, demodulation cosine templates I ' is calculated wherein in step (4) with Gabor filter₁In each Phase is blocked at pixel (x, y) placeCarry out as follows：

(4.1) will demodulation cosine templates I '₁Convolution is carried out with Gaussian kernel, convolution results z (x, y) are obtained：

Z (x, y)=I '₁(x,y)*g(x,y,λ)

Wherein, * represents convolution symbol, and g (x, y, λ) is Gaussian kernel, and λ is the wavelength of Gaussian filter, I '₁(x, y) is demodulation cosine Template I '₁Intensity level；

(4.2) according to convolution results z (x, y), phase is blocked in calculating

Wherein, Im (z (x, y)) is the complex number part of z (x, y), and Re (z (x, y)) is the real part of z (x, y), calculates what is obtained Block phase be limited in [- π, π) in.

5. the method according to right 1, according to demodulation De Bruijn sequence templates I wherein in step (5)₂' and block phase PositionPhase unwrapping value Φ (x, y) at pixel (x, y) place in deformation pattern I is solved, is carried out as follows：

(5.1) to demodulation cosine templates I '₁(x, y) carries out continuum division：

Define zoning function

WhereinAny pixel point blocks phase value,To block phase value on i-th direction neighborhood of the pixel, wherein I=1,2 ..., 8,

Define τ₁It is the first neighborhood thresholding, τ₂It is the second neighborhood thresholding,

The pixel and its neighborhood that following condition will be met are divided into the same area：

WhereinIt is the phase value that blocks on n-th direction neighborhood, n=3,7；It is the phase value that blocks on m-th neighborhood, m =1,2,4,5,6,8；

(5.2) on the basis of continuum divides, to demodulation De Bruijn sequence templates I '₂Enter line period division, solve Phase cycling n is blocked in demodulation De Bruijn sequence template code values, acquisition；

(5.3) phase unwrapping value Φ (x, y) of pixel (x, y) in deformation pattern I is calculated：

WhereinIt is the phase value that blocks of pixel (x, y), n is to block phase cycling.

6. the method according to right 1, according to principle of triangulation wherein in step (7), using pixel (x, y) with match Point (x_p,y_p) space geometry relation, calculate object under test depth information value, carry out as follows：

(7.1) world coordinate system is set up by origin of the photocentre of video camera V, the wherein x-axis of world coordinate system is imaged along video camera V The horizontal direction of plane, the y-axis of world coordinate system along video camera V imaging planes vertical direction, the z-axis edge of world coordinate system The optical axis direction of video camera V, and and imaging plane perpendicular；

(7.2) pixel (x, y) and match point (x in deformation pattern I are calculated_p,y_p) between parallax Δ_y：

${\mathrm{\Δ}}_y=\frac{\mathrm{\Φ}(x,y)}{2\mathrm{\π}}\×T-y_p,$

Wherein T is the periodic quantity of striped, and Φ (x, y) is phase unwrapping value, y_pIt is the y directions coordinate value of match point；

(7.3) the corresponding depth information value d (x, y) of pixel (x, y) is calculated：

$d(x,y)=\frac{f\×b}{{\mathrm{\Δ}}_y},$

Wherein Δ_yIt is parallax, f is expressed as the focal length of video camera, and b is the parallax range between video camera and projecting apparatus.