WO2018219156A1 - 结构光编码方法、装置及终端设备 - Google Patents

结构光编码方法、装置及终端设备 Download PDF

Info

Publication number
WO2018219156A1
WO2018219156A1 PCT/CN2018/087373 CN2018087373W WO2018219156A1 WO 2018219156 A1 WO2018219156 A1 WO 2018219156A1 CN 2018087373 W CN2018087373 W CN 2018087373W WO 2018219156 A1 WO2018219156 A1 WO 2018219156A1
Authority
WO
WIPO (PCT)
Prior art keywords
coding
pattern
coding pattern
black
determining
Prior art date
Application number
PCT/CN2018/087373
Other languages
English (en)
French (fr)
Inventor
唐苏明
宋展
李丽
王维
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018219156A1 publication Critical patent/WO2018219156A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object

Definitions

  • the present application relates to image processing technologies, and in particular, to a structured optical coding method, apparatus, and terminal device.
  • the structured light three-dimensional scanning technology realizes three-dimensional scanning by actively controlling the light source, and has higher reliability than other three-dimensional scanning technologies, and has great development potential.
  • structured light three-dimensional scanning technology has been widely used in industrial detection, 3D printing, film and television special effects production, cultural relics and other fields.
  • the structured light three-dimensional scanning technology refers to projecting the designed coding pattern to the target object, and the camera captures the target object to form another coding pattern, and determines the feature points of the coding pattern, and adopts optical trigonometry and system calibration methods for the feature points. To obtain three-dimensional information of the target object, thereby achieving three-dimensional reconstruction of the target object.
  • the structured optical coding method is the premise and basis of the structured light three-dimensional scanning technology.
  • the coding strategy of the structured optical coding technology is mainly divided into time coding and spatial coding.
  • spatial coding is more advantageous than time coding in terms of measurement speed, making it easy to implement dynamic 3D reconstruction.
  • spatial coding has the problem that the coding density and the coding window size are contradictory, that is, the larger the coding density, the larger the coding window.
  • the larger the coding window the higher the difficulty of decoding.
  • the prior art achieves the purpose of reducing the window size and increasing the encoding density by adding color information or brightness information.
  • the color information or brightness information of the projection affects the acquisition of the three-dimensional information of the target object, which causes the problem of poor robustness of the entire structured optical coding system.
  • the present application provides a structured optical coding method, apparatus and terminal device, thereby improving the robustness of the structured optical coding system.
  • the present application provides a structured optical coding method, including: generating a pseudo-random array with an encoding window of 2*2 and an encoding element of N; wherein the N coding elements are represented by N different black and white binary geometric figures, N is a positive integer greater than or equal to 8; a rectangular pattern formed by a black and white diamond pattern is generated; a black and white binary geometry is embedded in the white diamond pattern as a foreground pattern of the first coding pattern, and the black diamond pattern is used as the first code A background pattern of the pattern to generate a first encoded pattern.
  • the beneficial effects of the present application are: since the present application implements encoding by N kinds of black and white binary geometry, the black and white binary geometry is stronger for the texture, color and reflection characteristics of the target object than the conventional color and brightness encoding methods. Robustness can be used for more complex dynamic 3D reconstruction of the surface of the target object. Thereby improving the robustness of the structured optical coding system.
  • the method further includes: projecting the first coding pattern to the target object; capturing the projected target object, acquiring the second coding pattern; determining feature points of the second coding pattern, wherein the feature point is used to determine the three-dimensional contour of the target object information.
  • determining a feature point of the second coding pattern includes: dividing the second coding pattern into a plurality of first regions; determining at least one candidate feature point in each of the first regions, and determining among all the candidate feature points Feature points.
  • determining at least one candidate feature point in each of the first regions including: passing a formula Calculating any pixel point (i, j) in each first region, corresponding to the value H; wherein I(i+ ⁇ , j+ ⁇ ) represents the gray value of the pixel point (i+ ⁇ , j+ ⁇ )
  • I(i+ ⁇ , j+ ⁇ ) represents the gray value of the pixel point (i+ ⁇ , j+ ⁇ )
  • the maximum value of the value is determined in each of the first regions, and the pixel corresponding to the maximum value is taken as the candidate feature point.
  • the true feature points of the second coding pattern should also have this feature. That is, the value H calculated by the X-type template is the largest in the first region, and this method can be used to eliminate most of the ordinary points.
  • determining the feature point among all candidate feature points includes: determining a second region where any candidate feature point is located, and determining a symmetric region after the second region is rotated by 180 degrees; Calculating the symmetry coefficient ⁇ of any candidate feature point; where I(m,n) represents the gray value of the pixel point (m,n); I'(m,n) represents the pixel point (m,n) in the symmetrical region Gray value of the symmetry point; An average value of gray values of all the pixels in the second region is determined; and candidate feature points whose symmetry coefficient is smaller than a preset threshold are determined as feature points.
  • the feature points in the first coding pattern have rotationally invariant symmetry (rotation 180 degrees)
  • the feature points in the second coding pattern substantially follow this feature.
  • the symmetry coefficient ⁇ represents the degree of local symmetry of the second coding pattern, and the smaller the symmetry coefficient ⁇ , the better the degree of local symmetry of the second coding pattern. Based on this, the symmetry coefficient ⁇ of each candidate feature point is calculated by using the above formula, and by setting a preset threshold value, the erroneous feature points in the candidate feature points are eliminated, and the final feature points can be effectively obtained.
  • the structure of the optical coding device can be used to perform the first aspect and the corresponding alternative manner of the first aspect.
  • the implementation principle and technical effects are similar, and details are not described herein again.
  • the present application provides a structured optical coding apparatus, including: a first generation module, configured to generate a pseudo-random array with an encoding window of 2*2 and an encoding element of N; wherein the N coding elements pass N different types A black and white binary geometrical representation, N is a positive integer greater than or equal to 8; a second generation module for generating a rectangular pattern of black and white diamond patterns; and a third generation module for embedding black and white binary geometry in white In the diamond pattern, as the foreground pattern of the first coding pattern, a black diamond pattern is used as the background pattern of the first coding pattern to generate a first coding pattern.
  • the method further includes: a projection module, configured to project the first coding pattern to the target object; an acquisition module, configured to capture the projected target object, obtain a second coding pattern; and a determining module, configured to determine the second coding pattern Feature points, wherein the feature points are used to determine three-dimensional contour information of the target object.
  • a projection module configured to project the first coding pattern to the target object
  • an acquisition module configured to capture the projected target object, obtain a second coding pattern
  • a determining module configured to determine the second coding pattern Feature points, wherein the feature points are used to determine three-dimensional contour information of the target object.
  • the determining module is specifically configured to: divide the second coding pattern into a plurality of first regions; determine at least one candidate feature point in each of the first regions, and determine the feature points among all the candidate feature points.
  • the determining module is specifically configured to: pass the formula Calculating any pixel point (i, j) in each first region, corresponding to the value H; wherein I(i+ ⁇ , j+ ⁇ ) represents the gray value of the pixel point (i+ ⁇ , j+ ⁇ ) The maximum value of the value is determined in each of the first regions, and the pixel corresponding to the maximum value is taken as the candidate feature point.
  • the determining module is specifically configured to: determine a second area where any candidate feature point is located, and determine a symmetric area after the second area is rotated by 180 degrees;
  • the terminal device is described below, and the terminal device can be used to perform the first aspect and the corresponding mode corresponding to the first aspect.
  • the implementation principle and the technical effect are similar, and details are not described herein again.
  • the application provides a terminal device, including: a structured light template, configured with a first coding pattern; wherein the first coding pattern is a pseudo-random array with an encoding window of 2*2, an encoding element of N, and a plurality of black and white A rectangular pattern formed by interphase diamond patterns is generated; N coding elements are represented by N different black and white binary geometric figures, N is a positive integer greater than or equal to 8; black and white binary geometric figures are embedded in corresponding white diamond patterns to form a foreground pattern of the first coding pattern, the black diamond pattern forms a background pattern of the first coding pattern; a projection unit for generating projection light; forming a first coding pattern by the generated projection light through the structured light template, and the first coding pattern Projecting to a target object; a photographing unit for photographing the projected target object, acquiring a second encoding pattern; a processor for determining feature points of the second encoding pattern; and decoding each feature point to determine each feature Pointing at
  • the processor is specifically configured to: pass a formula Calculating any pixel point (i, j) in each first region, corresponding to the value H; wherein I(i+ ⁇ , j+ ⁇ ) represents the gray value of the pixel point (i+ ⁇ , j+ ⁇ ) The maximum value of the value is determined in each of the first regions, and the pixel corresponding to the maximum value is taken as the candidate feature point.
  • the processor is specifically configured to: determine a second region where any candidate feature point is located, and determine a symmetric region after the second region is rotated by 180 degrees; Calculating the symmetry coefficient ⁇ of any candidate feature point; where I(m,n) represents the gray value of the pixel point (m,n); I'(m,n) represents the pixel point (m,n) in the symmetrical region Gray value of the symmetry point; An average value of gray values of all the pixels in the second region is determined; and candidate feature points whose symmetry coefficient is smaller than a preset threshold are determined as feature points.
  • the present application provides a structured optical coding method, apparatus, and terminal device, including: generating a pseudo-random array with an encoding window of 2*2 and an encoding element of N; wherein the N coding elements are represented by N different black and white binary geometric figures , N is a positive integer greater than or equal to 8; a rectangular pattern formed by a black and white diamond pattern is generated; a black and white binary geometry is embedded in the white diamond pattern as the foreground pattern of the first coding pattern, and the black diamond pattern is taken as the first A background pattern of the pattern is encoded to generate a first encoded pattern.
  • the coding window of the first coding pattern is 2*2, and the coding window has reached the minimum coding window.
  • the present application can ensure that the coding density (the 2*2 coding window includes two types of feature points) is sufficiently large. Further, since the present application implements encoding by N kinds of black and white binary geometry, the black and white binary geometry is more robust to the texture, color and reflection characteristics of the target object than the conventional color and brightness coding methods. It can be used for more complex dynamic 3D reconstruction of the surface of the target object. Thereby improving the robustness of the structured optical coding system.
  • FIG. 1 is a schematic diagram of a structured optical coding system according to an embodiment of the present application.
  • FIG. 2 is a flowchart of a structured optical coding method according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a first coding pattern provided by the present application.
  • FIG. 4 is a partial enlarged view of a first coding pattern provided by the present application.
  • FIG. 5 is a schematic structural diagram of a structure optical coding apparatus according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • Structured light technology can be divided into two categories, one is to project a laser line or a lattice.
  • the advantage of this type is that the structure is simple and the correspondence is clear.
  • the scanning method is point-by-point and line-by-line, the measurement speed is slow, and only applies to For specific occasions.
  • the other type is structured light 3D scanning technology, which is to project single or multiple coding patterns onto the target object to realize one-time measurement of the whole surface. It has obvious advantages of high speed and high precision, and makes up for the point and line structure optical measurement. Insufficient efficiency, so structured light 3D scanning technology has become the mainstream technology.
  • the coding method of structured light three-dimensional scanning technology can be roughly divided into time coding method and spatial coding method.
  • the former projects multiple coding patterns in time series to obtain corresponding coded image sequences
  • the decoding uses stripe image sequence combination to solve the coding pattern and
  • the corresponding problem of the fringe image has the advantages of high measurement accuracy and high measurement resolution, but its measurement speed is slow, so it is suitable for three-dimensional information acquisition of static targets and scenes.
  • the latter only needs to project a coding pattern, and the measurement speed is fast, so it is suitable for three-dimensional information acquisition of dynamic targets and scenes.
  • spatial coding has the problem that the coding density and the coding window size are contradictory, that is, the larger the coding density, the larger the coding window.
  • the prior art achieves the purpose of reducing the coding window size and increasing the coding density by adding color information or luminance information.
  • the color information or brightness information of the projection affects the acquisition of the three-dimensional information of the target object, which causes the problem of poor robustness of the entire structured optical coding system.
  • the present application provides a structured optical coding method, apparatus and terminal device, thereby improving the robustness of the structured optical coding system.
  • the main idea of the present application is to use N kinds of black and white binary geometry as a spatial coding scheme of coding elements. Compared with the traditional color and brightness coding methods, the black and white binary geometry has the texture, color and reflection characteristics of the target object. More robustness, which can be used for more complex dynamic 3D reconstruction of the surface of the target object.
  • FIG. 1 is a schematic diagram of a structured optical coding system according to an embodiment of the present application.
  • the structured optical coding system includes: a smart device 11 , a projection device 12 , and a photographic device 13 and the target object 14.
  • the smart device 11 can be a smart device such as a computer, a notebook computer, a tablet, or a mobile phone.
  • the working principle of the structured optical coding system is that the smart device 11 generates a first coding pattern and transmits it to the projection device 12, the projection device 12 projects the first coding pattern to the target object 14, and the photographic device 13 captures the projected target object. , obtaining a second coding pattern.
  • the photographic device 13 can transmit the second coding pattern to the smart device 11, and the smart device 11 determines feature points of the second coding pattern; and decodes each feature point to determine a corresponding position of each feature point in the first coding pattern. Determining a depth of each feature point in the second coding pattern according to a corresponding position of each feature point in the first coding pattern; determining three-dimensional contour information of the target object according to the depth of each feature point in the second coding pattern .
  • the smart device, the projection device, and the photographic device in the structured optical coding system can be integrated into the terminal device, that is, the terminal device can implement various functions of the above-mentioned structured optical coding system.
  • the smart device corresponds to a processor of the terminal device
  • the projection device corresponds to a projection unit of the terminal device
  • the photographic device corresponds to a photographic unit of the terminal device.
  • the present application provides a structured optical coding method, apparatus, and terminal device. To improve the robustness of the structured optical coding system.
  • FIG. 2 is a flowchart of a structured optical coding method according to an embodiment of the present application. As shown in FIG. 2, the method includes:
  • Step S201 generating a pseudo-random array with an encoding window of 2*2 and an encoding element of N; wherein the N coding elements are represented by N different black and white binary geometric figures, and N is a positive integer greater than or equal to 8;
  • Step S202 generating a rectangular pattern formed by a black and white diamond pattern
  • Step S203 embedding the black and white binary geometry into the white diamond pattern as the foreground pattern of the first coding pattern, and using the black diamond pattern as the background pattern of the first coding pattern to generate the first coding pattern.
  • the pseudo-random sequence is converted to a pseudo-random array using the folding principle.
  • the pseudo random array has an encoding window of 2*2 and a coding element of 8. Generating a rectangular pattern of black and white diamond patterns; embedding black and white binary geometry in a white diamond pattern in a black and white diamond pattern as a foreground pattern of the first coding pattern, and a black diamond pattern in a black and white diamond pattern
  • the background pattern of the first coding pattern is used to generate a first coding pattern.
  • 3 is a schematic diagram of a first coding pattern provided by the present application
  • FIG. 4 is a partial enlarged view of a first coding pattern provided by the present application. As shown in FIG. 3 and FIG. 4, the coding window of the first coding pattern is 2*. 2, the coding element is 8, and the coding capacity is 65 ⁇ 63.
  • the first coding pattern feature points are defined as intersections of adjacent diamond elements, so the first coding pattern includes two types of feature points, one is a P1 point as shown in FIG. 3, and the P1 feature points are up and down. It is a background element; the other is the P2 point as shown in Fig. 3, and the left and right of the P2 feature point are background elements.
  • the definition of the codeword for each feature point is: if it is a P1 type feature point, the codeword is composed of the left and right elements and the two elements below the left and right elements; if it is a P2 type feature point, the codeword is up and down The element and the elements on the right side of the upper and lower elements. Therefore, as shown in FIG. 3, the P1 feature point and the P2 feature point have the same code word, that is, c1-c2-c3-c4.
  • the 2*2 coding window includes two types of feature points.
  • the black and white rhombic pattern of the present application may be a black and white square pattern or a pattern obtained by stretching and rotating the diamond pattern. This application does not limit this.
  • the first coding pattern generated by the structured optical coding method provided by the present application has an encoding window of 2*2, and the coding window has reached a minimum coding window. Moreover, the present application can ensure that the coding density (the 2*2 coding window includes two types of feature points) is sufficiently large. Further, since the present application implements encoding by N kinds of black and white binary geometry, the black and white binary geometry is more robust to the texture, color and reflection characteristics of the target object than the conventional color and brightness coding methods. It can be used for more complex dynamic 3D reconstruction of the surface of the target object. Thereby improving the robustness of the structured optical coding system.
  • the method further includes: projecting the first coding pattern to the target object; capturing the projected target object, acquiring the second coding pattern; determining a feature point of the second coding pattern, wherein the feature point is used to determine the target object 3D contour information.
  • the feature points of the second coding pattern are in one-to-one correspondence with the feature points of the first coding pattern.
  • a method for determining a feature point of the second coding pattern comprising: dividing the second coding pattern into a plurality of first regions; determining at least one candidate feature point in each of the first regions, and determining features among all the candidate feature points point.
  • the first area may generally be selected from the above black diamond pattern or 2/3 of the size of the white diamond pattern.
  • the second coding pattern is converted into a gray image, and the gray image is smoothed by using a filter; then an X-type template is designed, and the smoothed image is convoluted, and the local non-maximum is selected.
  • the value suppression method is used to determine the candidate feature points; wherein the local non-maximum suppression method is selected to determine the candidate feature points, including:
  • the true feature points of the second coding pattern should also have this feature. That is, the value H calculated by the X-type template is the largest in the first region, and this method can be used to eliminate most of the ordinary points.
  • determining the feature points among all the candidate feature points includes: determining a second region where any candidate feature points are located, and determining a symmetric region after the second region is rotated by 180 degrees.
  • the feature points in the first coding pattern have rotationally invariant symmetry (rotation 180 degrees)
  • the feature points in the second coding pattern substantially follow this feature.
  • the symmetry coefficient ⁇ represents the degree of local symmetry of the second coding pattern, and the smaller the symmetry coefficient ⁇ , the better the degree of local symmetry of the second coding pattern. Based on this, the symmetry coefficient ⁇ of each candidate feature point is calculated by using the above formula, and a wrong threshold point in the candidate feature points is removed by setting a preset threshold value to obtain the final feature point.
  • the feature points on the second coding pattern can be accurately determined by the above two formulas, thereby ensuring accurate acquisition of the three-dimensional information of the target object.
  • FIG. 5 is a schematic structural diagram of a structure optical coding apparatus according to an embodiment of the present invention. As shown in FIG. 5, the apparatus includes:
  • the first generation module 501 is configured to generate a pseudo-random array with an encoding window of 2*2 and an encoding element of N; wherein the N coding elements are represented by N different black and white binary geometric figures, and N is greater than or equal to 8 Integer.
  • the second generation module 502 is configured to generate a rectangular pattern formed by a black and white diamond pattern.
  • the third generation module 503 is configured to embed the black and white binary geometry into the white diamond pattern as the foreground pattern of the first coding pattern, and use the black diamond pattern as the background pattern of the first coding pattern to generate the first coding pattern.
  • the method further includes: a projection module 504, configured to project the first coding pattern to the target object; an acquisition module 505, configured to capture the target object after the projection, to obtain a second coding pattern, and a determining module 506, configured to determine A feature point of the two coding patterns, wherein the feature points are used to determine three-dimensional contour information of the target object.
  • a projection module 504 configured to project the first coding pattern to the target object
  • an acquisition module 505 configured to capture the target object after the projection, to obtain a second coding pattern
  • a determining module 506 configured to determine A feature point of the two coding patterns, wherein the feature points are used to determine three-dimensional contour information of the target object.
  • the generating function of the first generating module 501, the second generating module 502, and the third generating module 503 may be a real-time generating function.
  • the first generation module 501 needs to generate a pseudo-random array
  • the second generation module 502 needs to generate a rectangular pattern formed by a black and white diamond pattern
  • the third generation module 503 needs The first coding pattern is generated once.
  • the generating function of the first generating module 501, the second generating module 502, and the third generating module 503 may also be an extracting or maintaining function.
  • the third generation module 503 has generated the first coding pattern in the first structured optical coding process, and the generated first coding pattern is saved in the storage module of the structured optical coding device. Next, in each structured optical coding process, the third generation module 503 only needs to extract the first coding pattern in the storage module.
  • the projection module 504 can project the first coding pattern to the target object using the projection light.
  • the acquisition module 505 has a projection light collection capability to capture the projected target object and acquire a second coding pattern.
  • the projection light may be a light emitting diode (LED) or an infrared light emitted by an infrared laser source.
  • the acquisition module 505 has an infrared light collection capability to capture the projected target object and acquire a second coding pattern.
  • the determining module 506 is specifically configured to: divide the second coding pattern into a plurality of first regions; determine at least one candidate feature point in each of the first regions, and determine the feature points among all the candidate feature points .
  • the determining module 506 is specifically configured to:
  • the determining module 505 is specifically configured to: determine a second area where any candidate feature point is located, and determine a symmetric area after the second area is rotated by 180 degrees;
  • the first coding pattern generated by the structured optical coding apparatus provided by the present application has an encoding window of 2*2, and the coding window has reached a minimum coding window. Moreover, this application can ensure that the coding density is large enough. Further, since the present application implements encoding by N kinds of black and white binary geometry, the black and white binary geometry is more robust to the texture, color and reflection characteristics of the target object than the conventional color and brightness coding methods. It can be used for more complex dynamic 3D reconstruction of the surface of the target object. Thereby improving the robustness of the structured optical coding system.
  • FIG. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • the terminal device includes: a structured optical template 601 is provided with a first coding pattern; wherein the first coding pattern is encoded by the coding window. *2, a pseudo-random array with a coding element of N and a rectangular pattern of a plurality of black and white diamond patterns are generated; N coding elements are represented by N different black and white binary geometric figures, and N is a positive integer greater than or equal to 8.
  • the black and white binary geometry is embedded in the corresponding white diamond pattern to form a foreground pattern of the first coding pattern, the black diamond pattern forms a background pattern of the first coding pattern;
  • the projection unit 602 is configured to generate projection light; and the generated projection light is generated
  • the first coding pattern is formed by the structured light template 601, and the first coding pattern is projected to the target object;
  • the photographing unit 603 is configured to capture the projected target object, and obtain the second coding pattern;
  • the processor 604 is configured to determine the second Coding feature points of the pattern; and decoding each feature point to determine a corresponding position of each feature point in the first coding pattern;
  • the corresponding positions of the feature points in the first coding pattern determine the depth of each feature point in the second coding pattern; and the three-dimensional contour information of the target object is determined according to the depth of each feature point in the second coding pattern.
  • the terminal device has generated the first coding pattern, and based on this, a structured light template 601 is formed.
  • the structured light template 601 is provided with a first coding pattern.
  • a first coding pattern is formed and the first coding pattern can be projected onto a target object. That is to say, the terminal device provided by the present application does not need to generate the first coding pattern every time, and only needs to generate the first coding pattern once, and then realizes the projection through the structured light template 601 to generate the first coding pattern.
  • the processor 604 is specifically configured to: divide the second coding pattern into a plurality of first regions; determine at least one candidate feature point in each of the first regions, and determine the feature points among all the candidate feature points .
  • the processor 604 is specifically configured to: pass a formula Calculating any pixel point (i, j) in each first region, corresponding to the value H; wherein I(i+ ⁇ , j+ ⁇ ) represents the gray value of the pixel point (i+ ⁇ , j+ ⁇ ) The maximum value of the numerical value is determined in each of the first regions, and the pixel corresponding to the maximum value is taken as the candidate feature point.
  • the processor 604 is specifically configured to: determine a second area where any candidate feature point is located, and determine a symmetric area after the second area is rotated by 180 degrees;
  • the first coding pattern used by the terminal device provided by the application has an encoding window of 2*2, and the encoding window has reached a minimum encoding window. Moreover, this application can ensure that the coding density is large enough. Further, since the present application implements encoding by N kinds of black and white binary geometry, the black and white binary geometry is more robust to the texture, color and reflection characteristics of the target object than the conventional color and brightness coding methods. It can be used for more complex dynamic 3D reconstruction of the surface of the target object. Thereby improving the robustness of the structured optical coding system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

本申请提供一种结构光编码方法、装置及终端设备，包括：生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数；生成黑白相间的菱形图案构成的矩形图案；将黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。在保证编码窗口足够小、编码密度足够大的情况下，由于本申请通过N种黑白二值几何图形实现编码，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，从而提高结构光编码***的鲁棒性。

Description

结构光编码方法、装置及终端设备
本申请要求于2017年05月31日提交中国专利局、申请号为201710401331.X、申请名称为“结构光编码方法、装置及终端设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术，尤其涉及一种结构光编码方法、装置及终端设备。
背景技术
结构光三维扫描技术通过主动控制光源来实现三维扫描，与其他三维扫描技术相比具有更高的可靠度，发展潜力巨大。目前结构光三维扫描技术已广泛应用于工业检测、3D打印、影视特效制作、文物仿形等领域。结构光三维扫描技术是指将设计好的编码图案投影至目标物体，由摄像机拍摄目标物体形成另一编码图案，确定该编码图案的特征点，对这些特征点采用光学三角法和***标定法，以获取目标物体的三维信息，从而实现目标物体的三维重建。
结构光编码方法是结构光三维扫描技术的前提和基础，结构光编码技术的编码策略主要分为时间编码和空间编码。相比较而言，空间编码较时间编码在测量速度方面更具优势，易于实现动态三维重建。而空间编码存在编码密度与编码窗口尺寸两者相矛盾的问题，即编码密度越大，则编码窗口就越大。然而，编码窗口越大，则解码的难度就越高。现有技术通过增加颜色信息或亮度信息来达到减小窗口尺寸、增大编码密度的目的。然而当目标物体表面颜色较为丰富、纹理结构较为复杂时，投影的颜色信息或亮度信息会影响目标物体的三维信息的获取，进而造成整个结构光编码***的鲁棒性较差的问题。
发明内容
本申请提供一种结构光编码方法、装置及终端设备，从而提高结构光编码***的鲁棒性。
第一方面，本申请提供一种结构光编码方法，包括：生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数；生成黑白相间的菱形图案构成的矩形图案；将黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。
本申请的有益效果是：由于本申请通过N种黑白二值几何图形实现编码，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，可用于目标物体表面更为复杂的动态三维重建。从而提高结构光编码***的鲁棒性。
可选地，还包括：将第一编码图案投影至目标物体；拍摄投影后的目标物体，获取第二编码图案；确定第二编码图案的特征点，其中特征点用于确定目标物体的三维轮廓信息。
可选地，确定第二编码图案的特征点，包括：将第二编码图案划分为多个第一区域；在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定特征点。
可选地，在每个第一区域中确定至少一个候选特征点，包括：通过公式
Figure PCTCN2018087373-appb-000001
计算每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；在每个第一区域中确定数值的极大值，并将极大值对应的像素点作为候选特征点。
由于在第一编码图案真正的特征点处，水平方向上的点和垂直方向上的点的灰度值的差值在局部区域内是最大的。同样的，第二编码图案真正的特征点也应该具有该特征。即用X型模板计算出的数值H在第一区域内是最大的，利用这个方法可以排除掉绝大部分的普通点。
可选地，在所有候选特征点中确定所述特征点，包括：确定任一候选特征点所在的第二区域，并确定第二区域旋转180度后的对称区域；通过公式
Figure PCTCN2018087373-appb-000002
计算任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示像素点(m,n)在对称区域中的对称点的灰度值；
Figure PCTCN2018087373-appb-000003
表示第二区域中，所有像素点的灰度值的平均值；确定对称系数小于预设阈值的候选特征点为特征点。
由于第一编码图案中的特征点具有旋转不变对称性(旋转180度)，因此第二编码图案中的特征点基本也遵循这个特征。对称系数δ表示第二编码图案的局部对称程度，该对称系数δ越小，表示第二编码图案的局部对称程度越好。基于此，采用上述公式计算每个候选特征点的对称系数δ，并通过设定一个预设阈值，以此剔除候选特征点中的错误特征点，可有有效获取最终的特征点。
下面将介绍结构光编码装置，该结构光编码装置可以用于执行第一方面及第一方面对应的可选方式，其实现原理和技术效果类似，此处不再赘述。
第二方面，本申请提供一种结构光编码装置，包括：第一生成模块，用于生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数；第二生成模块，用于生成黑白相间的菱形图案构成的矩形图案；第三生成模块，用于将黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。
可选地，还包括：投影模块，用于将第一编码图案投影至目标物体；获取模块，用于拍摄投影后的目标物体，获取第二编码图案；确定模块，用于确定第二编码图案的特征点，其中特征点用于确定目标物体的三维轮廓信息。
可选地，确定模块具体用于：将第二编码图案划分为多个第一区域；在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定特征点。
可选地，确定模块具体用于：通过公式
Figure PCTCN2018087373-appb-000004
计算每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；在每个第一区域中确定数值的极大值，并将极大值对应的像素点作为候选特征点。
可选地，确定模块具体用于：确定任一候选特征点所在的第二区域，并确定第二区域旋转180度后的对称区域；
通过公式
Figure PCTCN2018087373-appb-000005
计算任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示像素点(m,n)在对称区域中的对称点的灰度值；
Figure PCTCN2018087373-appb-000006
表示第二区域中，所有像素点的灰度值的平均值；确定对称系数小于预设阈值的候选特征点为特征点。
下面将介绍终端设备，该终端设备可以用于执行第一方面及第一方面对应的可选方式，其实现原理和技术效果类似，此处不再赘述。
第三方面，本申请提供一种终端设备，包括：结构光模板，设置有第一编码图案；其中第一编码图案由编码窗口为2*2、编码元素为N的伪随机阵列和多个黑白相间的菱形图案构成的矩形图案生成；N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数；黑白二值几何图形嵌入对应的白色菱形图案中，形成第一编码图案的前景图案，黑色菱形图案形成第一编码图案的背景图案；投影单元，用于产生投影光；通过产生的投影光经过结构光模板形成第一编码图案，并将第一编码图案投影至目标物体；摄影单元，用于拍摄投影后的目标物体，获取第二编码图案；处理器，用于确定第二编码图案的特征点；并对每个特征点进行解码，确定每个特征点在第一编码图案中的对应位置；根据每个特征点在第一编码图案中的对应位置确定每个特征点在第二编码图案中的深度；根据每个特征点在第二编码图案中的深度，确定目标物体的三维轮廓信息。
可选地，处理器具体用于：通过公式
Figure PCTCN2018087373-appb-000007
计算每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；在每个第一区域中确定数值的极大值，并将极大值对应的像素点作为候选特征点。
可选地，处理器具体用于：确定任一候选特征点所在的第二区域，并确定第二区域旋转180度后的对称区域；通过公式
Figure PCTCN2018087373-appb-000008
计算任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示像素点(m,n)在对称区域中的对称点的灰度值；
Figure PCTCN2018087373-appb-000009
表示第二区域中，所有像素点的灰度值的平均值；确定对称系数小于预设阈值的候选特征点为特征点。
本申请提供一种结构光编码方法、装置及终端设备，包括：生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个编码元素通过N种不同的黑白二值几何图形表示，N 为大于或者等于8的正整数；生成黑白相间的菱形图案构成的矩形图案；将黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。其中第一编码图案的编码窗口为2*2，该编码窗口已达到最小编码窗口。而且本申请可以保证编码密度(2*2的编码窗口包括两种类型的特征点)足够大。进一步地，由于本申请通过N种黑白二值几何图形实现编码，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，可用于目标物体表面更为复杂的动态三维重建。从而提高结构光编码***的鲁棒性。
附图说明
图1为本申请一实施例提供的结构光编码***的示意图；
图2为本申请一实施例提供的一种结构光编码方法的流程图；
图3为本申请提供的第一编码图案的示意图；
图4为本申请提供的第一编码图案的局部放大图；
图5为本申请一实施例提供的一种结构光编码装置的结构示意图；
图6为本申请一实施例提供的一种终端设备的结构示意图。
具体实施方式
结构光技术可以分为两类，一类是投射激光线或者点阵，该类的优点是结构简单，对应关系明确，但由于其扫描方式为逐点和逐线，测量速度慢，仅适用于特定场合。另一类为结构光三维扫描技术，即将单幅或多幅编码图案投射至目标物体，实现整个面的一次性测量，具有速度快和精度高的明显优势，弥补了点、线结构光法测量效率低的不足，因此结构光三维扫描技术已成为目前的主流技术手段。
结构光三维扫描技术的编码方法大致可划分为时间编码法和空间编码法，前者将多幅编码图案按时序投射、得到相应的编码图像序列，解码利用条纹图像序列组合实现，从而解决编码图案和条纹图像的对应问题，具有测量精度较高、测量分辨率高的优点，但其测量速度较慢，因此适合于静态目标和场景的三维信息获取。后者仅需投影一幅编码图案，测量速度较快，因此适合于动态目标和场景的三维信息获取。而空间编码存在编码密度与编码窗口尺寸两者相矛盾的问题，即编码密度越大，则编码窗口就越大。然而，编码窗口越大，则解码的难度就越高。现有技术通过增加颜色信息或亮度信息来达到减小编码窗口尺寸、增大编码密度的目的。然而当目标物体表面颜色较为丰富、纹理结构较为复杂时，投影的颜色信息或亮度信息会影响目标物体的三维信息的获取，进而造成整个结构光编码***的鲁棒性较差的问题。
在保证编码密度足够大、编码窗口尺寸足够小的同时，本申请提供一种结构光编码方法、装置及终端设备，从而可以提高结构光编码***的鲁棒性。本申请的主旨思想在于：通过N种黑白二值几何图形作为编码元素的空间编码方案，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，可用于目标物体表面更为复杂的动态三维重建。
本申请适用于如下结构光编码***，图1为本申请一实施例提供的结构光编码***的示意图，如图1所示，该结构光编码***包括：智能设备11、投影设备12、摄影设备13和目标物体14。该智能设备11可以是计算机、笔记本电脑、平板、手机等智能设备。该结构光编码***的工作原理是：智能设备11生成第一编码图案，将其传输给投影设备12，投影设备12将第一编码图案投影至目标物体14，摄影设备13拍摄投影后的目标物体，获取第二编码图案。摄影设备13可以将第二编码图案传输给智能设备11，智能设备11确定第二编码图案的特征点；并对每个特征点进行解码，确定每个特征点在第一编码图案中的对应位置；根据每个特征点在第一编码图案中的对应位置确定每个特征点在第二编码图案中的深度；根据每个特征点在第二编码图案中的深度，确定目标物体的三维轮廓信息。
其中该结构光编码***中的智能设备、投影设备、摄影设备可以集成于终端设备中，即终端设备可以实现上述结构光编码***的各个功能。智能设备对应于终端设备的处理器，投影设备对应于终端设备的投影单元，摄影设备对应于终端设备的摄影单元。
基于上述结构光编码***，本申请提供一种结构光编码方法、装置及终端设备。以提高结构光编码***的鲁棒性。
具体地，图2为本申请一实施例提供的一种结构光编码方法的流程图，如图2所示，该方法包括：
步骤S201：生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数；
步骤S202：生成黑白相间的菱形图案构成的矩形图案；
步骤S203：将黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。
例如，根据现有技术提供的查找伽罗华域大小为8，维度为4的本原多项式：h(x)＝x 4+x+α 3。产生伪随机序列时需要遵循的规则为：a 3+a+1＝0与a 7＝1。
生成的多元的伪随机序列的元素值域为：GF(8)＝{0,1,a,a 2,a 3,a 4,a 5,a 6}。
利用折叠原理将伪随机序列转化为伪随机阵列。该伪随机阵列的编码窗口为2*2、编码元素为8。生成黑白相间的菱形图案构成的矩形图案；将黑白二值几何图形嵌入黑白相间的菱形图案中的白色菱形图案中，作为第一编码图案的前景图案，将黑白相间的菱形图案中的黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。图3为本申请提供的第一编码图案的示意图，图4为本申请提供的第一编码图案的局部放大图，如图3和图4所示，该第一编码图案的编码窗口为2*2，编码元素为8，编码容量为65×63。
在第一编码图案中特征点被定义为相邻菱形元素的交点，所以第一编码图案中包括两种类型的特征点，一种是如图3所示的P1点，该P1特征点的上下是背景元素；另一种是如图3所示的P2点，该P2特征点的左右是背景元素。对于每个特征点的码字的定义为：若为P1型特征点，码字由其左右的元素以及左右元素的下方的两个元素组成；若为P2型特征点，码字由其上下的元素以及上下元素的右边的元素组成。因此，如图3所示，P1特征点和P2特征点具有相同的码字，即为c1-c2-c3-c4。本申请中，2*2的编码窗口包括两种类型的特征点。
需要说明的是，本申请上述黑白相间的菱形图案还可以是黑白相间的方形图案，或者是对上述菱形图案进行拉伸、旋转后的图案。本申请对此不做限制。
综上，本申请提供的结构光编码方法所生成的第一编码图案，其编码窗口为2*2，该编码窗口已达到最小编码窗口。而且本申请可以保证编码密度(2*2的编码窗口包括两种类型的特征点)足够大。进一步地，由于本申请通过N种黑白二值几何图形实现编码，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，可用于目标物体表面更为复杂的动态三维重建。从而提高结构光编码***的鲁棒性。
进一步地，该方法还包括：将第一编码图案投影至目标物体；拍摄投影后的目标物体，获取第二编码图案；确定第二编码图案的特征点，其中该特征点用于确定目标物体的三维轮廓信息。
其中，该第二编码图案的特征点与第一编码图案的特征点一一对应。而确定第二编码图案的特征点的方法，包括：将第二编码图案划分为多个第一区域；在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定特征点。其中第一区域一般可以选取上述黑色菱形图案或者白色菱形图案尺寸的2/3。
具体地，将第二编码图案转化为灰度图像，利用滤波器对该灰度图像进行平滑处理；然后设计一个X型模板，对经过平滑处理后的图像进行卷积运算，利用选取局部非最大值抑制法来确定候选特征点；其中选取局部非最大值抑制法来确定候选特征点，具体包括：通过公式：
Figure PCTCN2018087373-appb-000010
计算每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；L表示X型模板的尺寸，在每个第一区域中确定所述数值的极大值，并将极大值对应的像素点作为候选特征点。
如图4所示，在第一编码图案真正的特征点C处，水平方向上的点和垂直方向上的点的灰度值的差值在局部区域内是最大的。同样的，第二编码图案真正的特征点也应该具有该特征。即用X型模板计算出的数值H在第一区域内是最大的，利用这个方法可以排除掉绝大部分的普通点。
可选地，在所有候选特征点中确定特征点，包括：确定任一候选特征点所在的第二区域，并确定第二区域旋转180度后的对称区域。
通过公式
Figure PCTCN2018087373-appb-000011
计算任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示所述像素点(m,n)在对称区域中的对称点的灰度值；
Figure PCTCN2018087373-appb-000012
表示第二区域中，所有像素点的灰度值的平均值；确定对称系数小于预设阈值的候选特征点为特征点。其中任一候选特征点所在的第二区域一般是以该候选特征点为中心的一个圆形区域。
由于第一编码图案中的特征点具有旋转不变对称性(旋转180度)，因此第二编码图案中的特征点基本也遵循这个特征。对称系数δ表示第二编码图案的局部对称程度，该对称系数δ越小，表示第二编码图案的局部对称程度越好。基于此，采用上述公式计算每个候选特征点的对称系数δ，并通过设定一个预设阈值，以此剔除候选特征点中的错误特征 点，获取最终的特征点。
本申请中，通过上述两个公式可以准确的确定第二编码图案上的特征点，进而保证能够准确的获取目标物体的三维信息。
图5为本申请一实施例提供的一种结构光编码装置的结构示意图，如图5所示，该装置包括：
第一生成模块501，用于生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数。第二生成模块502，用于生成黑白相间的菱形图案构成的矩形图案。第三生成模块503，用于将黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为第一编码图案的背景图案，以生成第一编码图案。
可选地，还包括：投影模块504，用于将第一编码图案投影至目标物体；获取模块505，用于拍摄投影后的目标物体，获取第二编码图案；确定模块506，用于确定第二编码图案的特征点，其中特征点用于确定目标物体的三维轮廓信息。
具体地，第一生成模块501、第二生成模块502和第三生成模块503的生成功能可以是实时生成功能。例如：在每次结构光编码过程中，第一生成模块501都需要生成一次伪随机阵列；第二生成模块502都需要生成一次黑白相间的菱形图案构成的矩形图案；第三生成模块503都需要生成一次第一编码图案。或者，第一生成模块501、第二生成模块502和第三生成模块503的生成功能也可以是一种提取或者维护功能。例如：第三生成模块503在第一次结构光编码过程中已经生成了第一编码图案，生成的第一编码图案被保存在结构光编码装置的存储模块中。接下来在每次结构光编码过程中，第三生成模块503只需要在存储模块中提取该第一编码图案即可。
进一步地，投影模块504可以利用投影光将第一编码图案投影至目标物体。相应的，获取模块505具有投影光采集能力，以拍摄投影后的目标物体，获取第二编码图案。例如：该投影光可以是发光二极管(Light Emitting Diode，LED)或者红外激光光源发出的红外光。获取模块505具有红外光采集能力，以拍摄投影后的目标物体，获取第二编码图案。
可选地，确定模块506具体用于：将第二编码图案划分为多个第一区域；在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定所述特征点。
可选地，确定模块506具体用于：
通过公式
Figure PCTCN2018087373-appb-000013
计算每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；在每个第一区域中确定数值的极大值，并将极大值对应的像素点作为候选特征点。
可选地，确定模块505具体用于：确定任一候选特征点所在的第二区域，并确定第二区域旋转180度后的对称区域；
通过公式
Figure PCTCN2018087373-appb-000014
计算任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示像素点(m,n)在对称区域中的对称点的灰 度值；
Figure PCTCN2018087373-appb-000015
表示第二区域中，所有像素点的灰度值的平均值；确定对称系数小于预设阈值的候选特征点为特征点。
本申请提供的结构光编码装置所生成的第一编码图案，其编码窗口为2*2，该编码窗口已达到最小编码窗口。而且本申请可以保证编码密度足够大。进一步地，由于本申请通过N种黑白二值几何图形实现编码，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，可用于目标物体表面更为复杂的动态三维重建。从而提高结构光编码***的鲁棒性。
图6为本申请一实施例提供的一种终端设备的结构示意图，如图6所示，该终端设备包括：结构光模板601设置有第一编码图案；其中第一编码图案由编码窗口为2*2、编码元素为N的伪随机阵列和多个黑白相间的菱形图案构成的矩形图案生成；N个编码元素通过N种不同的黑白二值几何图形表示，N为大于或者等于8的正整数；黑白二值几何图形嵌入对应的白色菱形图案中，形成第一编码图案的前景图案，黑色菱形图案形成第一编码图案的背景图案；投影单元602，用于产生投影光；通过产生的投影光经过结构光模板601形成第一编码图案，并将第一编码图案投影至目标物体；摄影单元603，用于拍摄投影后的目标物体，获取第二编码图案；处理器604，用于确定第二编码图案的特征点；并对每个特征点进行解码，确定每个特征点在第一编码图案中的对应位置；根据每个特征点在第一编码图案中的对应位置确定每个特征点在第二编码图案中的深度；根据每个特征点在第二编码图案中的深度，确定目标物体的三维轮廓信息。
具体地，终端设备已经生成了第一编码图案，基于此，制作一个结构光模板601，该结构光模板601设置有第一编码图案，当投影单元602产生的投影光经过结构光模板601，将形成第一编码图案，并可以将该第一编码图案投影至目标物体。也就是说，本申请提供的终端设备不用每次都生成第一编码图案，仅需要生成一次第一编码图案，之后通过结构光模板601实现投影即可产生第一编码图案。
可选地，处理器604具体用于：将第二编码图案划分为多个第一区域；在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定所述特征点。
可选地，处理器604具体用于：通过公式
Figure PCTCN2018087373-appb-000016
计算每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；在每个第一区域中确定所述数值的极大值，并将极大值对应的像素点作为候选特征点。
可选地，处理器604具体用于：确定任一候选特征点所在的第二区域，并确定第二区域旋转180度后的对称区域；
通过公式
Figure PCTCN2018087373-appb-000017
计算任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示像素点(m,n)在对称区域中的对称点的灰度值；
Figure PCTCN2018087373-appb-000018
表示第二区域中，所有像素点的灰度值的平均值；确定对称系数小于预设阈值的候选特征点为所述特征点。
本申请提供的终端设备所使用的第一编码图案，其编码窗口为2*2，该编码窗口已达到最小编码窗口。而且本申请可以保证编码密度足够大。进一步地，由于本申请通过N种黑白二值几何图形实现编码，与传统的颜色、亮度编码方法相比，黑白二值几何图形对于目标物体的纹理、颜色及反射特性具有更强的鲁棒性，可用于目标物体表面更为复杂的动态三维重建。从而提高结构光编码***的鲁棒性。

Claims (14)

  1. 一种结构光编码方法，其特征在于，包括：
    生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个所述编码元素通过N种不同的黑白二值几何图形表示，所述N为大于或者等于8的正整数；
    生成黑白相间的菱形图案构成的矩形图案；
    将所述黑白二值几何图形嵌入白色菱形图案中，作为第一编码图案的前景图案，将黑色菱形图案作为所述第一编码图案的背景图案，以生成所述第一编码图案。
  2. 根据权利要求1所述的方法，其特征在于，还包括：
    将所述第一编码图案投影至目标物体；
    拍摄投影后的目标物体，获取第二编码图案；
    确定所述第二编码图案的特征点，其中所述特征点用于确定所述目标物体的三维轮廓信息。
  3. 根据权利要求2所述的方法，其特征在于，所述确定所述第二编码图案的特征点，包括：
    将所述第二编码图案划分为多个第一区域；
    在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定所述特征点。
  4. 根据权利要求3所述的方法，其特征在于，所述在每个第一区域中确定至少一个候选特征点，包括：
    通过公式
    Figure PCTCN2018087373-appb-100001
    计算所述每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；
    在所述每个第一区域中确定所述数值的极大值，并将所述极大值对应的像素点作为所述候选特征点。
  5. 根据权利要求3或4所述的方法，其特征在于，所述在所有候选特征点中确定所述特征点，包括：
    确定任一候选特征点所在的第二区域，并确定所述第二区域旋转180度后的对称区域；
    通过公式
    Figure PCTCN2018087373-appb-100002
    计算所述任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示所述像素点(m,n)在所述对称区域中的对称点的灰度值；
    Figure PCTCN2018087373-appb-100003
    表示所述第二区域中，所有像素点的灰度值的平均值；
    确定所述对称系数小于预设阈值的候选特征点为所述特征点。
  6. 一种结构光编码装置，其特征在于，包括：
    第一生成模块，用于生成编码窗口为2*2、编码元素为N的伪随机阵列；其中N个所述编码元素通过N种不同的黑白二值几何图形表示，所述N为大于或者等于8的正整数；
    第二生成模块，用于生成黑白相间的菱形图案构成的矩形图案；
    第三生成模块，用于将所述黑白二值几何图形嵌入白色菱形图案中，作为第一编码图 案的前景图案，将黑色菱形图案作为所述第一编码图案的背景图案，以生成所述第一编码图案。
  7. 根据权利要求6所述的装置，其特征在于，还包括：
    投影模块，用于将所述第一编码图案投影至目标物体；
    获取模块，用于拍摄投影后的目标物体，获取第二编码图案；
    确定模块，用于确定所述第二编码图案的特征点，其中所述特征点用于确定所述目标物体的三维轮廓信息。
  8. 根据权利要求7所述的装置，其特征在于，所述确定模块具体用于：
    将所述第二编码图案划分为多个第一区域；
    在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定所述特征点。
  9. 根据权利要求8所述的装置，其特征在于，所述确定模块具体用于：
    通过公式
    Figure PCTCN2018087373-appb-100004
    计算所述每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；
    在所述每个第一区域中确定所述数值的极大值，并将所述极大值对应的像素点作为所述候选特征点。
  10. 根据权利要求8或9所述的装置，其特征在于，所述确定模块具体用于：
    确定任一候选特征点所在的第二区域，并确定所述第二区域旋转180度后的对称区域；
    通过公式
    Figure PCTCN2018087373-appb-100005
    计算所述任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示所述像素点(m,n)在所述对称区域中的对称点的灰度值；
    Figure PCTCN2018087373-appb-100006
    表示所述第二区域中，所有像素点的灰度值的平均值；
    确定所述对称系数小于预设阈值的候选特征点为所述特征点。
  11. 一种终端设备，其特征在于，包括：
    结构光模板，设置有第一编码图案；其中所述第一编码图案由编码窗口为2*2、编码元素为N的伪随机阵列和多个黑白相间的菱形图案构成的矩形图案生成；N个所述编码元素通过N种不同的黑白二值几何图形表示，所述N为大于或者等于8的正整数；所述黑白二值几何图形嵌入对应的白色菱形图案中，形成所述第一编码图案的前景图案，黑色菱形图案形成所述第一编码图案的背景图案；
    投影单元，用于产生投影光；通过产生的投影光经过所述结构光模板形成所述第一编码图案，并将所述第一编码图案投影至目标物体；
    摄影单元，用于拍摄投影后的目标物体，获取第二编码图案；
    处理器，用于确定所述第二编码图案的特征点；并对每个特征点进行解码，确定所述每个特征点在所述第一编码图案中的对应位置；根据所述每个特征点在所述第一编码图案中的对应位置确定所述每个特征点在所述第二编码图案中的深度；根据所述每个特征点在所述第二编码图案中的深度，确定所述目标物体的三维轮廓信息。
  12. 根据权利要求11所述的终端设备，其特征在于，所述处理器具体用于：
    将所述第二编码图案划分为多个第一区域；
    在每个第一区域中确定至少一个候选特征点，并在所有候选特征点中确定所述特征点。
  13. 根据权利要求12所述的终端设备，其特征在于，所述处理器具体用于：
    通过公式
    Figure PCTCN2018087373-appb-100007
    计算所述每个第一区域中的任一像素点(i,j)，对应的数值H；其中I(i+α,j+β)表示像素点(i+α,j+β)的灰度值；
    在所述每个第一区域中确定所述数值的极大值，并将所述极大值对应的像素点作为所述候选特征点。
  14. 根据权利要求12或13所述的终端设备，其特征在于，所述处理器具体用于：
    确定任一候选特征点所在的第二区域，并确定所述第二区域旋转180度后的对称区域；
    通过公式
    Figure PCTCN2018087373-appb-100008
    计算所述任一候选特征点的对称系数δ；其中I(m,n)表示像素点(m,n)的灰度值；I'(m,n)表示所述像素点(m,n)在所述对称区域中的对称点的灰度值；
    Figure PCTCN2018087373-appb-100009
    表示所述第二区域中，所有像素点的灰度值的平均值；
    确定所述对称系数小于预设阈值的候选特征点为所述特征点。
PCT/CN2018/087373 2017-05-31 2018-05-17 结构光编码方法、装置及终端设备 WO2018219156A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710401331.XA CN108986177A (zh) 2017-05-31 2017-05-31 结构光编码方法、装置及终端设备
CN201710401331.X 2017-05-31

Publications (1)

Publication Number Publication Date
WO2018219156A1 true WO2018219156A1 (zh) 2018-12-06

Family

ID=64454289

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/087373 WO2018219156A1 (zh) 2017-05-31 2018-05-17 结构光编码方法、装置及终端设备

Country Status (2)

Country Link
CN (1) CN108986177A (zh)
WO (1) WO2018219156A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109993826B (zh) * 2019-03-26 2023-06-13 中国科学院深圳先进技术研究院 一种结构光三维图像重建方法、设备及***
CN110557622B (zh) * 2019-09-03 2021-04-02 歌尔光学科技有限公司 基于结构光的深度信息获取方法及装置、设备及介质
JP7298025B2 (ja) * 2019-10-24 2023-06-26 先臨三維科技股▲ふん▼有限公司 三次元スキャナー及び三次元走査方法
CN110926370B (zh) * 2019-11-12 2021-10-29 季华实验室 测量方法和***
CN113405461B (zh) * 2021-04-23 2023-03-21 封泽希 用于深度检测的结构光编码、解码方法及编码、解码装置
CN113188453B (zh) * 2021-04-30 2022-11-01 东北电力大学 一种用于膜结构非接触位移、应变测量的散斑产生装置
CN115082621B (zh) * 2022-06-21 2023-01-31 中国科学院半导体研究所 一种三维成像方法、装置、***、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140380115A1 (en) * 2013-06-24 2014-12-25 David D. Bar-On Error detecting and correcting structured light patterns
CN104457607A (zh) * 2014-12-10 2015-03-25 东北大学 一种面向深度测量的沙漏码结构光的编解码方法
CN106225719A (zh) * 2016-08-04 2016-12-14 西安交通大学 一种符号阵列结构光编码图案的生成方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102002165B1 (ko) * 2011-09-28 2019-07-25 포토내이션 리미티드 라이트 필드 이미지 파일의 인코딩 및 디코딩을 위한 시스템 및 방법
CN103983213B (zh) * 2014-05-30 2016-12-07 深圳先进技术研究院 一种结构光编码方法及相关装置
CN104197861B (zh) * 2014-08-25 2017-03-01 深圳大学 基于结构光灰度向量的三维数字成像方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140380115A1 (en) * 2013-06-24 2014-12-25 David D. Bar-On Error detecting and correcting structured light patterns
CN104457607A (zh) * 2014-12-10 2015-03-25 东北大学 一种面向深度测量的沙漏码结构光的编解码方法
CN106225719A (zh) * 2016-08-04 2016-12-14 西安交通大学 一种符号阵列结构光编码图案的生成方法及装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HAI BO ET AL.: "A Twofold Symmetry based Approach for the Feature Detection of Pseudorandom Color Pattern", PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND TECHNOLOGY, 31 December 2014 (2014-12-31), pages 623 - 626, XP032657295 *
JIANG HUALIE ET AL.: "A Robust Feature Detection Algorithm for the Binary Encoded Single-Shot Structured Light System", PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON INFORMATION AND AUTOMATION, 31 August 2016 (2016-08-31), pages 264 - 269, XP055551383 *

Also Published As

Publication number Publication date
CN108986177A (zh) 2018-12-11

Similar Documents

Publication Publication Date Title
WO2018219156A1 (zh) 结构光编码方法、装置及终端设备
CN107607040B (zh) 一种适用于强反射表面的三维扫描测量装置及方法
US8805057B2 (en) Method and system for generating structured light with spatio-temporal patterns for 3D scene reconstruction
US7103212B2 (en) Acquisition of three-dimensional images by an active stereo technique using locally unique patterns
JP6513887B2 (ja) 構造化光深度マップのアウトライアの検出および補正のための方法および装置
CN104197861B (zh) 基于结构光灰度向量的三维数字成像方法
Lei et al. Shape from polarization for complex scenes in the wild
US20040151365A1 (en) Multiframe correspondence estimation
CN105890546A (zh) 基于正交格雷码和线移相结合的结构光三维测量方法
JP5633058B1 (ja) ３次元計測装置及び３次元計測方法
TW201421419A (zh) 用於深度假影之消除之影像處理方法及設備
WO2022126870A1 (en) Three-dimensional imaging method and method based on light field camera and three-dimensional imaging measuring production line
CN108895979B (zh) 线段编码的结构光深度获取方法
CN110044927B (zh) 一种空间编码光场对曲面玻璃表面缺陷的检测方法
CN112991517B (zh) 一种纹理影像编解码自动匹配的三维重建方法
CN104200456A (zh) 一种用于线结构光三维测量的解码方法
KR20200046789A (ko) 이동하는 물체의 3차원 데이터를 생성하는 방법 및 장치
CN111336949B (zh) 一种空间编码结构光三维扫描方法与***
Kim et al. Antipodal gray codes for structured light
Cheng et al. 3D object scanning system by coded structured light
TWI480507B (zh) 三維模型重建方法及其系統
KR20190103833A (ko) 실시간 3차원 데이터 측정 방법
CN111307069A (zh) 一种密集平行线结构光三维扫描方法与***
US20200286248A1 (en) Structured light subpixel accuracy isp pipeline/fast super resolution structured light
WO2019113912A1 (zh) 基于结构光的三维图像重建方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18809936

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18809936

Country of ref document: EP

Kind code of ref document: A1