CN116124036A

CN116124036A - Three-dimensional scanning system

Info

Publication number: CN116124036A
Application number: CN202310090689.0A
Authority: CN
Inventors: 徐玉华; 刘国帅; 吴禹; 黄泽铗
Original assignee: Orbbec Inc
Current assignee: Orbbec Inc
Priority date: 2023-01-18
Filing date: 2023-01-18
Publication date: 2023-05-16

Abstract

The application provides a three-dimensional scanning system, comprising: the transmitting end is used for transmitting a composite patterned beam to the scanned object, wherein the composite pattern comprises a multi-line pattern and a coding pattern, the multi-line pattern comprises a plurality of lines, and the lines are coded uniquely by the coding pattern; the receiving end is used for collecting the composite patterned light beam reflected by the scanned object and generating a composite image, and the composite image comprises a multi-line image and a coded image; and the processor is used for decoding the multi-line image according to the coded image to identify a plurality of lines, and calculating the depth information of the scanned object based on the line laser scanning principle by utilizing the identified plurality of lines. Compared with the prior art, the three-dimensional scanning system provided by the application is high in precision and low in power consumption.

Description

Three-dimensional scanning system

Technical Field

The application relates to the technical field of three-dimensional imaging, in particular to a three-dimensional scanning system.

Background

The existing three-dimensional scanning system generally adopts DLP to project a preset multi-frame fringe pattern (such as Gray code or phase shift fringe) to the surface of an object at a transmitting end, so that a measurement result with high measurement accuracy is easy to obtain. However, DLP has higher cost and complex structure, and the projection of multi-frame stripes is unfavorable for dynamic three-dimensional measurement. While using techniques based on speckle-structured light (e.g., kinect, realSense D435/D455) to project a single frame speckle pattern onto the object surface, dynamic objects can be measured with limited accuracy by computing the depth of the scanned object point through a matching algorithm.

In order to solve the above-mentioned problems, the prior art proposes a line laser scanning technique using laser lines, that is, a transmitting end of a scanner projects a single or multiple laser lines onto a surface of an object and is collected by a receiving end. Based on the line laser scanning principle, each laser line corresponds to an optical plane equation, a ray can be determined according to the optical center of the receiving end and a certain point on the laser line in the image, and the three-dimensional coordinates of the scanned object point can be determined according to the intersection point of the ray and the optical plane equation. However, when the transmitting end transmits a plurality of laser lines, the receiving end acquires the plurality of laser lines to obtain a multi-line image; thus, a ray from a certain point on any one laser line in the multi-line image may intersect with a plurality of light planes to obtain a plurality of intersection points, so that a light plane equation corresponding to the laser line cannot be uniquely determined.

In order to uniquely determine the plane equation corresponding to the laser line, the prior art proposes to use laser lines of multiple colors, and to implement the encoding of the laser lines by using combinations of different colors of adjacent laser lines in space. In this scheme, a color camera is used to capture the laser line image, and a Bayer (Bayer) filter in the color camera reduces the sensitivity of the camera. For example, when a blue or red laser line is used, of all pixels of the color camera, only 1/4 pixels can sensitively respond, and the remaining 3/4 pixels are hardly responsive; while only half of the pixels in a color camera can sensitively respond when using a green laser line. Furthermore, encoding laser lines with color combinations is not suitable for scanning color rich objects.

Disclosure of Invention

The application provides a three-dimensional scanning system, which aims to solve the problems of low precision and high power consumption when the three-dimensional scanning system scans a scanned object in the related technology.

In order to solve the above technical problem, the present application provides a three-dimensional scanning system, including: the transmitting end is used for transmitting a composite patterned beam to the scanned object, wherein the composite patterned beam comprises a multi-line pattern and a coding pattern, the multi-line pattern comprises a plurality of lines, and the lines are coded uniquely by the coding pattern; the receiving end is used for collecting the composite patterned light beam reflected by the scanned object and generating a composite image, and the composite image comprises a multi-line image and a coded image; and the processor is used for decoding the multi-line image according to the coded image to identify a plurality of lines, and calculating the depth information of the scanned object based on the line laser scanning principle by utilizing the identified plurality of lines.

The beneficial effects of this application are: compared with the prior art, the three-dimensional scanning system combining the coding pattern and the multi-line pattern can acquire high-precision three-dimensional scanning information by using only a few light sources, and on the premise of ensuring the precision, the cost and the power consumption are reduced, mark points do not need to be externally attached to a scanned object, the labor cost is reduced, the scanning speed is increased, and the three-dimensional reconstruction speed is increased.

Drawings

In order to more clearly illustrate the technology of the related art or the technical solutions in the embodiments of the present application, the following description will briefly introduce the drawings that are required to be used in the description of the related technology or the embodiments of the present application, and it is apparent that the drawings in the following description are only some embodiments of the present application, but not all embodiments, and that other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.

Fig. 1 is a schematic structural diagram of a three-dimensional scanning system according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a multi-line pattern in a composite patterned beam according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a coding pattern in a composite patterned beam according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of the encoding principle of a composite patterned beam according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of the encoding principle of a composite patterned beam according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of the encoding principle of another composite patterned beam according to an embodiment of the present disclosure;

FIGS. 7-8 are schematic diagrams illustrating encoding principles of another composite patterned beam according to embodiments of the present disclosure;

FIG. 9 is a schematic diagram of the encoding principle of another composite patterned beam according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of a pattern modulation element according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of a further pattern modulation element according to an embodiment of the present disclosure;

fig. 12 is a schematic structural diagram of a line laser scanning principle according to an embodiment of the present application;

FIG. 13 is a schematic diagram of an optical system according to an embodiment of the disclosure;

FIG. 14 is a schematic view of another optical system according to an embodiment of the present disclosure;

FIG. 15 is a schematic view of another optical system according to an embodiment of the present disclosure;

fig. 16 is a schematic structural diagram of another optical system according to an embodiment of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the present application more apparent and understandable, the present application will be clearly and completely described in the following description with reference to the embodiments of the present application and the corresponding drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. It should be understood that the following description of the embodiments of the present application is provided for illustrative purposes only and is not intended to limit the application, i.e., all other embodiments that may be obtained by one of ordinary skill in the art without making any inventive effort are within the scope of the present application. Furthermore, the technical features referred to in the embodiments of the present application described below may be combined with each other as long as they do not constitute a conflict with each other.

Fig. 1 is a schematic system structure of a three-dimensional scanning system according to the present application. The system comprises a transmitting end 10, a receiving end 11 and a processor (not shown), wherein the transmitting end 10 is used for transmitting a composite patterned beam 12 to a scanned object, the composite patterned beam comprises a multi-line pattern 120 and a coding pattern 121, and the coding pattern 121 uniquely codes a plurality of lines contained in the multi-line pattern; the receiving end 11 is used for collecting the composite patterned light beam 12 reflected by the scanned object and generating a composite image, and transmitting the composite image to the processor, wherein the composite image comprises a multi-line image and a coded image; the processor is used for decoding the multi-line image according to the coded image to identify a plurality of lines, and calculating depth information of the scanned object based on a line laser scanning principle by utilizing the identified plurality of lines.

Further, after obtaining the depth information of the scanned object, the processor may be configured to obtain the inner parameter and the outer parameter of the transmitting end 10 and the receiving end 11 in the three-dimensional scanning system, and combine the inner parameter and the outer parameter with the depth information of the scanned object to obtain a point cloud image of the scanned object; on the other hand, the processor can be further used for carrying out three-dimensional reconstruction on the obtained point cloud image of the scanned object to obtain a three-dimensional model of the scanned object.

In one embodiment, the multi-line pattern 120 includes a plurality of lines, the lines being laser lines, the plurality of laser lines projected by the transmitting end 10 including a vertical direction or a near vertical direction, the plurality of laser lines being parallel to each other, as shown in fig. 2. It should be noted that, the vertical laser line described in this embodiment is defined as the extending direction of the laser line and the base lines between the transmitting end 10 and the receiving end 11 are perpendicular to each other, that is, when the transmitting end 10 transmits the laser line in the horizontal direction, the laser line is perpendicular to the base lines between the transmitting end 10 and the receiving end 11, and also belongs to the laser line in the vertical direction described in this embodiment.

In one embodiment, the encoding pattern 121 is a spot, at least one spot is arranged near or along each laser line, and the plurality of lines are respectively encoded according to the uniqueness of the shape or size of the single spot or the uniqueness of the distribution combination of the plurality of spots in the encoding pattern, so as to realize the uniqueness encoding of each line in the multi-line pattern. It should be noted that, in this embodiment, the encoding pattern may be, besides the spots, a triangle, a quadrilateral, a pentagon, or other polygonal or other regular or irregular two-dimensional patterns, and the encoding pattern on a single line may be a two-dimensional pattern with one shape or a combination of two-dimensional patterns with multiple shapes, as shown in fig. 3, which is not limited herein.

Fig. 4 is a schematic structural diagram of a composite patterned beam emitted by an emitting end according to the present application, taking a line as an example of a laser line, which illustrates a principle of uniquely encoding each laser line in a multi-line pattern by using a distribution position of the encoding pattern. In some embodiments, a plurality of encoding patterns are distributed near or along each laser line in the multi-line pattern, whereby each laser line in the multi-line pattern is uniquely encoded according to the distribution location of the encoding patterns. The distribution positions of the coding patterns can be regular or random, and only the unique coding of each laser line is ensured; the size or shape of the coding pattern is not limited in this embodiment, and may be the same or different. It should be noted that fig. 4 (a) - (d) only illustrate that the coding pattern is distributed along the laser line, and when the coding pattern is arranged near the laser line, as shown in fig. 4 (e), the coding principle is the same as that of the coding pattern distributed along the laser line.

In one embodiment, as shown in FIG. 4, uniquely encoding each laser line of the multi-line pattern with the encoding pattern when the transmitting end projects the composite patterned beam includes: in fig. 4 (a), at least one coding pattern is projected at the position of each laser line (vertical solid line) included in the multi-line pattern, so that all the coding patterns are distributed on the same horizontal line (horizontal dashed line), but such coding patterns cannot realize coding of each line. Thus, on the basis of fig. 4 (a), the projection positions of the coding patterns are slightly shifted up and down, for example, the coding patterns at certain positions are shifted up by a fixed shift amount or the coding patterns at certain positions are shifted down by a fixed shift amount so as to be positioned on different horizontal lines, as shown in fig. 4 (b). Assuming that fig. 4 (a) is a preset reference code pattern, each code pattern has at least 2 distribution states in the vertical direction, i.e., the position of the code pattern is changed and unchanged, compared with the preset reference code pattern, so that each laser line is uniquely coded by using the distribution states of at least 2 code patterns adjacent to each laser line in the horizontal direction. Based on this, a unique encoding of at least 4 laser lines can be achieved.

In fig. 4 (b), each code pattern has three distribution states in the vertical direction, which are respectively denoted as high (H), medium (M), and low (L), and the unique coding of each laser line is implemented by using the distribution states of 3 code patterns adjacent to each laser line in the horizontal direction as an example, the coding of the No. 1 laser line is MLH, the coding of the No. 2 laser line is LHM, the coding of the No. 3 laser line is HMM, and the coding patterns missing for the No. 0 laser line and the last laser line are preferably represented by N (None), for example, the coding of the No. 0 laser line is NML. Therefore, at least 27 codes can be realized according to the coding principle, namely at least 27 unique codes of the laser lines can be realized, and the codes of each laser line are different for at least 3 coding patterns which are in three distribution states and are adjacent to the current laser line in the horizontal direction of each laser line.

It should be noted that, the basic principle of the present embodiment for uniquely encoding each laser line by using the distribution position of the encoding pattern is as follows: when the X axis (horizontal direction) of the transmitting end is parallel to the base lines of the transmitting end and the receiving end, the coding pattern originally on the same horizontal line of the transmitting end can be ensured, and the coding pattern is also on the same horizontal line when the receiving end images, so that the upper and lower positions of the coding pattern in the coding image generated by the receiving end are not changed, and the processor can decode the obtained multi-line image by utilizing the coding image and according to the coding principle so as to identify each laser line, thereby realizing unique coding of each laser line. In addition, the multi-line pattern in fig. 4 is a vertical solid line portion, and a horizontal line, i.e., a dotted line portion, is not present in actual use, and is used only for better description of the coding principle.

In another embodiment, taking 27 laser lines as an example, since the plurality of laser lines projected by the emitting end have a certain length, taking 3 coding patterns arranged on each laser line as an example, in order to realize coding of the full-field laser line, the coding pattern shown in fig. 4 (b) is duplicated in multiple copies along the vertical direction, and taking 3 copies as an example, as shown in fig. 4 (c), coding of the current laser line is realized by using at least 3 adjacent coding patterns in the horizontal direction near each laser line according to the coding principle, and at this time, coding of the number 1 laser line is MLH. Therefore, when the transmitting end projects the composite patterned light beam as shown in fig. 4 (c) and is acquired by the receiving end to generate a composite image comprising a multi-line image and an encoded image and transmitted to the processor, the processor can decode the multi-line image by utilizing the encoded image in the composite image to uniquely determine each laser line in the multi-line pattern as shown in fig. 4 (d) transmitted by the transmitting end, so that each laser line in the multi-line pattern transmitted by the transmitting end and each laser line in the multi-line image acquired by the receiving end form a one-to-one correspondence, and the processor can utilize each laser line with a one-to-one correspondence and perform depth calculation based on a line laser scanning principle to obtain depth information of a scanned object; the process of decoding the multi-line image by using the coding image in the composite image is the inverse process of coding the multi-line image by using the coding pattern.

Taking the distribution position of the coding pattern to carry out unique coding on any laser line as an example to describe the decoding process, preferably, corresponding numbers of each laser line in the multi-line pattern are given in advance, and the corresponding relation (such as a lookup table) between the unique coding of each laser line and the numbers of each laser line is stored in the multi-line pattern, the processor identifies the positions of at least 2 coding patterns adjacent to the current laser line according to the pixel value difference of the neighborhood pixels of the pixel area corresponding to the current laser line in the composite graph, and obtains the unique coding of the current laser line based on the relative positions of the at least 2 coding patterns adjacent to the current laser line and the preset reference coding graph, thereby searching the laser line number corresponding to the coding from the corresponding relation to identify the current laser line, and decoding of each laser line can be realized; the preset reference code pattern is that the code patterns on each laser line are distributed on the same horizontal line, as shown in fig. 4 (a).

That is, for the code pattern distributed near a certain laser line, the code obtained according to the relative position change thereof is MLH compared with the preset reference code pattern, and according to fig. 4 (b), the code corresponds to the laser line No. 1, thereby realizing decoding of the laser line to uniquely determine the current laser line.

In one embodiment, the processor identifies the position of the coding pattern according to the pixel value difference of the neighborhood pixels of the pixel region where the current laser line is located in the composite graph, and obtains the unique coding of the current laser line based on at least three coding pattern positions adjacent to the current laser line, including: when the receiving end collects the composite patterned light beam reflected by the scanned object, partial pixels in the receiving end can respond to the reflected composite patterned light beam to generate a composite graph and transmit the composite graph to the processor, namely, partial pixels in the receiving end do not respond; wherein, the pixel value corresponding to the pixel with response is different from the pixel value corresponding to the pixel without response in value, and the pixel value corresponding to the pixel without response is 0 or a constant value. The pixel value threshold value can be preset, pixel-by-pixel detection is carried out on the composite line image, the pixel value of each pixel in the composite line image is compared with the pixel value threshold value, the pixels which are larger than and/or equal to the pixel value threshold value are defined as the pixels which respond, and the pixels which respond are searched for in the neighborhood pixels of the pixel area corresponding to the current laser line, so that the position of the coding pattern is identified, and the current laser line is decoded according to the position of the coding pattern corresponding to the current laser line, so that the current laser line is uniquely determined.

It should be noted that, in the above embodiments, the encoded image and the multi-line image included in the composite image may be a frame image. In another embodiment, the encoded image and the multi-line image included in the composite image may be two independent frames of images, and at least some pixels between the two frames of images have a one-to-one alignment relationship, so when a certain laser line in the multi-line pattern needs to be decoded, an adjacent encoded pattern of the current laser line can be obtained according to the coordinate information of the current laser line and the alignment relationship between the images to realize decoding.

Fig. 5 is a schematic structural diagram of a coding principle of a composite patterned beam according to an embodiment of the present application. In one embodiment, when the plurality of laser lines projected by the emitting end have a preset length, a plurality of encoding patterns can be distributed in the extending direction of each laser line, and each line is uniquely encoded by using the plurality of encoding patterns distributed in the extending direction. Taking 3 coding patterns arranged on each laser line as an example, the embodiment can also realize the coding of the current laser line by using at least 3 coding patterns in the vertical direction near each laser line, and compared with a preset reference coding chart, the coding of the number 0 laser line is HHH, the coding of the number 1 laser line is HHM, and so on, thereby realizing the unique coding of 27 laser lines. It should be understood that, in general, the angle of view of the transmitting end 10 is greater than the angle of view of the receiving end 11, and thus, the pattern of the composite patterned beam projected by the transmitting end 10 at the front and rear ends is not necessarily collected by the receiving end 11, and in practical application, the codec calculation of the laser lines at the two ends in the multi-line pattern can be omitted.

In summary, the embodiment shown in fig. 4 to 5 only takes arranging 3 coding patterns on each laser line as an example, and when each coding pattern has 3 offset positions in the vertical direction, the coding patterns may also be arranged near the laser line, and the coding principle is the same as that of the above embodiment, and will not be repeated here. In addition, according to the coding principle of the present application, when each laser line has 4 offset positions in the vertical direction, 64 (4 ³ ) Seed coding, which is to uniquely code 64 laser lines; thus, when n coding patterns are arranged on each laser line or near each laser line, m can be obtained when each coding pattern has m offset positions in the vertical direction ⁿ Seed coding, i.e. m ⁿ Unique encoding of the root laser line.

Fig. 6 is a schematic structural diagram of the coding principle of yet another composite patterned beam according to the present application. In one embodiment, the encoding pattern is a two-dimensional pattern of varying shapes, each two-dimensional pattern being arrangeable along each laser line, each laser line of the multi-line pattern being uniquely encoded according to the shape of the two-dimensional pattern. Specifically, when the coding pattern projected by the transmitting end is a two-dimensional pattern with different shapes, the multi-line pattern projected by the transmitting end can be in a horizontal direction or a vertical direction, at this time, the two-dimensional pattern projected by the transmitting end needs to be distributed along each laser line in the multi-line pattern, and as the shapes of the two-dimensional patterns are different, one laser line can be coded through the two-dimensional pattern with one shape, and the corresponding relation between the shape of the two-dimensional pattern and the serial number of each laser line is constructed, so that the unique coding of each laser line in the multi-line pattern by the two-dimensional pattern with different shapes is realized.

Fig. 7 to 8 are schematic structural diagrams of coding principles of another composite patterned beam according to the present application. In one embodiment, as shown in fig. 7, the coding pattern includes a plurality of coding patterns with different sizes, and each coding pattern is respectively arranged along a plurality of laser lines, so as to realize unique coding on each laser line in the multi-line pattern according to the size of each coding pattern. It should be noted that, the present embodiment does not limit the position of the code pattern on the laser line and the direction of the laser line, and it may be located at any position on the laser line. In addition, one or more coding patterns with the same size can be arranged on each laser line in the embodiment, and the sizes of the coding patterns among the laser lines are different.

Specifically, the transmitting end needs to include at least two light sources with different wavelengths, one light source is used for projecting the code pattern, the other light source is used for projecting the multi-line pattern, wherein the code pattern projected by the transmitting end needs to be distributed along each laser line in the multi-line pattern, and as the sizes of the code patterns in the code pattern patterns are different, the code pattern with one size corresponds to one laser line, the corresponding relation between the size of the code pattern and the serial number of each laser line is constructed, and therefore unique coding can be carried out on each laser line in the multi-line pattern according to the code patterns with different sizes.

In another embodiment, the code pattern is an arrangement combination of at least two code patterns with different sizes, as shown in fig. 8, and each code pattern arrangement combination may be respectively arranged along each laser line or near each laser line, so as to code each laser line according to the uniqueness of each code pattern arrangement combination in the multi-line pattern. It should be noted that, the present embodiment does not limit the position of the code pattern on the laser line and the direction of the laser line, and it may be located at any position on the laser line.

Specifically, the transmitting end needs to include at least two light sources with different wavelengths, one light source is used for projecting the coding pattern, and the other light source is used for projecting the multi-line pattern; the coding patterns projected by the transmitting end are arranged along or near each laser line in the multi-line pattern, and the corresponding relation between different arrangement combinations of the sizes of the coding patterns and the serial numbers of the laser lines is constructed because the projected coding patterns have different arrangement combination forms, so that each laser line of the multi-line pattern can be coded according to the uniqueness of the arrangement combinations of the coding patterns. Taking the example that the code pattern projected by the transmitting end comprises 3 code patterns with different sizes, as shown in fig. 8, the code patterns are marked as large (L), medium (M) and small (S), the current laser line is coded by using at least 3 code patterns attached to each laser line, for example, the code pattern of the number 1 laser line is SML, the code pattern of the number 2 laser line is SML, and the like, 27 permutation and combination forms can be provided, so that the unique code of 27 laser lines is realized.

It should be noted that, the principle of coding each laser line by using the arrangement combination of coding patterns with different sizes is similar to that of coding each laser line by using the position offset of the coding patterns, that is, one or a combination of the position, the size and the shape characteristics of the coding patterns can code each laser line in the multi-line image, but only the coding representation forms are different, but the coding is essentially unique to each laser line in the multi-line pattern.

The embodiments illustrated in fig. 2-8 are each illustrated with lines in a multi-line pattern as laser lines. Fig. 9 is a schematic structural diagram of the coding principle of another composite patterned beam according to the present application. In one embodiment, the plurality of lines in the multi-line pattern consists of a coding pattern. Specifically, each line is formed by arranging a plurality of coding pattern sets, each coding pattern set is formed by closely arranging one or a plurality of coding patterns, and preset gaps are reserved among the coding pattern sets; the coding pattern in this embodiment corresponds to the arrangement mode of a plurality of coding pattern sets in each line, and the corresponding relationship between the arrangement mode of the coding pattern set on each line and the line numbers is constructed, so that the line obtained based on the coding pattern set has uniqueness according to the arrangement mode of the coding pattern set on each line. Further, the arrangement mode of the coding pattern sets characterizes the arrangement sequence of the coding pattern sets in each line, and the arrangement mode of the coding pattern sets can be obtained according to the arrangement sequence of each coding pattern set and the number of the coding patterns in each coding pattern set. Taking the example that the transmitting end projects a multi-line pattern in the vertical direction in fig. 9 as an example, for the line No. 0, the multi-line pattern includes 5 code pattern sets, and the arrangement mode is 23523, wherein the sequence of each number indicates the arrangement sequence of each code pattern set, and each number indicates the number of code patterns included in each code pattern set.

The coding pattern used in the present application may be represented by words, or any combination of the above-mentioned numbers, letters, and words, in addition to the above-mentioned numbers, letters, and the like, and is not limited thereto.

Based on the encoding/decoding principles corresponding to fig. 3-9, the present application describes an exemplary system design of a three-dimensional scanning system with reference to fig. 1, 10, and 11.

In some embodiments, as shown in fig. 1, the transmitting end 10 includes a light source 101 and a pattern modulation element 102, wherein the light source 101 is configured to transmit a light beam to the pattern modulation element 102, and the pattern modulation element 102 modulates the light beam transmitted by the light source 101 and projects a composite patterned light beam 12 including a coding pattern 121 and a multi-line pattern 120 toward a scanned object.

In some embodiments, the light source 101 may be a light source such as a Light Emitting Diode (LED), an Edge Emitting Laser (EEL), a Vertical Cavity Surface Emitting Laser (VCSEL), or the like, or may be an array of light sources formed by a plurality of light sources, which emit any one or more of visible light, infrared light, blue light, green light, ultraviolet light, or the like.

In one embodiment, when the light source 101 in the emission end 10 is a single light source, the light source 101 emits a light beam to the pattern modulation element 102, and the pattern modulation element 102 modulates the light beam emitted from the light source 101 and projects the composite patterned light beam 12 toward the scanned object. Preferably, when the light source 101 in the emitting end 10 is a single light source, its corresponding pattern modulation element 102 comprises at least one Mask (Mask) or Diffractive Optical Element (DOE).

Specifically, when the pattern modulation element 102 is a Mask or DOE, the Mask or DOE may be lithographically patterned with a pattern that only allows the multi-line pattern and the code pattern to pass through, so that the light beam emitted by the light source 101 passes through the single Mask or DOE to obtain the composite patterned light beam 12; when the pattern modulation element 102 includes multiple masks or DOEs, each Mask or DOE may be respectively subjected to photolithography to only allow a part of the multi-line pattern or the code pattern to pass through, so that the light beam emitted by the light source 101 passes through each Mask or each DOE to obtain the composite patterned light beam 12. Taking the illustration of the pattern modulation element 102 comprising only one Mask, it comprises: according to the preset input beam and the preset output beam, a pattern meeting the preset requirements is designed, and the pattern is manufactured on a Mask plate through a photolithography technology to form a pattern modulation element Mask required by the embodiment, as shown in fig. 10. It should be noted that, in this embodiment, the pattern meeting the preset requirement only allows the multi-line pattern and the coding pattern to penetrate when the light beam emitted by the light source passes through the pattern modulation element, so as to implement unique coding of each line in the multi-line pattern by using the coding pattern.

In another embodiment, when the light source 101 in the transmitting end 10 includes at least two sub-light sources, the at least two sub-light sources are used to respectively transmit light beams to the pattern modulation element, and the pattern modulation element is used to modulate the light beams transmitted by the at least 2 sub-light sources and project a multi-line pattern and a code pattern to the scanned object; the wavelengths of at least two sub-light sources can be the same or different, each sub-light source can share one pattern modulation element or pattern modulation elements are respectively arranged on the light emitting sides of each sub-light source, and each pattern modulation element comprises a Mask or a DOE.

Specifically, when the wavelengths of the at least two sub-light sources are the same, the light emitting sides of the at least two sub-light sources need to be respectively provided with a pattern modulation element and transmit light beams in a time-sharing manner to obtain a coding pattern and a multi-line pattern, so that the coding pattern and the multi-line pattern are prevented from being overlapped when the receiving end 11 images, and the coding pattern and the multi-line pattern cannot be distinguished; when the wavelengths of the at least two light sources are different, the at least two sub-light sources may share one pattern modulation element or respectively correspond to the pattern modulation elements, which may emit light beams simultaneously or in a time-sharing manner, which is not limited herein.

Further, if pattern modulation elements are respectively disposed on the light emitting sides of the sub-light sources, and the pattern modulation elements are taken as masks, as shown in fig. 11, at least two masks corresponding to the sub-light sources respectively are designed, and one Mask only allows the multi-line pattern to pass through, and the other Mask only allows the coding pattern to pass through. Each sub-light source in the transmitting end respectively transmits light beams to different pattern modulation elements, and the different pattern modulation elements respectively modulate the light beams transmitted by each sub-light source to obtain multi-line patterns and coding patterns and project the multi-line patterns and the coding patterns to a scanned object so as to form a composite patterned light beam 12 containing the multi-line patterns and the coding patterns when reaching the surface of the scanned object.

In some embodiments, the receiving end 11 is configured to receive the light beam reflected back from the scanned object and generate a composite image for transmission to the processor. Specifically, when the transmitting end 10 transmits the coding pattern and the multi-line pattern to the scanned object in a time-sharing manner, the coding pattern and the multi-line pattern are sequentially reflected to the receiving end 11 through the scanned object, the receiving end 11 sequentially collects the coding pattern and the multi-line pattern and correspondingly generates a composite image containing the coding image and the multi-line image, namely, the coding image and the multi-line image at the moment are two frames of images which are mutually independent; when the transmitting end 10 simultaneously transmits the composite patterned beam containing the coding pattern and the multi-line pattern to the scanned object, the composite patterned beam is reflected to the receiving end 11 by the scanned object, and the receiving end 11 simultaneously collects the composite patterned beam and generates a composite image, wherein the composite image comprises a coding image and a multi-line image, i.e. the coding image and the multi-line image are one frame of image.

In one embodiment, the receiving end 11 includes a black-and-white image sensor and a filter, where the black-and-white image sensor is used to collect the composite patterned beam reflected by the scanned object and generate a composite image for transmission to the processor, or is used only to collect the multi-line pattern in the composite patterned beam reflected by the scanned object and generate a multi-line image for transmission to the processor. The black-and-white image sensor can be any one or more of a Charge Coupled Device (CCD), a Complementary Metal Oxide Semiconductor (CMOS), an Avalanche Diode (AD), a Single Photon Avalanche Diode (SPAD) and the like; the optical filter is arranged on the light incident side of the black-and-white image sensor, preferably a narrow-band optical filter matched with the wavelength of the light beam emitted by the emitting end and used for inhibiting the background light noise of other wave bands, so that only the composite patterned light beam or the multi-line pattern in the composite patterned light beam passes through the optical filter to be collected by the black-and-white image sensor.

In another embodiment, the receiving end 11 further includes a color image sensor, where the color image sensor includes an image sensor and a Bayer filter disposed on an incident side of the image sensor, and the Bayer filter is used to collect textures of the scanned object and transmit the textures to the processor, and the processor renders the point cloud module of the scanned object according to the textures of the scanned object to obtain texture point cloud images or performs texture mapping on the three-dimensional model of the scanned object to obtain a texture three-dimensional model.

Further, when the black-and-white image sensor only collects multi-line patterns in the composite patterned beam and generates multi-line images, the light incident side of the color image sensor is further provided with a narrow-band filter matched with the beam wavelength of the emitted coding pattern, so that the color image sensor is only used for collecting the coding pattern in the composite patterned beam and generating a coding image and transmitting the coding image to the processor, and the processor can identify each line in the multi-line images according to the coding image.

It should be understood that, compared with a color image sensor, the black-and-white image sensor has good sensitivity and high signal-to-noise ratio of acquired images, and the black-and-white sensor is used for acquiring the composite patterned beam or the multi-line pattern in the composite patterned beam, which is beneficial to high-precision measurement of the scanned object; in the present embodiment, the color image sensor is only used to collect the encoding pattern used for encoding or the texture of the scanned object, and is not directly used for measuring the scanned object, so that the measurement accuracy is not affected.

In one embodiment, the receiving end 12 may further include an imaging lens for receiving and reflecting the composite patterned beam back from the object, such that the composite patterned beam propagates to and through the optical filter to be imaged on a corresponding pixel of the image sensor. Preferably, the imaging lens includes one lens or a lens group composed of a plurality of lenses.

In some embodiments, the processor may be a separate dedicated circuit, such as a dedicated SOC chip including a CPU, memory, bus, etc., an FPGA chip, an ASIC chip, etc., or may include a general purpose processing circuit, such as when the three-dimensional scanning system is integrated into a smart terminal, such as a cell phone, television, computer, scanner, etc., where the processing circuit in the terminal may be at least a portion of the processor.

In some embodiments, the processor decodes the multi-line image from the encoded image to identify a plurality of lines in the multi-line image, calculates depth information for the scanned object using the identified plurality of lines and based on line laser scanning principles. Specifically, as shown in fig. 12, based on the line laser scanning principle, a line emitted from the emitting end to the scanned object may form a plane of the optical cutter, where a plane of the optical cutter corresponds to a plane equation of the optical cutter, and the plane equation of the optical cutter may be obtained by calibration; the receiving end collects the light beam reflected by the scanned object and forms a multi-line image containing one line on the imaging plane of the camera. The processor extracts any point on any line in the multi-line image, forms a ray with any point (marked as p) of the line from the optical center of the receiving end, and obtains the intersection point of the ray and the optical knife plane to determine the three-dimensional coordinate of p, so as to obtain the depth information of the scanned object.

Further, the processor can also extract the central line on the line in the multi-line image through the central line extraction algorithm, form a ray with any central point of the line from the optical center of the receiving end, calculate the intersection point of the ray and the plane of the optical knife, and then determine the three-dimensional coordinates of the corresponding central point to obtain the depth information of the scanned object. Compared with the method that the depth information of the scanned object is calculated directly through any point on the line, the method can obtain the corresponding center point with sub-pixel coordinates through calculating the center line on the line, and the accuracy of the depth information can be improved through the depth information of the scanned object through the center point of the sub-pixel.

However, if the three-dimensional scanning system only transmits one line to scan the scanned object, the obtained data will be sparse, and in order to obtain dense data, multiple lines need to be transmitted. When the transmitting end transmits a plurality of lines, correspondingly, the receiving end acquires the lines reflected by the measured object to obtain a multi-line image; the ray formed by the optical center of the receiving end from any point on any line in the multi-line image can be intersected with the optical knife plane corresponding to the lines transmitted by the transmitting end to obtain a plurality of intersection points, so that an optical plane equation corresponding to the current line in the multi-line image cannot be uniquely determined.

In order to uniquely determine the plane equation of the optical knife corresponding to the current line in the multi-line image, the embodiment controls the transmitting end to transmit the composite patterned light beam comprising the coding pattern and the multi-line pattern, and uses the coding pattern to uniquely code the multiple lines in the multi-line pattern, so that the receiving end collects the reflected composite patterned light beam and generates a composite image comprising the coding image and the multi-line image, and therefore, the processor can decode the multi-line image by using the coding image in the composite image to identify each line, so as to uniquely determine the plane equation of the optical knife corresponding to the current line, and further obtain the depth information of the scanned object according to the line laser scanning principle.

Fig. 13 to 16 exemplarily show specific optical system architectures of the three-dimensional scanning system based on the system design of the three-dimensional scanning system provided in the embodiment of the present application.

Fig. 13 is a schematic structural diagram of an optical system of a three-dimensional scanning system according to the present application. Specifically, the optical system includes a first transmitting end 20, a second transmitting end 21, and a receiving end 22, where the first transmitting end 20 and the second transmitting end are configured to respectively transmit a composite patterned beam including a multi-line pattern and a coding pattern to a scanned object, and the receiving end 22 is configured to collect the composite patterned beam reflected back by the scanned object and generate a composite image including a coding image and a multi-line image. It should be noted that, in order to better illustrate the optical system provided in the present embodiment, the present embodiment only uses the first transmitting end 20 and the second transmitting end 21 as an example, but the transmitting ends may be integrally or independently disposed in the practical application process, which is not limited herein.

In some embodiments, each emitting end includes a light source and a pattern modulation element, as shown in fig. 13, the first emitting end 20 includes a first light source 200 and a first pattern modulation element 202, and the second emitting end 21 includes a second light source 210 and a second pattern modulation element 212; the first light source 200 and the second light source 210 are configured to emit light beams to the first pattern modulation element 202 and the second pattern modulation element 212, respectively, and the light beams emitted by the first light source 200 are modulated by the first pattern modulation element 202 to form a multi-line pattern and are projected to the scanned object 26, as shown in fig. 13 (a); the light beam emitted from the second light source 210 is modulated by the second pattern modulation element 212 to form a coding pattern and projected onto the scanned object 26, as shown in fig. 13 (b).

Further, the wavelength of the light beams emitted from the first emitting end 20 and the second emitting end 21 is the same or different. In one embodiment, when the wavelengths of the light beams emitted by the emitting ends are different, the first light source 200 and the second light source 210 are preferably a blue laser light source and a green laser light source, which are used for respectively emitting blue light and green light to the corresponding pattern modulation elements, wherein the blue laser light source forms a blue light multi-line pattern after being modulated by the first pattern modulation element 202, and the green laser light source forms a green light coding pattern after being modulated by the second pattern modulation element 212. The multi-line pattern and the coding pattern obtained by utilizing different wavelengths are projected to the scanned object through the embodiment, and can be used for scanning objects which are easy to scatter, such as semitransparent teeth.

In other embodiments, when the wavelengths of the light beams emitted from the emitting ends are the same, the first light source 200 and the second light source 210 are preferably near infrared laser light sources, which are used for respectively emitting or simultaneously emitting the light beams to the pattern modulation element, and the light beams are modulated by the pattern modulation element to form a multi-line pattern and a coding pattern, respectively, for scanning a general object.

Further, when the light beam emitted by the emitting end is a near infrared laser beam, in one embodiment, if the coding pattern in the projected composite patterned light beam is a coding pattern, the emitting end includes at least two light sources and at least one pattern modulation element, and the at least two light sources are preferably VCSEL arrays, where one light source emits the light beam to the pattern modulation element to form a multi-line pattern, and the other light source can directly emit the coding pattern by designing the arrangement manner of the VCSEL arrays, without passing through the pattern modulation element. In another embodiment, if the composite patterned beams are all of the coding pattern, the emitting end may further include at least one light source, preferably a VCSEL array, and the arrangement of the VCSEL array is designed such that the light source can directly emit the coding pattern.

It should be noted that, what wavelength is selected by the emitting end of the optical system depends on the attribute of the scanned object, and as mentioned above, if the scanned object is a semitransparent tooth or other object that is easy to emit and scatter, blue laser, green laser or other laser with shorter wavelength may be used, and if the scanned object is a general object, near infrared laser is generally used, which is not limited in this application.

In one embodiment, the emitting end further includes a collimating element disposed between the light source and the pattern modulating element, for collimating the light beam emitted by the light source to the pattern modulating element, such as a collimating element 201 disposed between the first light source 200 and the first pattern modulating element 202 and a collimating element 211 disposed between the second light source 210 and the second pattern modulating element 212. Preferably, the collimating element 201 comprises a lens or a lens group consisting of a plurality of lenses. The collimating element 201 may be used to collimate the light beam emitted from the light source, and also to homogenize the light beam emitted from the light source.

In some embodiments, the receiving end 22 includes a black-and-white image sensor 220, a color image sensor 221, a light splitting element 222 and an imaging lens 223, where the black-and-white image sensor and the color image sensor are set up on two sides of the light splitting element 222, and are preferably arranged in an "L-shape" to realize sharing of one imaging lens 223. Preferably, a filter is disposed between the black-and-white image sensor 220 and the color image sensor 221 and the light splitting element 222; the composite patterned beam reflected by the scanned object is collimated and focused by the imaging lens 223 to the beam splitter 222, the beam splitter 222 splits the reflected beam to obtain two paths of beams, one path of beam passes through the optical filter and then only leaves the blue light multi-line pattern in the composite patterned beam and is collected by the black-and-white image sensor 220, and the other path of beam passes through the optical filter and then leaves only the green light coding pattern in the composite patterned beam and is collected by the color image sensor. It should be noted that the optical filter in the present embodiment may be a single device, or may be integrated in the light splitting element 222 or the black-and-white image sensor 220 and the color image sensor 221, respectively, which is not limited herein.

In one embodiment, imaging lens 223 is a lens group consisting of a lens or a plurality of lenses for focusing the composite patterned beam reflected back by the scanned object so that the composite patterned beam reflected back by the scanned object is correspondingly imaged onto a corresponding pixel in the image sensor.

In another embodiment, when texture information of the scanned object 26 is to be acquired, the color image sensor 221 in the receiving end 22 is used to acquire the texture information of the scanned object 26, and the black-and-white image sensor 220 is used to acquire the composite patterned beam reflected back by the scanned object. To improve imaging definition of the color image sensor 221, the optical system in this embodiment further includes a coaxial illumination section 23. Preferably, the coaxial illumination portion 23 is disposed on the light incident side of the receiving end 22, and is used for providing an optical signal similar to ambient light for the color image sensor 221; the coaxial illumination section 23 includes an illumination light source 230, a collimating lens 231 and a transflective element 232, the illumination light source 230 emits a light beam to the collimating lens 231, and the light beam is collimated by the collimating lens 231 and then is directed to the transflective element 232, wherein a part of the light beam is deflected to the scanned object 26 by the transflective element 232 to supplement the scanned object 26 with light, so that the color image sensor 221 clearly collects texture information of the scanned object 26. It should be noted that, the illumination light source 230 of the coaxial illumination portion 23 is preferably a white light source, such as an LED.

Specifically, when the composite patterned beam reflected by the scanned object and the beam emitted by the coaxial illumination portion 23 enter the receiving end 22 at the same time, the beam splitting element 222 in the receiving end 22 splits the beam received by the receiving end to obtain two beams, wherein one beam enters the color image sensor 221 to obtain the texture image of the scanned object, the other beam enters the black-and-white image sensor 220, and a narrowband filter is disposed in front of the black-and-white image sensor 220 to filter the ambient light, so as to allow only the composite patterned beam emitted by the emitting end to pass through to form a composite image on the black-and-white image sensor 220. When the color image sensor 221 is used to collect texture information of the scanned object 26, the light-entering side of the color image sensor 221 does not need to be provided with a filter.

In one embodiment, a collimator lens 24 is further disposed on the light incident side of the coaxial illumination portion 23, and the collimator lens 24 is used for collimating the composite patterned beam reflected by the scanned object to the coaxial illumination portion 23 and transmitting the composite patterned beam through the coaxial illumination portion 23. Preferably, the collimator lens 24 may be a single lens or a lens group composed of a plurality of lenses.

In one embodiment, the three-dimensional scanning system further includes a deflection optical element 25 for receiving the light beam emitted by the emitting end and reflecting the light beam to the scanned object 26 to scan the scanned object 26, and for receiving the light beam reflected by the scanned object 26 and deflecting the light beam to the receiving end 22, so as to change the emitting light path and the receiving light path of the three-dimensional scanning system, so that the emitting light path and the receiving light path are partially coaxial, reduce the volume, and realize miniaturization of the three-dimensional scanning system. Preferably, the deflecting optical element 25 may include any one of a turning mirror, a reflecting mirror, a prism, and a MEMS, which is not limited herein.

Fig. 14 is a schematic view of an optical system structure of a three-dimensional scanning system according to still another embodiment of the present disclosure. Compared with the optical system structure shown in fig. 13, the optical system provided in this embodiment is different in the transmitting end, the receiving end is the same as that described above, and the description thereof will not be repeated here.

In one embodiment, the emitting end 30 includes an emitting end including at least a first light source 300, a second light source 301, a beam homogenizing and combining element 302 and a pattern modulation element 303, wherein the first light source 300 and the second light source 301 are respectively disposed on different light incident sides of the beam homogenizing and combining element 302, and the pattern modulation element 303 is disposed on a light emitting side of the beam homogenizing and combining element 302. Specifically, the first light source 300 and the second light source 301 are configured to emit light beams with different wavelengths to the beam homogenizing and combining element 302 simultaneously or in a time-sharing manner, the beam homogenizing and combining element 302 is configured to homogenize the light beams with two different wavelengths and integrate the light beams into the same optical path to propagate to the pattern modulating element 303, and the pattern modulating element 303 is configured to modulate the received light beams to obtain a multi-line pattern and a coding pattern, respectively, and to project the multi-line pattern and the coding pattern to the scanned object 26.

In one embodiment, the beam homogenizing element 302 includes at least two beam homogenizing elements and at least one beam combining element 3022; at least one light homogenizing element is disposed on the light emitting side of at least two light sources with different wavelengths, for example, a first light homogenizing element 3020 corresponding to the first light source 300 and a second light homogenizing element 3021 corresponding to the second light source 301, so that the light beams emitted by the light sources are homogenized and collimated on the energy level and propagated to the beam combining element 3022, thereby improving the light energy utilization rate and avoiding the occurrence of coherent light; the beam combining element 3022 is configured to integrate the light beams emitted by the different light sources collimated by the collimating element onto the same optical path and project the light beams to the pattern modulating element 303. It should be noted that, the light homogenizing element in the present embodiment may be a single lens or a combination of a plurality of lenses, and the lenses may be one or a combination of a fly eye lens, a diffuser, a micro lens, etc.; the beam combining element preferably comprises one or more combinations of dichroic mirrors, prisms, and the like, without limitation.

Further, since the light beams emitted by the light sources with different wavelengths are integrated onto the same optical path by the beam homogenizing and combining element 302 and projected onto the pattern modulation element in this embodiment, the pattern modulation element 303 includes at least one DOE or at least one Mask in one embodiment, for modulating the light beams projected by the beam homogenizing and combining element 302 to generate a multi-line pattern and a code pattern. For example, when the code pattern is a code pattern, to achieve simultaneous generation of the multi-line pattern and the code pattern by only one pattern modulation element, only lines of the light source wavelength corresponding to the transmitted multi-line pattern and only the code pattern transmitting the code pattern may be lithographically transmitted on the same DOE or the same Mask.

In one embodiment, the emitting end further comprises a projection lens 304 for collimating the light beam projected by the pattern modulation element 303 to the scanned object. It should be noted that the projection lens includes a single lens or a combination of a plurality of lenses, which is not limited herein.

Fig. 15 is a schematic view of an optical system structure of another three-dimensional scanning system according to the present application. The optical system includes the transmitting end 40 and the receiving end 22, and it should be noted that the functions of the optical elements with the same names of the transmitting end and the receiving end in this embodiment are the same as those described above, and are not repeated here.

In some embodiments, the emitting end 40 includes at least one light source 400, a collimating and homogenizing part 401, and a pattern modulating element 402, where the collimating and homogenizing part 401 is disposed between the light source 400 and the pattern modulating element 402, the light source 400 emits a light beam to the collimating and homogenizing part 401, and the collimating and homogenizing part 401 is configured to collimate the light beam and then project the collimated light beam to the pattern modulating element 402; the pattern modulation element 402 is lithographically shaped to allow only the multi-line pattern and the code pattern to pass through, and when the collimated and uniform beam is projected onto the pattern modulation element 402, the pattern modulation element 402 can modulate the beam to generate the multi-line pattern and the code pattern to be projected onto the scanned object 26.

In one embodiment, the collimating and homogenizing unit 401 includes a collimating element 4010 and a homogenizing element 4011, the collimating element 4010 is configured to collimate the light beam emitted from the light source 400 until reaching the homogenizing element 4011, and the homogenizing element 4011 is configured to perform a homogenizing process on the light beam collimated by the collimating element 4010 and project the light beam onto the pattern modulating element 402, so that the energy of the light beam emitted from the light source 400 is homogenized, and the light energy utilization rate can be improved when the light beam emitted from the light source is transmitted through the pattern modulating element 402. It should be noted that, the collimating element 4010 is preferably a single lens or a plurality of lens groups, and the light homogenizing element 4011 may be one or a combination of a fly eye lens, a diffuser, a micro lens, and the like. It should be noted that, the collimating element 4010 and the light homogenizing element 4011 may be integrally designed or independently provided, which is not limited in this application.

In one embodiment, the emitting end 40 further comprises a projection lens 403 for collimating the light beam projected by the pattern modulation element 401 to the scanned object 26. The projection lens 403 may include a single lens or a combination of a plurality of lenses, and is not limited thereto.

In one embodiment, the receiving end 22 includes at least one black-and-white sensor 220 and at least one imaging lens 223, wherein the imaging lens 223 is configured to focus the composite patterned beam reflected back by the scanned object, so that the composite patterned beam reflected back by the scanned object is correspondingly imaged on the corresponding pixel of the black-and-white sensor 220; the black and white sensor 220 is configured to receive the multi-line pattern and the encoded pattern reflected back from the scanned object 26 and generate a composite image comprising the multi-line image and the encoded image for transmission to a processor for acquisition of depth information of the scanned object. Further, the receiving end may include two black-and-white sensors to form binocular stereoscopic vision for depth information acquisition of the scanned object, which is not limited herein.

In another embodiment, the receiving end may be the same as the receiving end shown in fig. 13, and will not be described here.

Fig. 16 is a schematic view of an optical system structure of another three-dimensional scanning system according to the present application. The optical system includes the transmitting end 50, the receiving end 22 and the deflecting optical element 25, and it should be noted that the functions and compositions of the optical elements with the same names of the transmitting end in this embodiment are the same as those described above, and are not repeated here.

In one embodiment, the emitting end 50 includes a light source, a collimating and dodging portion, a pattern modulation element, a reflective element, a transflective element, a light combining element, and a collimating element; the first light source 500 and the second light source 501 are configured to emit light beams with different wavelengths to the first collimating and homogenizing part 502 and the second collimating and homogenizing part 503, respectively; the first collimating and homogenizing part 502 collimates the light beam emitted by the first light source 500 to the corresponding first pattern modulation element 504, the second collimating and homogenizing part 503 collimates the light beam emitted by the second light source 501 to the reflecting element 507, and reflects the light beam to the second pattern modulation element 505 through the reflecting element 507; the first pattern modulation element 504 and the second pattern modulation element 505 modulate the light beam to form a multi-line pattern and a coded pattern, respectively, to project to the half mirror element 506. The reflecting element is used for reflecting the light beam after collimation and light homogenization to the pattern modulating element so as to change the emitting light path to realize miniaturization, and the reflecting element can comprise a reflecting mirror, a reflecting prism and the like, so that the effect of reflecting the light beam can be realized, and the reflecting element is not limited herein.

Further, the light beam emitted by the first light source 500 is modulated by the first pattern modulating element 504 to form a multi-line pattern and is transmitted to the light combining element 508 through the half-transmitting and half-reflecting element 506, and the light beam emitted by the second light source 501 is modulated by the second pattern modulating element 505 to form a coding pattern and is reflected to the light combining element 508 through the half-transmitting and half-reflecting element 506; the light combining element 508 is used for combining the multi-line pattern and the coding pattern into a composite patterned beam and projecting the composite patterned beam to the deflection optical element 25 after being collimated by the collimating element 509; the deflecting optical element 25 is used for changing the scanning direction of the composite patterned beam to scan the scanned object 26 by the composite patterned beam, and receiving the composite patterned beam reflected by the scanned object 26 and deflecting the composite patterned beam to the receiving end 22.

In one embodiment, the transmitting end 50 further includes a projection lens 509, and the projection lens 509 is disposed on the light emitting side of the light combining element 508, and the image planes thereof are exactly located on the light emitting surfaces of the first image modulating element 504 and the second image modulating element 504, so that the multi-line pattern and the encoding pattern can be highly aligned when being projected onto the scanned object 26, i.e. the encoding pattern can be exactly distributed and arranged along the multi-line pattern. Preferably, the projection lens 509 is used for collimating the light beam integrated by the light combining element 508 and projecting the light beam onto the deflecting optical element 25.

In one embodiment, the emitting end 50 further comprises an illumination unit 510 for providing ambient-like light to the scanned object 26. Specifically, when the scanned object 25 is in a dark environment (e.g., teeth, gums), the illumination unit 510 can be turned on to supplement the scanned object 25 with light to obtain texture information of the scanned object 26. Preferably, the illumination unit 510 includes an illumination light source and a collimating element, wherein the illumination light source is a white light source (such as an LED) to simulate ambient light, and at this time, the illumination light source emits a light beam to the collimating element and emits the light beam to the deflecting optical element 25 after being collimated by the collimating element, and the light beam is projected to the scanned object 26 under the deflection action of the deflecting optical element 25 to supplement light.

In one embodiment, the receiving end 22 in this embodiment is the same as the receiving end shown in fig. 13, and will not be described here again.

Specifically, the first light source 500 adopts a blue laser light source with a wavelength of 450nm, the second light source 501 adopts a blue laser light source with a wavelength of 405nm, the light beam emitted by the first light source 500 is modulated by the first pattern modulating element 504 to form a multi-line pattern, the light beam emitted by the second light source 501 is modulated by the second pattern modulating element 505 to form a coding pattern, the multi-line pattern and the coding pattern are integrated by the beam combining element 508 to form a composite patterned light beam, the composite patterned light beam is collimated and propagated to the deflection optical element 25 by the projection lens 509, and the deflection optical element 25 is used for changing the propagation direction of the composite patterned light beam so as to complete scanning of the scanned object 26.

Further, the emitting end 50 emits a white light beam and a blue laser beam with different wavelengths to the scanned object 26, the beam emitted from the emitting end 50 to the scanned object 26 is reflected back to the deflecting optical element 25, and the deflecting optical element 25 is used for deflecting the reflected beam back to the receiving end 22. The imaging lens 223 in the receiving end 22 is used for focusing the reflected light beam to the light splitting element 222, the light splitting element 222 splits the light beam received by the receiving end to obtain two paths of light beams, wherein one path of white light beam enters the color image sensor 221 to obtain a texture image of the scanned object, the other path of blue light beam (i.e. light beams with 450nm and 405nm wave bands) enters the black-and-white image sensor 220, a blue light filter is arranged in front of the black-and-white image sensor 220, and only the composite patterned light beam emitted by the emitting end is allowed to pass through so as to image on the black-and-white image sensor 220 to obtain a composite image comprising a coded image and a multi-line image, thereby filtering stray light.

It should be noted that, in the present application, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all referred to each other.

It should also be noted that in the present application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional scanning system, comprising:

the transmitting end is used for transmitting a composite patterned beam to a scanned object, wherein the composite pattern comprises a multi-line pattern and a coding pattern, and the coding pattern is used for uniquely coding a plurality of lines contained in the multi-line pattern;

the receiving end is used for collecting the composite patterned light beam reflected by the scanned object and generating a composite image, and the composite image comprises a multi-line image and a coded image;

and the processor is used for decoding the multi-line image according to the coded image so as to identify the lines, and calculating the depth information of the scanned object by utilizing the identified lines and based on a line laser scanning principle.

2. The three-dimensional scanning system of claim 2, wherein the processor is further configured to obtain internal and external parameters of the transmitting end and the receiving end, and combine depth information of the internal and external parameters and the scanned object to obtain a point cloud image of the scanned object.

3. The three-dimensional scanning system of claim 3, wherein the processor is further configured to reconstruct the point cloud image in three dimensions to obtain a three-dimensional model of the scanned object.

4. A three-dimensional scanning system according to any one of claims 1 to 3, wherein said calculating depth information of said scanned object using said plurality of identified lines and based on line laser scanning principles comprises:

pre-calibrating a plane equation of the optical knife corresponding to each line in the multi-line pattern transmitted by the transmitting end;

identifying each line in the multi-line image to uniquely determine a plane equation of the optical knife corresponding to each line, extracting any point on any line, forming a ray from any point of the currently extracted line starting from the optical center of the receiving end, and calculating the intersection point of the ray and the plane equation of the optical knife corresponding to the current line to obtain the depth information of the scanned object.

5. A three-dimensional scanning system according to any one of claims 1 to 3, wherein the plurality of lines projected by the emission end comprise a vertical direction or a near vertical direction; the vertical line is defined as an extending direction of the line, and the base lines between the transmitting end and the receiving end are perpendicular to each other.

6. The three-dimensional scanning system of claim 5, wherein the encoding pattern uniquely encodes a plurality of lines included in the multi-line pattern comprises: and uniquely encoding each line by using the size or the shape of the encoding pattern.

7. The three-dimensional scanning system of claim 5, wherein each line in the multi-line pattern is formed by arranging a plurality of sets of coding patterns, the coding patterns being equivalent to the arrangement of the plurality of sets of coding patterns in each line; each coding pattern set is formed by closely arranging one or more coding patterns, preset gaps are reserved among the coding pattern sets, and coding pattern sets in different arrangement modes enable lines obtained based on the coding pattern sets in different arrangement modes to have uniqueness.

8. The three-dimensional scanning system of claim 5, wherein at least one of the encoding patterns is distributed near or along each line, wherein the uniquely encoding the plurality of lines included in the multi-line pattern by the line comprises: and carrying out unique coding on each line in the multi-line pattern according to the distribution position of the coding pattern.

9. The three-dimensional scanning system of claim 8, wherein said uniquely encoding each line in the multi-line pattern according to the distribution position of the encoding pattern comprises:

taking a preset reference coding diagram with coding patterns distributed on the same horizontal line as a reference, and performing up-down offset adjustment on the positions of part of coding patterns projected on each line, so that the distribution positions of the adjusted coding patterns have at least 2 distribution states in the vertical direction;

Each line is uniquely encoded with a distribution state of at least 2 encoding patterns adjacent to each line in the horizontal direction.

10. The three-dimensional scanning system of claim 9, wherein said decoding the multi-line image from the encoded image to identify the plurality of lines comprises:

corresponding numbers of all lines in the multi-line pattern are given in advance, and the corresponding relation between the unique codes of the coding pattern to all lines and the numbers of all lines is stored;

identifying the distribution position of the adjacent coding pattern of the current line according to the pixel value difference of the neighborhood pixels of the pixel area corresponding to the current line in the composite graph, and acquiring the unique coding of the current line based on the relative position of the adjacent coding pattern of the current line and the preset reference coding graph;

and searching a line number corresponding to the unique code of the current line from the corresponding relation to identify the lines.

11. The three-dimensional scanning system of claim 8, wherein when the lines have a predetermined length, a plurality of encoding patterns are distributed along the extending direction of the lines, and each line is uniquely encoded using the encoding pattern distribution position along the extending direction of the lines.

12. A three-dimensional scanning system according to any one of claims 1 to 3, wherein the emitting end comprises a light source and a pattern modulation element, wherein the light source is configured to emit a light beam to the pattern modulation element, and the pattern modulation element modulates the light beam emitted by the light source and projects the composite patterned light beam comprising a coding pattern and a multi-line pattern onto the scanned object.

13. The three-dimensional scanning system of claim 12, wherein the emission end further comprises a collimating element disposed between the light source and the pattern modulating element for collimating the light beam emitted by the light source to the pattern modulating element.

14. The three-dimensional scanning system of claim 12, wherein the emitting end further comprises a collimating and homogenizing portion disposed between the light source and the pattern modulation element, for collimating and homogenizing the light beam emitted by the light source and projecting the collimated and homogenized light beam onto the pattern modulation element.

15. The three-dimensional scanning system of claim 14, wherein the collimating and homogenizing portion comprises a collimating element and a homogenizing element, wherein the collimating element is configured to collimate the light beam emitted by the light source to the homogenizing element; the light homogenizing element is used for homogenizing the light beam collimated by the collimating element and projecting the light beam to the pattern modulating element; the collimating element and the light splitting element are integrally designed or independently arranged.

16. The three-dimensional scanning system of claim 15, wherein the light homogenizing element comprises any one or more of a fly-eye lens, a diffuser, or a microlens.

17. The three-dimensional scanning system according to any one of claims 13-16, wherein the emission end further comprises a projection lens for collimating the light beam projected by the pattern modulation element to the scanned object.

18. The three-dimensional scanning system of any of claims 13-16, wherein the pattern modulation element comprises at least one Mask or at least one DOE when the light source is a single light source.

19. The three-dimensional scanning system of claim 18, wherein when said pattern modulation element is a Mask or DOE, said Mask or DOE is lithographically patterned with a pattern that only allows said multi-line pattern and said encoded pattern to pass therethrough, such that a beam emitted by said light source passes through said Mask or said DOE to obtain said composite patterned beam.

20. The three-dimensional scanning system of claim 18, wherein when the pattern modulation element comprises a plurality of masks or DOEs, each Mask or DOE is patterned by photolithography to allow only a multi-line pattern or a code pattern to pass therethrough, so that the light beam emitted by the light source passes through each Mask or each DOE to obtain a composite patterned light beam.

21. The three-dimensional scanning system of any of claims 13-16, wherein when the light source comprises at least two sub-light sources, the at least two sub-light sources are configured to respectively emit light beams to the pattern modulation element, and the pattern modulation element is configured to modulate the light beams emitted by the at least two sub-light sources and project the multi-line pattern and the encoded pattern to the scanned object.

22. The three-dimensional scanning system of claim 21, wherein the wavelengths of the at least two sub-light sources may be the same or different, and each sub-light source may share a pattern modulation element or have a pattern modulation element disposed on the light emitting side of each sub-light source.

23. The three-dimensional scanning system of claim 22, wherein when the wavelengths of the light beams emitted by the at least two sub-light sources are different, the emitting end further comprises a beam homogenizing and combining element disposed between the light source and the pattern modulating element, for homogenizing the light beams emitted by the different wavelengths and integrating into the same optical path to propagate to the pattern modulating element.

24. The three-dimensional scanning system of claim 23, wherein the beam homogenizing and combining element comprises a homogenizing element and a beam combining element, wherein the homogenizing element is used for homogenizing and collimating and transmitting the light beams emitted by the light sources to the beam combining element, and the beam combining element is used for integrating the light beams emitted by different light sources collimated by the collimating element onto the same light path and projecting the light beams to the pattern modulating element.

25. The three-dimensional scanning system of claim 24, wherein the light homogenizing element comprises any one of a fly-eye lens, a diffuser, and a microlens, and the light combining element comprises a dichroic mirror or a prism.

26. The three-dimensional scanning system according to any one of claims 22 to 25, wherein when the at least two sub-light sources have different wavelengths, the at least two sub-light sources are respectively a blue laser light source and a green laser light source, wherein the blue laser light source is used for forming a blue multi-line pattern, and the green laser light source is used for forming a green code pattern.

27. The three-dimensional scanning system of claim 22, wherein when the wavelengths of the light beams emitted by the at least two sub-light sources are different, the emitting end further comprises a collimating and homogenizing portion for collimating and homogenizing the light beams emitted by the light sources, respectively, and transmitting the collimated and homogenized light beams to the pattern modulation element.

28. The three-dimensional scanning system of claim 27, wherein the emitting end further comprises a reflecting element disposed on the light path between the collimating and homogenizing unit and the pattern modulating element, for reflecting the collimated and homogenized light beam to the pattern modulating element to change the emitting light path for miniaturization.

29. The three-dimensional scanning system of claim 28, wherein the transmitting end further comprises a light combining element for receiving the multi-line pattern and the encoded pattern modulated by the pattern modulating element and synthesizing a composite patterned beam for projection onto the scanned object.

30. The three-dimensional scanning system of claim 29, wherein the transmitting end further comprises a semi-transparent and semi-reflective element disposed between the pattern modulation element and the light combining element, for transmitting or reflecting the multi-line pattern or the code pattern obtained by the pattern modulation element to the light combining element.

31. The three-dimensional scanning system according to any one of claims 27 to 30, wherein when the wavelengths of the at least two sub-light sources are different, the at least two sub-light sources are respectively blue laser light sources having different wavelengths.

32. The three-dimensional scanning system according to any one of claims 29 to 30, wherein the emission end further comprises a projection lens, the projection lens is disposed on the light emitting side of the light combining element, and an image plane of the projection lens is exactly located on the light emitting surface of the pattern modulating element, so that the coding pattern is exactly distributed and arranged along the multi-line pattern.

33. The three-dimensional scanning system of claim 22, wherein the at least two sub-light sources are the blue or near-infrared laser light sources when the at least two sub-light sources are the same wavelength.

34. A three-dimensional scanning system according to any one of claims 1-3, wherein said receiving end comprises a black and white image sensor for capturing said composite patterned beam reflected back by said scanned object and generating said composite map or for capturing only a multi-line pattern in said composite patterned beam reflected back by said scanned object and generating said multi-line image.

35. The three-dimensional scanning system of claim 34, wherein the black-and-white image sensor is provided with a filter on the light-entering side for suppressing background light noise such that only the composite patterned beam or the multi-line pattern in the composite patterned beam passes through the filter to be captured by the black-and-white image sensor.

36. The three-dimensional scanning system of claim 34, wherein the receiving end further comprises a color image sensor for acquiring texture of the scanned object.

37. The three-dimensional scanning system of claim 36, wherein when said black and white image sensor is used only to collect multi-line patterns in said composite patterned beam reflected back by said scanned object and to generate multi-line images, said color image sensor is further provided with a filter matched to the wavelength of the beam emitting said encoded pattern such that said color image sensor is used only to collect encoded patterns in said composite patterned beam.

38. The three-dimensional scanning system of claim 37, wherein said receiving end further comprises a light splitting element for splitting said reflected composite patterned beam so that said multi-line pattern and said encoded pattern are acquired by said black-and-white image sensor and said color image sensor, respectively.

39. The three-dimensional scanning system of any one of claims 35-38, wherein the receiving end further comprises an imaging lens for focusing the composite patterned beam reflected back from the scanned object to image on a corresponding pixel in an image sensor.

40. The three-dimensional scanning system of any one of claims 36-38, further comprising an illumination unit comprising an illumination source and a collimating element, wherein the illumination source emits a light beam to the collimating element and the light beam collimated by the collimating element provides ambient-like light to the scanned object.

41. The system of any one of claims 36 to 38, further comprising a coaxial illumination unit disposed on an incident side of the receiving end, for providing the color image sensor with ambient-like light to obtain a clear texture of the scanned object.

42. The three-dimensional scanning system of claim 41, wherein the coaxial illumination section comprises an illumination source, a collimating lens, and a transflective element, wherein the illumination source emits a light beam to the collimating lens and collimated by the collimating lens to the transflective element, and a portion of the light beam is deflected by the transflective element to the scanned object to supplement the scanned object.

43. A three-dimensional scanning system according to any one of claims 1 to 3, further comprising a deflection optical element for receiving the light beam emitted by the emission end and reflecting the light beam to the scanned object to perform scanning of the scanned object, and for receiving the light beam reflected by the scanned object and deflecting the light beam to the receiving end.

44. The three-dimensional scanning system of claim 43, wherein said deflection optical element comprises any one of a turning mirror, a reflecting mirror, a prism, or a MEMS.