TWI834493B - Three-dimensional reconstruction system and method based on multiple coding patterns - Google Patents
Three-dimensional reconstruction system and method based on multiple coding patterns Download PDFInfo
- Publication number
- TWI834493B TWI834493B TW112104406A TW112104406A TWI834493B TW I834493 B TWI834493 B TW I834493B TW 112104406 A TW112104406 A TW 112104406A TW 112104406 A TW112104406 A TW 112104406A TW I834493 B TWI834493 B TW I834493B
- Authority
- TW
- Taiwan
- Prior art keywords
- pixel
- sets
- decoding
- candidate
- decoded
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001914 filtration Methods 0.000 claims description 14
- 238000013519 translation Methods 0.000 claims description 2
- 238000012216 screening Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
本發明係關於結構光、立體視覺及光學檢測，特別是一種基於多組編碼圖案的三維重建系統及方法。 The present invention relates to structured light, stereo vision and optical detection, especially a three-dimensional reconstruction system and method based on multiple sets of coding patterns.
現有的結構光三維重建技術是藉由投影機主動投影編碼圖案於目標物體上，接著以相機擷取因物體表面形貌而變形的投影圖案，再藉由相機與投影機的相對位置，以三角量測法計算出物體形貌的三維點雲。 The existing structured light 3D reconstruction technology uses a projector to actively project a coding pattern on a target object, and then uses a camera to capture the projection pattern deformed due to the surface topography of the object. The measurement method calculates a three-dimensional point cloud of the object's shape.
然而，現有的結構光三維重建技術的空間解析度取決於投影機的解析度，當目標物體的細節小於投影機的解析度時，就無法重建出正確的形貌。例如：若目標物體的尺寸過小，且投影機因焦距限制而無法拉近拍攝距離，造成投影至目標物體上的編碼圖案相對目標物體過大，導致採樣不足，從而無法重建目標物體的原始樣貌。例如：為了拍攝完整的大型物體，使相機及投影機工作距離拉遠，讓物體本身的小細節無法被重建。另一方面，如果使用高解析度的投影機，則會增加硬體成本。因此，現有的結構光三維重建技術的精度受限於投影機的解析度以及投影機、相機的校正結果，以致於量測精度難以提升。 However, the spatial resolution of existing structured light 3D reconstruction technology depends on the resolution of the projector. When the details of the target object are smaller than the resolution of the projector, the correct shape cannot be reconstructed. For example, if the size of the target object is too small and the projector cannot close the shooting distance due to focal length limitations, the coding pattern projected onto the target object will be too large relative to the target object, resulting in insufficient sampling and making it impossible to reconstruct the original appearance of the target object. For example: in order to capture a complete large object, the working distance of the camera and projector is extended so that the small details of the object itself cannot be reconstructed. On the other hand, if a high-resolution projector is used, the hardware cost will increase. Therefore, the accuracy of existing structured light 3D reconstruction technology is limited by the resolution of the projector and the correction results of the projector and camera, making it difficult to improve the measurement accuracy.
有鑑於此，本發明提出一種基於多組編碼圖案的三維重建系統及方法，在不增加硬體成本及計算成本的前提下，提高空間解析度及三維重建精度。 In view of this, the present invention proposes a three-dimensional reconstruction system and method based on multiple sets of coding patterns, which can improve spatial resolution and three-dimensional reconstruction accuracy without increasing hardware costs and computing costs.
依據本發明一實施例的一種基於多組編碼圖案的三維重建方法，包括：在多個時間點，以投影機分別投影多組編碼圖案至目標物體，這些編碼圖案係以不同編碼方式產生或依據基本編碼圖案的平移或旋轉而產生；在投影所述多組編碼圖案於目標物體時，以第一相機及第二相機拍攝目標物體以產生多個照片集合；以處理器依據所述多個照片集合進行解碼產生多個解碼圖集合，所述多個解碼圖集合的每一者包括對應第一相機的第一解碼圖及對應第二相機的第二解碼圖；以處理器在所述多個解碼圖集合的每一者取得多個像素對，所述多個像素對的每一者包括位於第一解碼圖的第一像素及位於第二解碼圖的第二像素，第一像素及第二像素具有相同的解碼值；以及以處理器基於所述多個解碼圖集合的每一者的所述多個像素對進行三維重建。 According to an embodiment of the present invention, a three-dimensional reconstruction method based on multiple sets of coding patterns includes: using a projector to project multiple sets of coding patterns to a target object at multiple time points. These coding patterns are generated by different coding methods or based on generated by translation or rotation of the basic coding pattern; when projecting the plurality of sets of coding patterns on the target object, photographing the target object with the first camera and the second camera to generate a plurality of photo sets; and using the processor according to the plurality of photos Decoding the set generates a plurality of decoded picture sets, each of the plurality of decoded picture sets includes a first decoded picture corresponding to the first camera and a second decoded picture corresponding to the second camera; with the processor performing the processing on the plurality of decoded picture sets. Each of the decoded picture sets obtains a plurality of pixel pairs, each of the plurality of pixel pairs includes a first pixel located in the first decoded picture and a second pixel located in the second decoded picture, the first pixel and the second The pixels have the same decoded value; and the processor performs a three-dimensional reconstruction based on the plurality of pixel pairs of each of the plurality of decoded map sets.
依據本發明一實施例的一種基於多組編碼圖案的三維重建方法，包括：在多個時間點，以投影機分別投影多組編碼圖案至目標物體；在所述多個時間點，以第一相機及第二相機拍攝目標物體以產生多個照片集合；以處理器依據所述多個照片集合進行解碼產生多個解碼圖集合，所述多個解碼圖集合的每一者包括對應第一相機的第一解碼圖及對應第二相機的第二解碼圖；以處理器在所述多個解碼圖集合的每一者取得多個像素對，所述多個像素對的每一者包括位於第一解碼圖的第一像素及位 於第二解碼圖的第二像素，第一像素及第二像素具有相同的解碼值；以處理器依據候選解碼圖集合執行一篩選程序，其中候選解碼圖集合為所述多個解碼圖集合的每一者，且篩選程序包括：在候選像素對中，取得第一像素的第一座標及第二像素的第二座標，其中候選像素對是候選解碼圖集合的所述多個像素對的每一者；依據第一座標在非候選解碼圖集合的第一解碼圖中找到第一對應像素，依據第二座標在非候選解碼圖集合的第二解碼圖中找到第二對應像素，其中非候選解碼圖集合包括所述多個解碼圖集合中除了候選解碼圖集合以外的每一者；當第一對應像素及第二對應像素具有不同的解碼值時，累加一錯誤計數；及當錯誤計數超過一閾值時，刪除候選像素對；以及以處理器基於所述多個解碼圖集合的每一者的多個可用像素對進行三維重建，其中所述多個可用像素對包括未被刪除的所述多個像素對。 According to an embodiment of the present invention, a three-dimensional reconstruction method based on multiple sets of coding patterns includes: projecting multiple sets of coding patterns to a target object with a projector at multiple time points; using a first set of coding patterns at the multiple time points. The camera and the second camera capture the target object to generate multiple photo sets; the processor decodes the multiple photo sets to generate multiple decoded image sets, each of the multiple decoded image sets includes a corresponding first camera The first decoded picture and the second decoded picture corresponding to the second camera; using the processor to obtain a plurality of pixel pairs in each of the plurality of decoded picture sets, each of the plurality of pixel pairs includes a The first pixel and bit of a decoded image In the second pixel of the second decoded picture, the first pixel and the second pixel have the same decoding value; the processor performs a filtering process according to the set of candidate decoded pictures, wherein the set of candidate decoded pictures is a set of the plurality of decoded pictures. Each, and the filtering process includes: obtaining a first coordinate of a first pixel and a second coordinate of a second pixel in a candidate pixel pair, wherein the candidate pixel pair is each of the plurality of pixel pairs in the candidate decoding map set. One: find the first corresponding pixel in the first decoding picture of the non-candidate decoding picture set according to the first coordinate, and find the second corresponding pixel in the second decoding picture of the non-candidate decoding picture set according to the second coordinate, where the non-candidate The decoding map set includes each of the plurality of decoding map sets except the candidate decoding map set; when the first corresponding pixel and the second corresponding pixel have different decoding values, an error count is accumulated; and when the error count exceeds When a threshold is reached, the candidate pixel pairs are deleted; and the processor performs three-dimensional reconstruction based on a plurality of available pixel pairs of each of the plurality of decoded map sets, wherein the plurality of available pixel pairs include the undeleted Multiple pixel pairs.
依據本發明一實施例的一種基於多組編碼圖案的三維重建方法，包括：在多個時間點，以投影機分別投影多組編碼圖案至目標物體；在所述多個時間點，以第一相機及第二相機拍攝目標物體以產生多個照片集合；以處理器依據所述多個照片集合進行解碼產生多個解碼圖集合，所述多個解碼圖集合的每一者包括對應第一相機的第一解碼圖及對應第二相機的第二解碼圖；以處理器在所述多個解碼圖集合的每一者取得多個像素對，所述多個像素對的每一者包括位於第一解碼圖的第一像素及位於第二解碼圖的第二像素，第一像素及第二像素具有相同的解碼值；以處理器依據候選解碼圖集合執行一篩選程序，其中候選解碼圖集合為所述多個解碼圖集合的每一者，且篩選程序包括：在候選像素對中，依據第一像素在候選解碼圖集合的第二解碼圖中找到第一極 線，或依據第二像素在候選解碼圖集合的第一解碼圖中找到第二極線；及當第二像素與第一極線之間的距離超過一距離閾值時，或當第一像素與第二極線之間的距離超過距離閾值時，刪除候選像素對；以及以處理器基於所述多個解碼圖集合的每一者的多個可用像素對進行三維重建，其中所述多個可用像素對包括未被刪除的所述多個像素對。 According to an embodiment of the present invention, a three-dimensional reconstruction method based on multiple sets of coding patterns includes: projecting multiple sets of coding patterns to a target object with a projector at multiple time points; using a first set of coding patterns at the multiple time points. The camera and the second camera capture the target object to generate multiple photo sets; the processor decodes the multiple photo sets to generate multiple decoded image sets, each of the multiple decoded image sets includes a corresponding first camera The first decoded picture and the second decoded picture corresponding to the second camera; using the processor to obtain a plurality of pixel pairs in each of the plurality of decoded picture sets, each of the plurality of pixel pairs includes a The first pixel of a decoded picture and the second pixel of the second decoded picture, the first pixel and the second pixel have the same decoding value; the processor executes a screening process based on a set of candidate decoded pictures, wherein the set of candidate decoded pictures is Each of the plurality of decoding picture sets, and the filtering procedure includes: in the candidate pixel pair, finding the first pole in the second decoding picture of the candidate decoding picture set according to the first pixel line, or find the second epipolar line in the first decoding picture of the candidate decoding picture set based on the second pixel; and when the distance between the second pixel and the first epipolar line exceeds a distance threshold, or when the first pixel and When the distance between the second epipolar lines exceeds the distance threshold, the candidate pixel pairs are deleted; and the processor performs three-dimensional reconstruction based on a plurality of available pixel pairs of each of the plurality of decoding map sets, wherein the plurality of available pixel pairs The pixel pairs include the plurality of pixel pairs that have not been deleted.
本發明可增加三維空間中的點雲解析度，同時去除不當的雜訊干擾以改善三維重建的結果。本發明提出的方法不需要昂貴的設備及複雜的合併演算法，但依然可以達成工業上檢測所需要的高精度、高空間解析度及低雜訊的要求。 The present invention can increase the resolution of point clouds in three-dimensional space while removing inappropriate noise interference to improve three-dimensional reconstruction results. The method proposed by the present invention does not require expensive equipment and complex merging algorithms, but can still achieve the high precision, high spatial resolution and low noise requirements required for industrial detection.
以上之關於本揭露內容之說明及以下之實施方式之說明係用以示範與解釋本發明之精神與原理，並且提供本發明之專利申請範圍更進一步之解釋。 The above description of the present disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principles of the present invention, and to provide further explanation of the patent application scope of the present invention.
100:基於多組編碼圖案的三維重建系統 100: Three-dimensional reconstruction system based on multiple sets of coding patterns
10:投影機 10:Projector
20:第一相機 20:First camera
30:第二相機 30:Second camera
40:處理器 40: Processor
50:目標物體 50:Target object
52:待重建的部分 52: Part to be reconstructed
S1~S6:步驟 S1~S6: steps
x mn :第一像素(的座標) x mn : first pixel (coordinates)
C l ,C r :相機中心 C l ,C r : camera center
l mn :第一極線 l mn : first polar line
圖1是依據本發明一實施例的基於多組編碼圖案的三維重建系統及其運作示意圖；圖2是依據本發明一實施例的基於多組編碼圖案的三維重建方法的流程圖；圖3是依據本發明一實施例的多組編碼圖案的示意圖；圖4是像素對在多組編碼圖集合的範例；以及圖5是像素與其對應極線的示意圖。 Figure 1 is a schematic diagram of a three-dimensional reconstruction system based on multiple sets of coding patterns and its operation according to an embodiment of the present invention; Figure 2 is a flow chart of a three-dimensional reconstruction method based on multiple sets of coding patterns according to an embodiment of the present invention; Figure 3 is A schematic diagram of multiple sets of coding patterns according to an embodiment of the present invention; FIG. 4 is an example of a pixel pair in a multi-set coding pattern set; and FIG. 5 is a schematic diagram of a pixel and its corresponding epipolar line.
以下在實施方式中詳細敘述本發明之詳細特徵以及優點，其內容足以使任何熟習相關技藝者了解本發明之技術內容並據以實施，且根據本說明書所揭露之內容、申請專利範圍及圖式，任何熟習相關技藝者可輕易地理解本發明相關之目的及優點。以下之實施例係進一步詳細說明本發明之觀點，但非以任何觀點限制本發明之範疇。 The detailed features and advantages of the present invention are described in detail below in the implementation mode. The content is sufficient to enable anyone skilled in the relevant art to understand the technical content of the present invention and implement it according to the content disclosed in this specification, the patent scope and the drawings. , anyone familiar with the relevant art can easily understand the relevant objectives and advantages of the present invention. The following examples further illustrate the aspects of the present invention in detail, but do not limit the scope of the present invention in any way.
圖1是依據本發明一實施例的基於多組編碼圖案的三維重建系統及其運作示意圖。所述系統100包括投影機10、第一相機20、第二相機30以及處理器40。所述系統100基於結構光技術對目標物體50進行三維重建，待重建的部分52是投影機投射至目標物體50的光線、第一相機20的拍攝視角(field of view,FOV)以及第二相機30的拍攝視角等三者重疊的區域。 Figure 1 is a schematic diagram of a three-dimensional reconstruction system based on multiple sets of coding patterns and its operation according to an embodiment of the present invention. The system 100 includes a projector 10 , a first camera 20 , a second camera 30 and a processor 40 . The system 100 performs three-dimensional reconstruction of the target object 50 based on structured light technology. The part 52 to be reconstructed is the light projected by the projector to the target object 50 , the shooting angle of view (field of view, FOV) of the first camera 20 and the second camera 30° shooting angle and other areas where the three overlap.
在一實施例中，處理器40例如為個人電腦、平板電腦、智慧型手機、微控制器、特殊應用積體電路(Application specific integrated circuit,ASIC)或任何可以透過指令執行本發明一實施例的基於多組編碼圖案的三維重建方法，並且控制投影機10及相機20，30的電子裝置。本發明不限制用於實現處理器40的硬體類型。 In one embodiment, the processor 40 is, for example, a personal computer, a tablet computer, a smartphone, a microcontroller, an Application Specific Integrated Circuit (ASIC), or anything that can execute an embodiment of the present invention through instructions. A three-dimensional reconstruction method based on multiple sets of coding patterns, and an electronic device that controls the projector 10 and the cameras 20 and 30 . The present invention is not limited to the type of hardware used to implement processor 40.
圖2是依據本發明一實施例的基於多組編碼圖案的三維重建方法的流程圖，包括步驟S1至步驟S6。所述方法可使用圖1的系統100執行，以下說明每個步驟及其執行裝置。 FIG. 2 is a flowchart of a three-dimensional reconstruction method based on multiple sets of coding patterns according to an embodiment of the present invention, including steps S1 to S6. The method may be performed using the system 100 of FIG. 1 , and each step and its execution device are described below.
步驟S1，處理器40產生多組彼此不同的編碼圖案。 In step S1, the processor 40 generates multiple sets of encoding patterns that are different from each other.
在一實施例中，以多個角度旋轉基本編碼圖案組可滿足產生不同編碼圖案的需求。圖3是依據本發明一實施例的多組(三組)編 碼圖案的示意圖，其中第一列的多個圖案代表基本編碼圖案組，將基本編碼圖案組旋轉10度可產生第二列的編碼圖案組，旋轉20度可產生第三列的編碼圖案組。在一實施例中，基本編碼圖案組中的圖案可採用二元結構光序列，或是格雷碼(Gray code)結構光序列等多種編碼方法。本發明不限制旋轉角度的數值、編碼方法或是編碼圖案組的數量。 In one embodiment, rotating the basic encoding pattern group at multiple angles can satisfy the need to generate different encoding patterns. Figure 3 shows a multi-group (three-group) braid according to an embodiment of the present invention. A schematic diagram of a code pattern, in which multiple patterns in the first column represent a basic coding pattern group. Rotating the basic coding pattern group 10 degrees can produce a coding pattern group in the second column, and rotating it 20 degrees can produce a coding pattern group in the third column. In one embodiment, the patterns in the basic encoding pattern group can adopt multiple encoding methods such as binary structured light sequences or Gray code structured light sequences. The present invention does not limit the value of the rotation angle, the encoding method, or the number of encoding pattern groups.
步驟S2，投影機10在多個時間點分別投影多組編碼圖案至目標物體50，第一相機20和第二相機30拍攝目標物體50以產生多個照片集合。 In step S2, the projector 10 projects multiple sets of encoding patterns to the target object 50 at multiple time points, and the first camera 20 and the second camera 30 photograph the target object 50 to generate multiple photo sets.
在一實施例中，投影機10開始投影第i組編碼圖案組中的所有圖案，其中i=1,2,3,...,M，M代表編碼圖案組的數量。每當投影機10投影一組圖案至目標物體50，處理器40便控制第一相機20拍攝第一組照片，控制第二相機30拍攝第二組照片。依據相同編碼圖案組拍攝的第一組照片及第二組照片合稱為一個照片集合。如果所述多組編碼圖案是採用旋轉方式生成，則旋轉角度與編碼圖案組為一對一對應關係。 In one embodiment, the projector 10 starts to project all patterns in the i-th encoding pattern group, where i=1, 2, 3,...,M, and M represents the number of encoding pattern groups. Whenever the projector 10 projects a set of patterns to the target object 50, the processor 40 controls the first camera 20 to take a first set of photos, and controls the second camera 30 to take a second set of photos. The first group of photos and the second group of photos taken according to the same encoding pattern group are collectively called a photo collection. If the multiple sets of coding patterns are generated by rotation, then the rotation angle and the coding pattern groups have a one-to-one correspondence.
步驟S3，處理器40依據多組照片集合產生多組解碼圖集合。如果在步驟S2中投影機10投影M組編碼圖案，則步驟S3中將產生M×2個解碼圖。依據相同編碼圖案組解碼得到的第一解碼圖和第二解碼圖合稱為一個解碼圖集合。 Step S3: The processor 40 generates multiple sets of decoded image sets based on multiple sets of photo sets. If the projector 10 projects M sets of encoding patterns in step S2, M×2 decoded pictures will be generated in step S3. The first decoded picture and the second decoded picture obtained by decoding according to the same coding pattern group are collectively referred to as a decoded picture set.
在一實施例中，處理器40採用適應性二值化(adaptive binarization)技術對照片進行二值化，並使用中值濾波器剔除隨機雜訊，最後對畫面中每個像素進行解碼以產生解碼圖。每個解碼圖包括多個像素，每個像素具有處理器40從編碼圖案解碼得到的解碼值。解碼圖 的資料結構例如是矩陣形式，矩陣中的每個元素是解碼值，每個元素所在的位置對應照片中的座標。 In one embodiment, the processor 40 uses an adaptive binarization technology to binarize the photo, uses a median filter to remove random noise, and finally decodes each pixel in the picture to generate a decoded image. Figure. Each decoded map includes a plurality of pixels, each pixel having a decoded value decoded by processor 40 from the encoding pattern. decoding graph The data structure is, for example, in matrix form. Each element in the matrix is a decoded value, and the location of each element corresponds to the coordinates in the photo.
步驟S4，處理器40從解碼圖取得多個像素對。一個像素對(pixel pair)具有兩個像素，分別是對應第一相機20的第一解碼圖及第二相機30的第二解碼圖中具有相同解碼值的區域的中心點，這兩個像素分別為彼此的匹配點。在一實施例中，處理器40例如使用查表法，即比對矩陣中相同元素並取得其位置的方式，來實現步驟S4。 In step S4, the processor 40 obtains a plurality of pixel pairs from the decoded image. A pixel pair has two pixels, which are the center points of areas with the same decoding value in the first decoded image of the first camera 20 and the second decoded image of the second camera 30 respectively. The two pixels are respectively as matching points for each other. In one embodiment, the processor 40 uses, for example, a table lookup method, that is, a method of comparing identical elements in the matrix and obtaining their positions, to implement step S4.
步驟S5，處理器40對所述多個像素對執行篩選程序，藉此刪除屬於雜訊的像素對。 In step S5, the processor 40 executes a filtering process on the plurality of pixel pairs, thereby deleting pixel pairs that are noise.
在一實施例中，篩選程序包括第一篩選程序及第二篩選程序中的至少一者，但本發明不限制第一篩選程序及第二篩選程序的執行順序。在其他實施例中，可省略篩選程序，從步驟S4取得多個像素對之後進行步驟S6的三維重建。 In one embodiment, the screening procedure includes at least one of a first screening procedure and a second screening procedure, but the present invention does not limit the execution order of the first screening procedure and the second screening procedure. In other embodiments, the filtering procedure can be omitted, and multiple pixel pairs are obtained from step S4 and then the three-dimensional reconstruction of step S6 is performed.
第一篩選程序的實施細節如下：假設解碼圖集合有M個，則處理器40需要執行M次第一篩選程序。每次執行第一篩選程序時，處理器40選擇一個解碼圖集合作為候選解碼圖集合。假設被選擇的第m個候選解碼圖集合中的解碼圖具有Nm個像素對，則處理器40需要執行Nm次判斷。在每次判斷中，處理器40選擇一個像素對作為候選像素對。在候選像素對中，處理器40取得第一像素的第一座標及第二像素的第二座標。處理器40依據第一(二)座標在非候選解碼圖集合的第一(二)解碼圖中找到第一(二)對應像素，其中非候選解碼圖集合是所有解碼圖集合中除了候選解碼圖集合以外的任一解碼圖集合。當第一對應像素及 第二對應像素具有不同的解碼值時，處理器40累加一錯誤計數。當錯誤計數超過一閾值時，處理器40刪除候選像素對。 The implementation details of the first screening procedure are as follows: assuming there are M decoding picture sets, the processor 40 needs to execute the first screening procedure M times. Each time the first filtering procedure is executed, the processor 40 selects a decoding map set as a candidate decoding map set. Assuming that the selected decoding picture in the m-th candidate decoding picture set has N m pixel pairs, the processor 40 needs to perform N m judgments. In each determination, processor 40 selects a pixel pair as a candidate pixel pair. In the candidate pixel pair, the processor 40 obtains the first coordinate of the first pixel and the second coordinate of the second pixel. The processor 40 finds the first (second) corresponding pixel in the first (second) decoding picture of the non-candidate decoding picture set according to the first (second) coordinates, wherein the non-candidate decoding picture set is all decoding picture sets except the candidate decoding picture. Any set of decoding graphs other than the set. When the first corresponding pixel and the second corresponding pixel have different decoding values, the processor 40 accumulates an error count. When the error count exceeds a threshold, processor 40 deletes candidate pixel pairs.
整體而言，第一篩選程序是透過比對多個解碼圖集合來驗證當前像素對是否為雜訊。理論上，解碼圖中匹配的像素對具有相同的解碼值，代表第一相機中第一像素與第二相機中第二像素在三維空間中具有相同的空間座標，因此，第一相機中第一像素座標與第二相機中第二像素座標的解碼值即使在不同的編碼圖中仍會彼此相同。具體來說，如下方式一所示：
其中
因此，在找出每個解碼圖集合的像素對後，第一篩選程序將候選像素對及其他解碼圖集合代入式一進行檢查。如果式一的相等關係不成立，則標記候選像素對，最後再根據被標記的次數決定候選像素對的異碼率(different code rate,DCR)。異碼率代表候選像素對在M個解碼圖集合有中M’個錯誤的比率，即DCR=M'/M。因此，使用者可以 指定一個閾值T D ，當任一個像素對的DCR超過閾值T D 時，則不以該像素對進行重建，反之則以該像素對進行重建。 Therefore, after finding the pixel pairs of each decoding map set, the first filtering procedure substitutes the candidate pixel pairs and other decoding map sets into Equation 1 for inspection. If the equality relationship of Equation 1 does not hold, the candidate pixel pair is marked, and finally the different code rate (DCR) of the candidate pixel pair is determined based on the number of times it is marked. The different code rate represents the ratio of candidate pixel pairs that have M' errors in M decoding image sets, that is, DCR=M ' /M. Therefore, the user can specify a threshold TD. When the DCR of any pixel pair exceeds the threshold TD , the pixel pair will not be used for reconstruction. Otherwise , the pixel pair will be used for reconstruction.
總結上述，第一篩選程序基於多個解碼圖及計算每個像素對的異碼率，並刪除異碼率超過指定閾值的像素對。 To summarize the above, the first filtering procedure calculates the out-of-code rate of each pixel pair based on multiple decoded images, and deletes pixel pairs whose out-of-code rate exceeds a specified threshold.
圖4是像素對在多個解碼圖集合的範例。如圖4所示，第一座標和第二座標在前兩個解碼圖集合中對應到相同的解碼值(解碼圖中每個區塊代表一種解碼值)。然而，由於解碼圖案旋轉的關係，第一座標在第三個解碼圖集合的第一解碼圖中位於區塊的邊界的交點。這代表第一座標可能被解碼成交點周邊的任一個解碼值，從而相異於第二座標的解碼值。而上述的第一篩選程序則可刪除這種情況的像素對。 Figure 4 is an example of pixel pairs in multiple decoding map sets. As shown in Figure 4, the first coordinate and the second coordinate correspond to the same decoding value in the first two decoding map sets (each block in the decoding map represents a decoding value). However, due to the rotation of the decoding pattern, the first coordinate is located at the intersection of the boundary of the block in the first decoding picture of the third decoding picture set. This means that the first coordinate may be decoded into any decoded value around the intersection point, which is different from the decoded value of the second coordinate. The above-mentioned first filtering procedure can delete pixel pairs in this case.
第二篩選程序的實施細節如下：假設解碼圖集合有M個，則處理器40需要執行M次第二篩選程序。每次執行第二篩選程序時，處理器40選擇一個解碼圖集合作為候選解碼圖集合。假設被選擇的第m個候選解碼圖集合中的解碼圖具有Nm個像素對，則處理器40需要執行Nm次判斷。在每次判斷中，處理器40選擇一個像素對作為候選像素對。在候選像素對中，處理器40依據第一像素在第二解碼圖中找到第一極線，或是依據第二像素在第一解碼圖中找到第二極線。當第一像素與第二極線之間的距離超過一距離閾值時，或當第二像素與第一極線之間的距離超過該距離閾值時，處理器40刪除候選像素對。 The implementation details of the second screening procedure are as follows: assuming that there are M decoding picture sets, the processor 40 needs to execute the second screening procedure M times. Each time the second filtering procedure is executed, the processor 40 selects a decoding map set as a candidate decoding map set. Assuming that the selected decoding picture in the m-th candidate decoding picture set has N m pixel pairs, the processor 40 needs to perform N m judgments. In each determination, processor 40 selects a pixel pair as a candidate pixel pair. In the candidate pixel pair, the processor 40 finds the first epipolar line in the second decoded image based on the first pixel, or finds the second epipolar line in the first decoded image based on the second pixel. When the distance between the first pixel and the second epipolar line exceeds a distance threshold, or when the distance between the second pixel and the first epipolar line exceeds the distance threshold, the processor 40 deletes the candidate pixel pair.
整體而言，第二篩選程序是基於極線幾何(epipolar)的原理來驗證當前像素對是否為雜訊。如圖5所示，第一像素x mn 的匹配點
一組良好的像素對x mn 和
總結上述，第二篩選程序基於極線幾何驗證每個像素對，並刪除匹配點與對應極線距離太遠的像素對。 To summarize the above, a second filtering procedure verifies each pixel pair based on epipolar geometry and removes pixel pairs whose matching points are too far away from the corresponding epipolar line.
步驟S6，處理器40依據多個可用像素對進行三維重建。步驟S4取得的多個像素對經過步驟S5刪除了一部分，剩下的部分稱為可用像素對。雖然，依據不同的編碼圖案會產生不同的解碼圖以及不同的像素對，但是它們具有相同的坐標系。因此，僅需直接將點雲疊合即可。在一實施例中，可使用三角量測法進行點雲重建 In step S6, the processor 40 performs three-dimensional reconstruction based on multiple available pixel pairs. A part of the plurality of pixel pairs obtained in step S4 is deleted in step S5, and the remaining parts are called available pixel pairs. Although different decoding images and different pixel pairs will be generated according to different encoding patterns, they have the same coordinate system. Therefore, it is only necessary to directly overlay the point clouds. In one embodiment, point cloud reconstruction can be performed using triangulation.
綜上所述，本發明提出的基於多組編碼圖案的三維重建系統及方法，使用者可根據需求增加編碼圖案的數量以提升空間解析度，再藉由本發明提出的篩選程序刪除異常的匹配點，從而取得更高的重建精度。本發明提出的方法不需要昂貴的設備及複雜的合併演算法，但依然可以達成工業上檢測所需要的高精度、高空間解析度及低雜訊的要求。本發明適用於數位3D掃描機及數位光學3D列印機，此外也適用於產品檢測，如汽車、工具機加工零組件等三維尺寸量測，以及模具開發。 In summary, with the three-dimensional reconstruction system and method based on multiple sets of coding patterns proposed by the present invention, the user can increase the number of coding patterns according to needs to improve the spatial resolution, and then delete abnormal matching points through the screening program proposed by the present invention. , thereby achieving higher reconstruction accuracy. The method proposed by the present invention does not require expensive equipment and complex merging algorithms, but can still achieve the high precision, high spatial resolution and low noise requirements required for industrial detection. The invention is suitable for digital 3D scanners and digital optical 3D printers. In addition, it is also suitable for product inspection, such as three-dimensional dimensional measurement of automobiles, tool machine processing components, and mold development.
雖然本發明以前述之實施例揭露如上，然其並非用以限定本發明。在不脫離本發明之精神和範圍內，所為之更動與潤飾，均屬本發明之專利保護範圍。關於本發明所界定之保護範圍請參考所附之申請專利範圍。 Although the present invention is disclosed in the foregoing embodiments, they are not intended to limit the present invention. All changes and modifications made without departing from the spirit and scope of the present invention shall fall within the scope of patent protection of the present invention. Regarding the protection scope defined by the present invention, please refer to the attached patent application scope.
S1~S6:步驟 S1~S6: steps
Claims (2)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW112104406A TWI834493B (en) | 2023-02-08 | 2023-02-08 | Three-dimensional reconstruction system and method based on multiple coding patterns |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW112104406A TWI834493B (en) | 2023-02-08 | 2023-02-08 | Three-dimensional reconstruction system and method based on multiple coding patterns |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| TWI834493B true TWI834493B (en) | 2024-03-01 |
Family
ID=91269528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW112104406A TWI834493B (en) | 2023-02-08 | 2023-02-08 | Three-dimensional reconstruction system and method based on multiple coding patterns |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI834493B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697233A (en) * | 2009-10-16 | 2010-04-21 | 长春理工大学 | Structured light-based three-dimensional object surface reconstruction method |
| US20130218531A1 (en) * | 2010-07-12 | 2013-08-22 | 3Shape A/S | 3d modeling of an object using textural features |
| US20170372527A1 (en) * | 2016-06-22 | 2017-12-28 | Aquifi, Inc. | Systems and methods for scanning three-dimensional objects |
| CN110288699A (en) * | 2019-06-26 | 2019-09-27 | 电子科技大学 | A kind of three-dimensional rebuilding method based on structure light |
| US20200088508A1 (en) * | 2018-09-18 | 2020-03-19 | Electronics And Telecommunications Research Institute | Three-dimensional information generating device and method capable of self-calibration |
| TW202022321A (en) * | 2018-12-06 | 2020-06-16 | 宏達國際電子股份有限公司 | 3d image processing method, camera device, and non-transitory computer readable storage medium |
| TW202247106A (en) * | 2021-02-28 | 2022-12-01 | 美商雷亞有限公司 | A computer-implemented method and system of providing a three-dimensional model and related storage medium |
-
2023
- 2023-02-08 TW TW112104406A patent/TWI834493B/en active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697233A (en) * | 2009-10-16 | 2010-04-21 | 长春理工大学 | Structured light-based three-dimensional object surface reconstruction method |
| US20130218531A1 (en) * | 2010-07-12 | 2013-08-22 | 3Shape A/S | 3d modeling of an object using textural features |
| US20170372527A1 (en) * | 2016-06-22 | 2017-12-28 | Aquifi, Inc. | Systems and methods for scanning three-dimensional objects |
| US20200088508A1 (en) * | 2018-09-18 | 2020-03-19 | Electronics And Telecommunications Research Institute | Three-dimensional information generating device and method capable of self-calibration |
| TW202022321A (en) * | 2018-12-06 | 2020-06-16 | 宏達國際電子股份有限公司 | 3d image processing method, camera device, and non-transitory computer readable storage medium |
| CN110288699A (en) * | 2019-06-26 | 2019-09-27 | 电子科技大学 | A kind of three-dimensional rebuilding method based on structure light |
| TW202247106A (en) * | 2021-02-28 | 2022-12-01 | 美商雷亞有限公司 | A computer-implemented method and system of providing a three-dimensional model and related storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021115071A1 (en) | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device | |
| CN110135455B (en) | Image matching method, device and computer readable storage medium | |
| KR101643607B1 (en) | Method and apparatus for generating of image data | |
| US9886759B2 (en) | Method and system for three-dimensional data acquisition | |
| KR101121034B1 (en) | System and method for obtaining camera parameters from multiple images and computer program products thereof | |
| KR101733228B1 (en) | Apparatus for three dimensional scanning with structured light | |
| CN110070598B (en) | Mobile terminal for 3D scanning reconstruction and 3D scanning reconstruction method thereof | |
| CN103942754B (en) | Panoramic picture complementing method and device | |
| CN111080709B (en) | Multispectral stereo camera self-calibration algorithm based on track feature registration | |
| JP6216516B2 (en) | Digital watermark embedding method and digital watermark detection method | |
| CN109118544B (en) | Synthetic aperture imaging method based on perspective transformation | |
| CN110956661A (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| CN110969667A (en) | Multi-spectrum camera external parameter self-correction algorithm based on edge features | |
| Wang et al. | Bad-nerf: Bundle adjusted deblur neural radiance fields | |
| WO2018219156A1 (en) | Structured light coding method and apparatus, and terminal device | |
| GB2536429A (en) | Image noise reduction | |
| CN112862897B (en) | Phase-shift encoding circle-based rapid calibration method for camera in out-of-focus state | |
| GB2536430A (en) | Image noise reduction | |
| CN112243518A (en) | Method and device for acquiring depth map and computer storage medium | |
| CN114820563A (en) | Industrial component size estimation method and system based on multi-view stereo vision | |
| JP2018063693A (en) | Image processing device, image processing method, and program | |
| JP2001148012A (en) | Method and device for searching corresponding point | |
| TWI618394B (en) | Stereo matching apparatus using image property | |
| JP7489253B2 (en) | Depth map generating device and program thereof, and depth map generating system | |
| TWI834493B (en) | Three-dimensional reconstruction system and method based on multiple coding patterns |