TW201837857A - Method for operating a three dimension detection device - Google Patents
Method for operating a three dimension detection device Download PDFInfo
- Publication number
- TW201837857A TW201837857A TW106121733A TW106121733A TW201837857A TW 201837857 A TW201837857 A TW 201837857A TW 106121733 A TW106121733 A TW 106121733A TW 106121733 A TW106121733 A TW 106121733A TW 201837857 A TW201837857 A TW 201837857A
- Authority
- TW
- Taiwan
- Prior art keywords
- dimensional
- image
- corner
- corner point
- detecting device
- Prior art date
Links
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
本發明係有關於一種三維偵測裝置的操作方法，特別是一種能夠利用三維偵測裝置監控三維空間中的物件的方法。The invention relates to a method for operating a three-dimensional detecting device, in particular to a method for monitoring an object in a three-dimensional space by using a three-dimensional detecting device.
影像監控系統目前已被廣泛使用於公共區域的安全管理以及賣場倉儲管理或各式各樣的觀測用途，透過影像監控系統可以達到以少數人力監控複數區域的目的。然而，先前技術的影像監控系統是透過二維(two dimensional, 2D)影像來進行監控，因此很容易受到光影變化、拍攝角度及物件遮蔽而造成誤判。The image monitoring system has been widely used in public area security management and store warehousing management or a variety of observation purposes. Through the image monitoring system, it is possible to monitor multiple areas with a small number of people. However, the prior art image monitoring system monitors through two-dimensional (2D) images, and is therefore susceptible to misjudgment due to changes in light and shadow, shooting angles, and object shading.
第1圖為先前技術之影像監控系統所拍攝的畫面IMG0之示意圖。在畫面IMG0中，人物K站立於偵測線L0的一側而並未越界。然而由於影像監控系統的攝影機架設角度的緣故，在二維的畫面IMG0中，人物K的上半身與下半身卻看似分別位於偵測線L0的兩側。此時影像監控系統常會誤判人物K已越線，而與實際的情況不符。倘若人物K在原地跳躍，則影像監控系統誤判的機率還會更高。因此，隨著監控場所的環境及性質越來越多元，影像監控系統的準確性也受到越來越嚴格的挑戰。Figure 1 is a schematic diagram of a picture IMG0 taken by a prior art image monitoring system. In the picture IMG0, the character K stands on one side of the detection line L0 without crossing the boundary. However, due to the angle of the imaging frame of the image monitoring system, in the two-dimensional image IMG0, the upper body and the lower body of the character K appear to be located on both sides of the detection line L0. At this time, the image monitoring system often misjudges that the character K has crossed the line, but does not match the actual situation. If the character K jumps in place, the probability of misjudging the image monitoring system will be higher. Therefore, as the environment and nature of the monitoring site become more diverse, the accuracy of the image monitoring system is also increasingly rigorously challenged.
本發明之一實施例提供一種三維(three dimension, 3D)偵測裝置的操作方法，三維偵測裝置包含影像擷取單元、深度處理單元及設定輸入單元。An embodiment of the present invention provides a method for operating a three-dimensional (3D) detecting device. The three-dimensional detecting device includes an image capturing unit, a depth processing unit, and a setting input unit.
操作方法包含影像擷取單元擷取第一圖像，深度處理單元計算對應於三維偵測裝置之第一圖像中的物件深度資訊，透過設定輸入單元於第一圖像中界定第一角點及第二角點，其中第一角點及第二角點的連線為參考線段，取得第一角點對應於三維偵測裝置之第一角點攝像三維座標，取得第二角點對應於三維偵測裝置之第二角點攝像三維座標，及根據第一角點攝像三維座標、第二角點攝像三維座標及第一預設高度於第一圖像中標記出以參考線段為底邊之第一偵測區塊。The operation method includes the image capturing unit capturing the first image, the depth processing unit calculating the object depth information in the first image corresponding to the three-dimensional detecting device, and defining the first corner point in the first image through the setting input unit And a second corner point, wherein the first corner point and the second corner point are connected to the reference line segment, and the first corner point corresponds to the first corner point of the three-dimensional detecting device, and the second corner point corresponds to The second corner of the three-dimensional detecting device captures the three-dimensional coordinates, and marks the three-dimensional coordinates, the second corner camera three-dimensional coordinates, and the first preset height according to the first corner point, and marks the reference line segment as the bottom edge in the first image The first detection block.
第2圖為本發明一實施例之三維偵測裝置100的示意圖。三維偵測裝置100包含影像擷取單元110、深度處理單元120及設定輸入單元130。影像擷取單元110可根據三維偵測裝置100所欲監測的環境來設置，以便能夠擷取到所需的影像。第3圖為本發明一實施例之影像擷取單元110所擷取之第一圖像IMG1的示意圖。FIG. 2 is a schematic diagram of a three-dimensional detecting apparatus 100 according to an embodiment of the present invention. The three-dimensional detecting device 100 includes an image capturing unit 110, a depth processing unit 120, and a setting input unit 130. The image capturing unit 110 can be set according to the environment to be monitored by the three-dimensional detecting device 100, so that the desired image can be captured. FIG. 3 is a schematic diagram of a first image IMG1 captured by the image capturing unit 110 according to an embodiment of the invention.
深度處理單元120可以計算影像擷取單元110所擷取之影像中各個物件的深度。也就是說，透過深度處理單元120，三維偵測裝置100就能夠取得第一圖像IMG1中，物件A與三維偵測裝置100的距離深度，並藉此取得物件A在三維空間中的位置資訊。The depth processing unit 120 can calculate the depth of each object in the image captured by the image capturing unit 110. In other words, through the depth processing unit 120, the three-dimensional detecting device 100 can obtain the distance between the object A and the three-dimensional detecting device 100 in the first image IMG1, and thereby obtain the position information of the object A in the three-dimensional space. .
設定輸入單元130可例如但不限於為觸控螢幕或滑鼠，因此可供使用者在第一圖像IMG1中標記出所需的參考點，例如可標記出第一角點P1及第二角點P2，第一角點P1及第二角點P2的連線可作為三維偵測裝置100在監測時的參考線段L1。此外，為了提升監控的精確度，三維偵測裝置100還可根據參考線段L1在三維空間中的位置向上延伸，以形成對應於三維空間的平面偵測區塊，例如第3圖中的第一偵測區塊B1。The setting input unit 130 can be, for example but not limited to, a touch screen or a mouse, so that the user can mark the required reference point in the first image IMG1, for example, the first corner point P1 and the second corner can be marked. The point P2, the line connecting the first corner point P1 and the second corner point P2 can be used as the reference line segment L1 of the three-dimensional detecting device 100 during monitoring. In addition, in order to improve the accuracy of the monitoring, the three-dimensional detecting device 100 may further extend upward according to the position of the reference line segment L1 in the three-dimensional space to form a plane detecting block corresponding to the three-dimensional space, for example, the first in FIG. Block B1 is detected.
由於三維偵測裝置100能夠取得第一圖像IMG1中各個物件的三維空間位置，而當物件變動位置時，三維偵測裝置100也能夠得知物件在三維空間中實際的位置變化，因此三維偵測裝置100可利用對應於相同之三維空間的第一偵測區塊B1來對第一圖像中的物件進行監控。相較於先前技術僅根據二維的資訊進行監控，三維偵測裝置100能夠更加精準地監控物件的狀態。The three-dimensional detecting device 100 can obtain the three-dimensional spatial position of each object in the first image IMG1, and when the object changes position, the three-dimensional detecting device 100 can also know the actual position change of the object in the three-dimensional space, so the three-dimensional detecting The measuring device 100 can monitor the objects in the first image by using the first detecting block B1 corresponding to the same three-dimensional space. Compared with the prior art, only the two-dimensional information is monitored, and the three-dimensional detecting device 100 can more accurately monitor the state of the object.
第4圖為操作三維偵測裝置100的方法200流程圖。方法200包含步驟S210至S280，但不限於第4圖所示的順序。FIG. 4 is a flow chart of a method 200 of operating the three-dimensional detection device 100. The method 200 includes steps S210 to S280, but is not limited to the order shown in FIG.
S210： 影像擷取單元110擷取第一圖像IMG1；S210: The image capturing unit 110 captures the first image IMG1;
S220： 深度處理單元120計算對應於三維偵測裝置100之第一圖像IMG1中的物件深度資訊；S220: The depth processing unit 120 calculates object depth information corresponding to the first image IMG1 of the three-dimensional detecting device 100;
S230： 透過設定輸入單元130於第一圖像IMG1中界定第一角點P1及第二角點P2；S230: The first corner point P1 and the second corner point P2 are defined in the first image IMG1 through the setting input unit 130;
S240： 計算第一角點P1對應於三維偵測裝置100之第一角點攝像三維座標；S240: Calculate the first corner point P1 corresponding to the first corner point of the three-dimensional detecting device 100 to capture three-dimensional coordinates;
S250： 計算第二角點P2對應於三維偵測裝置100之第二角點攝像三維座標；S250: Calculating the second corner point P2 corresponding to the second corner point of the three-dimensional detecting device 100 to capture the three-dimensional coordinates;
S260： 根據第一角點攝像三維座標、第二角點攝像三維座標及第一預設高度於第一圖像IMG1中標記出以參考線段L1為底邊之第一偵測區塊B1；S260: marking the first detection block B1 with the reference line segment L1 as the bottom edge in the first image IMG1 according to the first corner camera three-dimensional coordinate, the second corner camera three-dimensional coordinate and the first preset height;
S270： 辨識第一圖像IMG1中之物件A；及S270: identifying the object A in the first image IMG1; and
S280： 當物件A自第一偵測區塊B1之第一側B11穿越第一偵測區塊B1至第一偵測區塊B1之第二側B12時，發出對應訊號。S280: When the object A traverses the first detection block B1 to the second side B12 of the first detection block B1 from the first side B11 of the first detection block B1, the corresponding signal is sent.
在步驟S210中，影像擷取單元110可擷取第一圖像IMG1，而在步驟S220中，深度處理單元120則會計算對應於三維偵測裝置100之第一圖像IMG1中的物件深度資訊。In step S210, the image capturing unit 110 may capture the first image IMG1, and in step S220, the depth processing unit 120 may calculate the object depth information in the first image IMG1 corresponding to the three-dimensional detecting device 100. .
在第2圖中，影像擷取單元110可包含兩個攝像鏡頭112及114，攝像鏡頭112及114可具有相同的焦距，並可在相異的位置上沿著相同的方向擷取影像。當影像擷取單元110利用第一攝像鏡頭112擷取第一圖像IMG1時，影像擷取單元110還可利用第二攝像鏡頭114擷取第二圖像IMG2。也就是說，影像擷取單元110所擷取的第一圖像IMG1及第二圖像IMG2實質上可視為同一個場景的左眼視覺及右眼視覺。In FIG. 2, the image capturing unit 110 can include two imaging lenses 112 and 114. The imaging lenses 112 and 114 can have the same focal length and can capture images in the same direction at different positions. When the image capturing unit 110 captures the first image IMG1 by using the first imaging lens 112, the image capturing unit 110 may also capture the second image IMG2 by using the second imaging lens 114. That is to say, the first image IMG1 and the second image IMG2 captured by the image capturing unit 110 can be substantially regarded as the left eye vision and the right eye vision of the same scene.
如此一來，深度處理單元120在步驟S220中就可以根據第一圖像IMG1及第二圖像IMG2的影像內容並以雙眼視覺概念為基礎的演算法來推算對應於三維偵測裝置100之第一圖像IMG1中的物件深度資訊。In this way, the depth processing unit 120 can estimate the corresponding to the three-dimensional detecting device 100 according to the image content of the first image IMG1 and the second image IMG2 and based on the binocular vision concept in step S220. Object depth information in the first image IMG1.
舉例來說，深度處理單元120可以根據第一圖像IMG1及第二圖像IMG2中各個畫素周圍的特徵值，例如色彩、邊界數量、變化量(gradient)…等資訊，來判斷第一圖像IMG1及第二圖像IMG2中的相同物件，並根據相同物件在第一圖像IMG1及第二圖像IMG2中的位置、攝像鏡頭112及114的焦距以及攝像鏡頭112及114之間的距離等參數來推算出第一圖像IMG1中各個物件與三維偵測裝置100的距離。For example, the depth processing unit 120 may determine the first map according to information such as color, number of boundaries, gradient, and the like around each pixel in the first image IMG1 and the second image IMG2. The same object in the IMG 1 and the second image IMG 2, and according to the position of the same object in the first image IMG1 and the second image IMG2, the focal length of the imaging lenses 112 and 114, and the distance between the imaging lenses 112 and 114 The parameters are used to derive the distance between each object in the first image IMG1 and the three-dimensional detecting device 100.
也就是說，三維偵測裝置100可以利用兩個攝像鏡頭分別對相同的場景擷取對應的圖像，並利用雙眼視覺的數學模型推算出圖像中各個物件的深度資訊。然而，在本發明的其他實施例中，三維偵測裝置100也可以利用其他的方式來計算圖像中各個物件的深度資訊。That is to say, the three-dimensional detecting device 100 can respectively extract corresponding images for the same scene by using two imaging lenses, and use the mathematical model of binocular vision to derive depth information of each object in the image. However, in other embodiments of the present invention, the three-dimensional detecting apparatus 100 may also use other methods to calculate depth information of each object in the image.
舉例來說，第5圖為本發明另一實施例之三維偵測裝置300的示意圖。三維偵測裝置300包含影像擷取單元310、深度處理單元320、設定輸入單元330及光學傳感器340。影像擷取單元310可單就攝像鏡頭312擷取第一圖像IMG1，而光學傳感器340則可向周圍的物件發射光線並紀錄接收反射光的時間，如此一來，當利用方法200來操作三維偵測裝置300時，深度處理單元320就可以在步驟S220中，利用光學傳感器340根據時差測距(Time of Flight，TOF)的方式來計算對應於三維偵測裝置300之第一圖像IMG1中的物件深度資訊。For example, FIG. 5 is a schematic diagram of a three-dimensional detecting device 300 according to another embodiment of the present invention. The three-dimensional detecting device 300 includes an image capturing unit 310, a depth processing unit 320, a setting input unit 330, and an optical sensor 340. The image capturing unit 310 can capture the first image IMG1 only for the camera lens 312, and the optical sensor 340 can emit light to the surrounding objects and record the time of receiving the reflected light, so that when the method 200 is used to operate the three-dimensional When detecting the device 300, the depth processing unit 320 may use the optical sensor 340 to calculate the first image IMG1 corresponding to the three-dimensional detecting device 300 according to the time of flight (TOF) in step S220. Object depth information.
在步驟S230中，使用者可以透過設定輸入單元130於第一圖像IMG1中界定第一角點P1及第二角點P2，而第一角點P1及第二角點P2的連線可為用來判斷物件狀態的參考線段L1。舉例來說，在第3圖中，使用者若要利用三維偵測裝置100來偵測有無人越過走廊入口，就可以將第一角點P1及第二角點P2分別設置於走廊入口的兩個牆角，而第一角點P1及第二角點P2的連線，亦即參考線段L1則會對應到走廊的入口處。接著，步驟S240至S250會進一步取得第一角點P1及第二角點P2於三維空間中的座標，並向上延伸而在三維空間中形成偵測面，並在第一圖像IMG1中標記出此偵測面，亦即第一偵測區塊B1。In step S230, the user can define the first corner point P1 and the second corner point P2 in the first image IMG1 through the setting input unit 130, and the connection between the first corner point P1 and the second corner point P2 can be The reference line segment L1 used to determine the state of the object. For example, in FIG. 3, if the user wants to use the three-dimensional detecting device 100 to detect that no one has crossed the entrance of the corridor, the first corner point P1 and the second corner point P2 may be respectively set at the entrance of the corridor. A corner of the wall, and the line connecting the first corner point P1 and the second corner point P2, that is, the reference line segment L1, corresponds to the entrance of the corridor. Then, steps S240 to S250 further obtain coordinates of the first corner point P1 and the second corner point P2 in the three-dimensional space, and extend upward to form a detection surface in the three-dimensional space, and mark the first image IMG1. The detection surface, that is, the first detection block B1.
在步驟S240中，三維偵測裝置100可根據第一角點P1於第一圖像IMG1中所對應之牆角的三維空間座標取得第一角點攝像三維座標。在步驟S250中，三維偵測裝置100可根據第二角點P2於第一圖像IMG1中所對應之牆角的三維空間座標取得第二角點攝像三維座標。In step S240, the three-dimensional detecting device 100 can obtain the first corner camera three-dimensional coordinates according to the three-dimensional space coordinates of the corner corresponding to the first image point I1 in the first image IMG1. In step S250, the three-dimensional detecting device 100 can obtain the second corner camera three-dimensional coordinates according to the three-dimensional coordinate of the corner corresponding to the corner in the first image IMG1 according to the second corner point P2.
在本發明的部分實施例中，為了確保三維偵測裝置100能夠準確取得第一角點P1在三維空間中的第一角點攝像三維座標以及第二角點P2在三維空間中的第二角點攝像三維座標，可選擇將第一角點P1及第二角點P2設置在第一圖像IMG1中的物件上，例如走廊的牆角，由於在步驟S220中已取得第一圖像IMG1中各個物件的距離深度資訊及其三維空間座標，因此在步驟S240及S250中，即可根據第一角點P1及第二角點P2所在之物件的三維空間座標來取得第一角點P1及第二角點P2的第一角點攝像三維座標及第二角點攝像三維座標。In some embodiments of the present invention, in order to ensure that the three-dimensional detecting device 100 can accurately obtain the first corner point P1 in the three-dimensional space, the first corner point captures the three-dimensional coordinates and the second corner point P2 in the second angle in the three-dimensional space. Point camera three-dimensional coordinates, the first corner point P1 and the second corner point P2 may be set on the object in the first image IMG1, such as the corner of the corridor, since each of the first image IMG1 has been obtained in step S220 The distance depth information of the object and the three-dimensional space coordinates thereof, so in steps S240 and S250, the first corner point P1 and the second point can be obtained according to the three-dimensional coordinates of the object where the first corner point P1 and the second corner point P2 are located. The first corner of the corner point P2 captures the three-dimensional coordinates and the second corner image captures the three-dimensional coordinates.
接著在步驟S260中，三維偵測裝置100則會在三維空間中，自第一角點攝像三維座標及第二角點攝像三維座標之連線的參考線段L1向上延伸第一預設高度H1，以取得在三維空間中以參考線段L1為底邊的第一偵測區塊B1，並於第一圖像IMG1中標記出第一偵測區塊B1。Then, in step S260, the three-dimensional detecting device 100 extends the first preset height H1 upward in the three-dimensional space from the reference line segment L1 of the connection of the three-dimensional coordinates and the second corner image capturing three-dimensional coordinates. The first detection block B1 with the reference line segment L1 as the base in the three-dimensional space is obtained, and the first detection block B1 is marked in the first image IMG1.
由於第一角點攝像三維座標及第二角點攝像三維座標是根據深度處理單元120在步驟S220中所取得的深度資訊，因此第一角點攝像三維座標及第二角點攝像三維座標主要是以影像擷取單元110為參考點。然而一般來說，為了能夠減少死角，影像擷取單元110可能會以帶有俯角的方式來擷取影像，因此在第3圖中，以影像擷取單元110為參考點的攝像三維座標系統，其X軸X1及Y軸Y1所延伸的平面與真實場景中的水平地面並非平行，而其Z軸Z1也與真實場景中的鉛直高度並非平行(在第3圖中，以影像擷取單元110為參考點的三維座標系統，其Z軸Z1可例如為通過攝像鏡頭之鏡心及焦點的軸向)。在其他的實施例中，影像擷取單元110還可能根據需求以其他的角度來擷取影像。Since the first corner camera three-dimensional coordinates and the second corner camera three-dimensional coordinates are based on the depth information obtained by the depth processing unit 120 in step S220, the first corner camera three-dimensional coordinates and the second corner image three-dimensional coordinates are mainly The image capturing unit 110 is used as a reference point. In general, in order to reduce the dead angle, the image capturing unit 110 may capture the image with a depression angle. Therefore, in FIG. 3, the image capturing unit 110 uses the image capturing unit 110 as a reference point for the camera three-dimensional coordinate system. The plane in which the X-axis X1 and the Y-axis Y1 extend is not parallel to the horizontal ground in the real scene, and the Z-axis Z1 is also not parallel to the vertical height in the real scene (in FIG. 3, the image capturing unit 110 is used) As the reference point three-dimensional coordinate system, the Z-axis Z1 can be, for example, the axial direction of the lens and the focus through the imaging lens. In other embodiments, the image capturing unit 110 may also capture images at other angles as needed.
為了讓參考線段L1所延伸形成的偵測面能夠對應於直立於真實場景中的水平地面，以利三維偵測裝置100進行監控，步驟S260可先將第一角點攝像三維座標及第二角點攝像三維座標轉換為真實世界的三維座標體系，如此一來，就能夠迅速取得三維偵測裝置100監控所需的第一偵測區塊B1。在本發明的部分實施例中，三維偵測系統100可根據真實世界的水平地面來設定世界座標系統之X軸及Y軸。如此一來，世界座標系統的Z軸就可對應於鉛直於真實場景中水平地面的方向，另外還可由使用者輸入影像擷取單元110的架設角度和高度等資訊。如此一來，三維偵測系統100就能夠根據上述的資訊，在影像擷取單元110的座標系統與世界座標系統之間轉換。In order to allow the detection surface formed by the extension of the reference line segment L1 to correspond to the horizontal ground standing in the real scene for the three-dimensional detection device 100 to monitor, the step S260 may first capture the first corner point to capture the three-dimensional coordinates and the second angle. The point camera 3D coordinates are converted into a real world 3D coordinate system, so that the first detection block B1 required for the 3D detection device 100 to be monitored can be quickly obtained. In some embodiments of the present invention, the three-dimensional detection system 100 can set the X-axis and the Y-axis of the world coordinate system according to the real world level ground. In this way, the Z-axis of the world coordinate system can correspond to the direction perpendicular to the horizontal ground in the real scene, and the user can input information such as the installation angle and height of the image capturing unit 110. In this way, the three-dimensional detection system 100 can switch between the coordinate system of the image capturing unit 110 and the world coordinate system according to the above information.
第6圖為本發明一實施例之步驟S260所包含的子步驟S261至S267。Figure 6 is a sub-step S261 to S267 included in step S260 of an embodiment of the present invention.
S261： 將第一角點攝像三維座標轉換為對應於世界座標系統之第一角點世界三維座標；S261: converting the first corner camera three-dimensional coordinates into a world three-dimensional coordinate corresponding to the first corner point of the world coordinate system;
S262： 將第二角點攝像三維座標轉換為對應於世界座標系統之第二角點世界三維座標；S262: converting the second corner camera three-dimensional coordinates into a world three-dimensional coordinate corresponding to the second corner point of the world coordinate system;
S263： 自第一角點世界三維座標沿著世界座標系統之Z軸Z2方向延伸第一預設高度H1以取得第三角點P3之第三角點世界三維座標；S263: extending a first preset height H1 from a first corner point world three-dimensional coordinate along a Z-axis Z2 direction of the world coordinate system to obtain a third-dimensional coordinate of the third corner point of the third corner point P3;
S264： 自第二角點世界三維座標沿著世界座標系統之Z軸Z2方向延伸第一預設高度H1以取得第四角點P4之第四角點世界三維座標；S264: extending a first preset height H1 from a second corner point world three-dimensional coordinate along a Z-axis Z2 direction of the world coordinate system to obtain a fourth three-dimensional coordinate of the fourth corner point of the fourth corner point P4;
S265： 將第三角點世界三維座標轉換為對應於三維偵測裝置100之第三角點攝像三維座標；S265: Converting the third corner point world three-dimensional coordinate to a third corner point camera three-dimensional coordinate corresponding to the three-dimensional detecting device 100;
S266： 將第四角點世界三維座標轉換為對應於三維偵測裝置100之第四角點攝像三維座標；S266: Convert the fourth corner point world three-dimensional coordinate to a fourth corner point camera three-dimensional coordinate corresponding to the three-dimensional detecting device 100;
S267： 計算第三角點攝像三維座標及第四角點攝像三維座標投影於第一圖像IMG1的位置；及S267: calculating a third corner point camera three-dimensional coordinate and a fourth corner point camera three-dimensional coordinate projected on the first image IMG1; and
S268： 於第一圖像IMG1中標記出第一角點P1、第二角點P2、第三角點P3及第四角點P4所圍繞之第一偵測區塊B1。S268: Mark the first detection block B1 surrounded by the first corner point P1, the second corner point P2, the third corner point P3, and the fourth corner point P4 in the first image IMG1.
在步驟S261及步驟S262中，三維偵測裝置100可將第一角點P1的第一角點攝像三維座標及第二角點P2的第二角點攝像三維座標轉換為對應於世界座標系統之第一角點世界三維座標及第二角點世界三維座標。In step S261 and step S262, the three-dimensional detecting device 100 can convert the first corner image capturing three-dimensional coordinates of the first corner point P1 and the second corner point capturing three-dimensional coordinates of the second corner point P2 into corresponding to the world coordinate system. The first corner of the world three-dimensional coordinates and the second corner of the world three-dimensional coordinates.
在本發明的部分實施例中，方法200還可先根據第一圖像IMG1中的水平地面設定世界座標系統之X軸X2及世界座標系統之Y軸X3。如此一來，世界座標系統的Z軸Z2就可對應於鉛直於真實場景中水平地面的方向。因此在步驟S263及S264中，三維偵測裝置100就可以自第一角點世界三維座標直接沿著世界座標系統之Z軸Z2方向延伸第一預設高度H1以取得第三角點P3之第三角點世界三維座標，並自第二角點世界三維座標沿著世界座標系統之Z軸Z2方向直接延伸第一預設高度H1以取得第四角點P4之第四角點世界三維座標。In some embodiments of the present invention, the method 200 may first set the X axis X2 of the world coordinate system and the Y axis X3 of the world coordinate system according to the horizontal ground in the first image IMG1. In this way, the Z-axis Z2 of the world coordinate system can correspond to the direction perpendicular to the horizontal ground in the real scene. Therefore, in steps S263 and S264, the three-dimensional detecting device 100 can extend from the first corner point world three-dimensional coordinate directly along the Z-axis Z2 direction of the world coordinate system to the first preset height H1 to obtain the third corner of the third corner point P3. Pointing the world three-dimensional coordinates, and extending the first preset height H1 directly from the second-dimensional world three-dimensional coordinates along the Z-axis Z2 direction of the world coordinate system to obtain the fourth three-dimensional coordinate of the fourth corner point of the fourth corner point P4.
此時，第一角點P1、第二角點P2、第三角點P3及第四角點P4就會在三維空間中圍繞出一個與水平地面垂直的偵測面，亦即第一偵測區塊B1。為能將第一偵測區塊B1顯示於第一圖像IMG1中，步驟S265會先將第三角點世界三維座標轉換為對應於三維偵測裝置100之第三角點攝像三維座標，而步驟S266則會將第四角點世界三維座標轉換為對應於三維偵測裝置100之第四角點攝像三維座標，也就是將原先以世界座標系之X軸X2、Y軸Y2及Z軸Z2為參考基礎的第三角點世界三維座標及第四角點世界三維座標轉換為以三維偵測裝置100之X軸X1、Y軸Y1及Z軸Z1為參考基礎的第三角點攝像三維座標及第四角點攝像三維座標。At this time, the first corner point P1, the second corner point P2, the third corner point P3, and the fourth corner point P4 surround a detection surface perpendicular to the horizontal ground in the three-dimensional space, that is, the first detection area. Block B1. In order to display the first detection block B1 in the first image IMG1, the step S265 first converts the third corner point world three-dimensional coordinates into the third corner point camera three-dimensional coordinates corresponding to the three-dimensional detecting device 100, and step S266 The fourth corner world three-dimensional coordinate is converted into a three-dimensional coordinate corresponding to the fourth corner of the three-dimensional detecting device 100, that is, the original X-axis X2, Y-axis Y2 and Z-axis Z2 of the world coordinate system are used as reference. The third corner point world three-dimensional coordinate and the fourth corner point world three-dimensional coordinate are converted into a third corner point camera three-dimensional coordinate and a fourth corner based on the X-axis X1, the Y-axis Y1 and the Z-axis Z1 of the three-dimensional detecting device 100. Point camera 3D coordinates.
如此一來，在步驟S267中，就可以根據第三角點P3之第三角點攝像三維座標、第四角點P4之第四角點攝像三維座標以及影像擷取單元110的相關參數，例如焦距，來計算得知第三角點攝像三維座標及第四角點攝像三維座標投影於第一圖像IMG1的位置。接著，在步驟S268中，三維偵測裝置100就可以在第一圖像IMG1中標記出第一角點P1、第二角點P2、第三角點P3及第四角點P4所圍繞之第一偵測區塊B1。In this way, in step S267, the three-dimensional coordinates of the third corner point P3, the fourth corner point of the fourth corner point P4, and the related parameters of the image capturing unit 110, such as the focal length, may be captured. The position of the third corner camera three-dimensional coordinate and the fourth corner image three-dimensional coordinate projected on the first image IMG1 is calculated. Then, in step S268, the three-dimensional detecting device 100 can mark the first corner point P1, the second corner point P2, the third corner point P3, and the first corner point P4 in the first image IMG1. Block B1 is detected.
在本發明的部分實施例中，第一預設高度H1可為三維偵測裝置100的預設值，又或是由使用者輸入之預設值。此外，為了避免第一預設高度H1太大，導致第一偵測區塊B1的第三角點P3及第四角點P4無法在第一圖像IMG1中顯示，方法200還可另根據第三角點P3及第四角點P4位於第一圖像IMG1內的最大高度設定第一預設高度。舉例來說，三維偵測裝置100可以根據第一圖像IMG1所能夠顯示的範圍，來限定第一預設高度H1的值，而當使用者或系統預設的第一預設高度H1過大，而將使第三角點P3及第四角點P4無法在第一圖像IMG1中顯示時，改以第一圖像IMG1所能夠顯示的最大高度來設定第一預設高度H1。In some embodiments of the present invention, the first preset height H1 may be a preset value of the three-dimensional detecting device 100 or a preset value input by a user. In addition, in order to prevent the first preset height H1 from being too large, the third corner point P3 and the fourth corner point P4 of the first detection block B1 cannot be displayed in the first image IMG1, and the method 200 may further be based on the third angle. The point P3 and the fourth corner point P4 are located at a maximum height within the first image IMG1 to set a first preset height. For example, the three-dimensional detecting device 100 can define the value of the first preset height H1 according to the range that the first image IMG1 can display, and when the first preset height H1 preset by the user or the system is too large, When the third corner point P3 and the fourth corner point P4 are not displayed in the first image IMG1, the first preset height H1 is set to the maximum height that the first image IMG1 can display.
在步驟S270中，三維偵測裝置100還可進一步辨識第一圖像IMG1中所欲監控的物件，舉例來說，在第3圖的實施例中，三維偵測裝置100可監控是否有人通過第一偵測區塊B1，因此若第一圖像IMG1中包含了物件A，三維偵測裝置100會在步驟S270中辨識確認物件A是否為人。接著在步驟S280中，三維偵測裝置100則會進一步偵測物件A是否通過了第一偵測區塊B1。In step S270, the three-dimensional detecting device 100 can further identify the object to be monitored in the first image IMG1. For example, in the embodiment of FIG. 3, the three-dimensional detecting device 100 can monitor whether someone passes the first A block B1 is detected. Therefore, if the object A is included in the first image IMG1, the three-dimensional detecting device 100 recognizes whether the object A is a person in step S270. Next, in step S280, the three-dimensional detecting device 100 further detects whether the object A has passed the first detecting block B1.
舉例來說，當物件A自第一偵測區塊B1之第一側B11穿越第一偵測區塊B1至第一偵測區塊B1之第二側B12時，發出對應的訊號。舉例來說，當三維偵測裝置100被應用於安全監控時，三維偵測裝置100可在偵測到有人穿越第一偵測區塊B1時，發出警告的訊號，以通知使用者有人入侵。For example, when the object A traverses the first detection block B1 to the second side B12 of the first detection block B1 from the first side B11 of the first detection block B1, a corresponding signal is sent. For example, when the three-dimensional detecting device 100 is applied to the security monitoring, the three-dimensional detecting device 100 can send a warning signal to notify the user that someone is invading when detecting that someone passes through the first detecting block B1.
然而在本發明的其他實施例中，三維偵測裝置100也可能應用於其他場域中，例如協助賣場統計場內的人數，此時三維偵測裝置100也可根據步驟S280所產生的對應訊號，使對應於第一偵測區塊B1之第二側B12之人數量加一，或使對應於第一偵測區塊B1之第一側B11之人數量減一。However, in other embodiments of the present invention, the three-dimensional detecting apparatus 100 may also be applied to other fields, for example, to assist the number of people in the market, and the three-dimensional detecting apparatus 100 may also generate corresponding signals according to step S280. The number of people corresponding to the second side B12 of the first detection block B1 is increased by one, or the number of persons corresponding to the first side B11 of the first detection block B1 is decreased by one.
透過方法200，三維偵測裝置100就可以將使用者界定的參考線段L1延伸為三維空間的偵測面，亦即第一偵測區塊B1。且由於三維偵測裝置100能夠取得第一圖像IMG1中各個物件的三維空間位置，且當物件變動位置時，三維偵測裝置100也能夠得知物件在三維空間中實際的位置變化，因此三維偵測裝置100可利用對應於相同之三維空間的第一偵測區塊B1來對第一圖像IMG1中的物件更加精準地進行監控。Through the method 200, the three-dimensional detecting device 100 can extend the user-defined reference line segment L1 into a three-dimensional detecting surface, that is, the first detecting block B1. Moreover, since the three-dimensional detecting device 100 can obtain the three-dimensional spatial position of each object in the first image IMG1, and when the object changes position, the three-dimensional detecting device 100 can also know the actual position change of the object in the three-dimensional space, and thus the three-dimensional The detecting device 100 can monitor the objects in the first image IMG1 more accurately by using the first detecting block B1 corresponding to the same three-dimensional space.
此外，三維偵測裝置100也可應用於倉儲管理，此時則可利用方法200來辨識對應區域中的物件是否被取走，又或是有新的物件被置入，進而即時更新物件的存量。第7圖為本發明另一實施例之三維偵測裝置100所擷取之第一圖像IMG1’的示意圖。In addition, the three-dimensional detecting device 100 can also be applied to warehouse management. In this case, the method 200 can be used to identify whether the object in the corresponding area is taken away, or a new object is placed, thereby updating the inventory of the object in real time. . FIG. 7 is a schematic diagram of a first image IMG1' captured by the three-dimensional detecting apparatus 100 according to another embodiment of the present invention.
在第7圖的實施例中，三維偵測裝置100可應用於倉儲管理。第7圖中，儲物架D具有兩個儲物格空間D1及D2，且分別存放不同類型的物件。因此使用者可將第一角點P1’及第二角點P2’設定為儲物架D的下方的兩個角落，並根據儲物格空間D1的高度來設定第一預定高度H1，如此一來，便可利用方法200取得第三角點P3’及第四角點P4’並建立第一偵測區塊B1’，並利用第一偵測區塊B1’監控儲物格空間D1中的物件是否被拿取，又或是否有新的物件被置入。In the embodiment of Fig. 7, the three-dimensional detecting device 100 can be applied to warehouse management. In Fig. 7, the storage rack D has two storage compartments D1 and D2, and respectively stores different types of objects. Therefore, the user can set the first corner point P1′ and the second corner point P2′ as two corners below the storage rack D, and set the first predetermined height H1 according to the height of the storage space D1, such that Then, the method 200 can be used to obtain the third corner point P3' and the fourth corner point P4' and establish the first detection block B1', and use the first detection block B1' to monitor the objects in the storage space D1. Whether it was taken, or whether new items were placed.
此外，方法200還可以利用與步驟S263至S268相似的方式，繼續建立第二偵測區塊B2， 並利用第一偵測區塊B1’監控儲物格空間D2中的物件是否被拿取，又或是否有新的物件被置入。In addition, the method 200 can continue to establish the second detection block B2 in a manner similar to steps S263 to S268, and use the first detection block B1' to monitor whether the object in the storage space D2 is taken. Or if there are new objects placed.
舉例來說，三維偵測裝置100可自第三角點P3’的第三角點世界三維座標沿著世界座標系統之Z軸Z2方向延伸第二預設高度H2以取得第五角點P5之第五角點世界三維座標，並自第四角點P4’之第四角點世界三維座標沿著世界座標系統之Z軸Z2方向延伸第二預設高度H2以取得第六角點P6之第六角點世界三維座標。For example, the three-dimensional detecting device 100 can extend from the third corner point of the third corner point P3' to the second preset height H2 along the Z-axis Z2 direction of the world coordinate system to obtain the fifth fifth point P5. The three-dimensional coordinates of the corner point world, and the third three-dimensional coordinates of the fourth corner point of the fourth corner point P4' extend along the Z-axis Z2 direction of the world coordinate system to the second preset height H2 to obtain the sixth hexagon of the sixth hexagon point P6. Point the world's three-dimensional coordinates.
接著，再將第五角點世界三維座標轉換為對應於三維偵測裝置100之第五角點攝像三維座標，並將第六角點世界三維座標轉換為對應於三維偵測裝置100之第六角點攝像三維座標。此時，三維偵測裝置100便可計算第五角點攝像三維座標及第六角點攝像三維座標投影於第一圖像IMG1’的位置，並可在第一圖像IMG1中標記出第三角點P3’、第四角點P4’、第五角點P5及第六角點P6所圍繞之第二偵測區塊B2。Then, the fifth corner point world three-dimensional coordinate is converted into a fifth corner point camera three-dimensional coordinate corresponding to the three-dimensional detecting device 100, and the sixth hexagonal world three-dimensional coordinate is converted into a sixth corresponding to the three-dimensional detecting device 100. Corner point camera three-dimensional coordinates. At this time, the three-dimensional detecting device 100 can calculate the position of the fifth corner point camera three-dimensional coordinate and the hexagonal point camera three-dimensional coordinate projected on the first image IMG1', and mark the third angle in the first image IMG1. The second detection block B2 surrounded by the point P3', the fourth corner point P4', the fifth corner point P5 and the sixth hexagon point P6.
在本實施例中，第二預設高度H2會對應於儲物格空間D2的高度，因此在建立第二偵測區塊B2之後，三維偵測裝置100便可利用第二偵測區塊B2監控儲物格空間D2中的物件是否被拿取，又或是否有新的物件被置入。In this embodiment, the second preset height H2 corresponds to the height of the storage space D2. Therefore, after the second detection block B2 is established, the three-dimensional detecting device 100 can utilize the second detection block B2. It is monitored whether the object in the storage space D2 is taken, or whether a new object is placed.
同理，在本發明的部分實施例中，還可利用與方法200相似的步驟，繼續增加新的偵測區塊來監控更多的空間。Similarly, in some embodiments of the present invention, steps similar to method 200 may be utilized to continue to add new detection blocks to monitor more space.
綜上所述，本發明之實施例所提供之三維偵測裝置的操作方法可以將使用者界定的參考線段延伸為三維空間的偵測面，且能夠取得圖像中各個物件的三維空間位置，因此當物件變動位置時，三維偵測裝置也能夠得知物件在三維空間中實際的位置變化。如此一來，三維偵測裝置就可以利用對應於相同之三維空間的偵測區塊來對圖像中的物件做更加精準地進行監控，而減少誤判的情況。 以上所述僅為本發明之較佳實施例，凡依本發明申請專利範圍所做之均等變化與修飾，皆應屬本發明之涵蓋範圍。In summary, the operation method of the three-dimensional detecting device provided by the embodiment of the present invention can extend the reference line segment defined by the user into the detecting surface of the three-dimensional space, and can obtain the three-dimensional spatial position of each object in the image. Therefore, when the object changes position, the three-dimensional detecting device can also know the actual position change of the object in the three-dimensional space. In this way, the three-dimensional detecting device can use the detecting block corresponding to the same three-dimensional space to more accurately monitor the objects in the image, thereby reducing the false positives. The above are only the preferred embodiments of the present invention, and all changes and modifications made to the scope of the present invention should be within the scope of the present invention.
IMG0‧‧‧畫面IMG0‧‧‧ screen
K‧‧‧人物K‧‧‧ character
L0‧‧‧偵測線L0‧‧‧ detection line
IMG1、IMG1’‧‧‧第一圖像IMG1, IMG1’‧‧‧ first image
IMG2‧‧‧第二圖像IMG2‧‧‧ second image
100、300‧‧‧三維偵測系統100, 300‧‧‧3D detection system
110、310‧‧‧影像擷取單元110, 310‧‧‧Image capture unit
112、114、312‧‧‧攝像鏡頭112, 114, 312‧‧ ‧ camera lens
120、320‧‧‧深度處理單元120, 320‧‧‧Deep processing unit
130、330‧‧‧設定輸入單元130, 330‧‧‧Set input unit
A‧‧‧物件A‧‧‧ objects
B1、B1’‧‧‧第一偵測面B1, B1’‧‧‧ first detection surface
P1、P1’‧‧‧第一角點P1, P1’‧‧‧ first corner
P2、P2’‧‧‧第二角點P2, P2’‧‧‧ second corner
P3、P3’‧‧‧第三角點P3, P3’‧‧‧ third corner
P4、P4’‧‧‧第四角點P4, P4’‧‧‧ fourth corner
H1‧‧‧第一預設高度H1‧‧‧First preset height
B11‧‧‧第一偵測面之第一側B11‧‧‧ first side of the first detection surface
B12‧‧‧第一偵測面之第二側B12‧‧‧ second side of the first detection surface
X1‧‧‧攝像三維座標系統之X軸X1‧‧‧ X-axis of camera three-dimensional coordinate system
Y1‧‧‧攝像三維座標系統之Y軸Y1‧‧‧ camera Y-axis of the three-dimensional coordinate system
Z1‧‧‧攝像三維座標系統之Z軸Z1‧‧‧Z-axis of the camera three-dimensional coordinate system
X2‧‧‧世界三維座標系統之X軸X2‧‧‧X-axis of the world's three-dimensional coordinate system
Y2‧‧‧世界三維座標系統之Y軸Y2‧‧‧Y-axis of the world's three-dimensional coordinate system
Z2‧‧‧世界三維座標系統之Z軸Z2‧‧‧Z-axis of the world's three-dimensional coordinate system
200‧‧‧方法200‧‧‧ method
S210至S280、S261至S268‧‧‧步驟Steps S210 to S280, S261 to S268‧‧
340‧‧‧光學傳感器340‧‧‧ optical sensor
D‧‧‧儲物架D‧‧‧ Storage shelves
D1、D2‧‧‧儲物格空間D1, D2‧‧‧ storage space
P5‧‧‧第五角點P5‧‧‧ fifth corner
P6‧‧‧第六角點P6‧‧‧ Sixth point
H2‧‧‧第二預設高度H2‧‧‧Second preset height
B2‧‧‧第二偵測面B2‧‧‧Second detection surface
第1圖為先前技術之影像監控系統所拍攝的畫面示意圖。 第2圖為本發明一實施例之三維偵測裝置的示意圖。 第3圖為第2圖之影像擷取單元所擷取之第一圖像的示意圖。 第4圖為操作第2圖之三維偵測裝置的方法流程圖。 第5圖為本發明另一實施例之三維偵測裝置的示意圖。 第6圖為第4圖之步驟所包含的子步驟的流程圖。 第7圖為本發明另一實施例之三維偵測裝置所擷取之第一圖像的示意圖。Figure 1 is a schematic diagram of a screen taken by a prior art image monitoring system. FIG. 2 is a schematic diagram of a three-dimensional detecting device according to an embodiment of the present invention. Figure 3 is a schematic diagram of the first image captured by the image capturing unit of Figure 2. Fig. 4 is a flow chart showing the method of operating the three-dimensional detecting device of Fig. 2. FIG. 5 is a schematic diagram of a three-dimensional detecting device according to another embodiment of the present invention. Figure 6 is a flow diagram of the sub-steps included in the steps of Figure 4. FIG. 7 is a schematic diagram of a first image captured by a three-dimensional detecting device according to another embodiment of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710600103.5A CN108665480A (en) | 2017-03-31 | 2017-07-21 | Operation method of three-dimensional detection device |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201762479325P | 2017-03-31 | 2017-03-31 | |
| US62/479,325 | 2017-03-31 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| TW201837857A true TW201837857A (en) | 2018-10-16 |
Family
ID=64797393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW106121733A TW201837857A (en) | 2017-03-31 | 2017-06-29 | Method for operating a three dimension detection device |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TW201837857A (en) |
-
2017
- 2017-06-29 TW TW106121733A patent/TW201837857A/en unknown
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| TWI696906B (en) | Method for processing a floor | |
| KR100776649B1 (en) | A depth information-based Stereo/Multi-view Stereo Image Matching Apparatus and Method | |
| CN107392958B (en) | Method and device for determining object volume based on binocular stereo camera | |
| CN107079093B (en) | Calibration device | |
| Holzmann et al. | Measuring distance with mobile phones using single-camera stereo vision | |
| US20170214899A1 (en) | Method and system for presenting at least part of an image of a real object in a view of a real environment, and method and system for selecting a subset of a plurality of images | |
| KR102354299B1 (en) | Camera calibration method using single image and apparatus therefor | |
| US20170140537A1 (en) | System and method for scoring clutter for use in 3d point cloud matching in a vision system | |
| CN104715479A (en) | Scene reproduction detection method based on augmented virtuality | |
| JP5951043B2 (en) | Image measuring device | |
| Borrmann et al. | Mutual calibration for 3D thermal mapping | |
| CN111787303B (en) | Three-dimensional image generation method and device, storage medium and computer equipment | |
| US20220189113A1 (en) | Method for generating 3d skeleton using joint-based calibration acquired from multi-view camera | |
| KR20200031019A (en) | Depth data processing system capable of performing image registration on depth maps to optimize depth data | |
| Yamaguchi et al. | Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry | |
| Beltran et al. | A comparison between active and passive 3d vision sensors: Bumblebeexb3 and Microsoft Kinect | |
| KR101620580B1 (en) | Method and system for dectecting run | |
| Kochi et al. | 3D modeling of architecture by edge-matching and integrating the point clouds of laser scanner and those of digital camera | |
| Sato et al. | Efficient hundreds-baseline stereo by counting interest points for moving omni-directional multi-camera system | |
| TW201837857A (en) | Method for operating a three dimension detection device | |
| KR101316387B1 (en) | Method of object recognition using vision sensing and distance sensing | |
| JP4269781B2 (en) | Optical flow detection system, detection method and detection program | |
| Anwer et al. | Calculating real world object dimensions from Kinect RGB-D image using dynamic resolution | |
| Abdelhamid et al. | Extracting depth information using a correlation matching algorithm |