TW201917697A - Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images - Google Patents
Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images Download PDFInfo
- Publication number
- TW201917697A TW201917697A TW106136425A TW106136425A TW201917697A TW 201917697 A TW201917697 A TW 201917697A TW 106136425 A TW106136425 A TW 106136425A TW 106136425 A TW106136425 A TW 106136425A TW 201917697 A TW201917697 A TW 201917697A
- Authority
- TW
- Taiwan
- Prior art keywords
- fisheye
- image
- lens
- distance
- center
- Prior art date
Links
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
Description
本發明是有關於一種影像擷取裝置及方法，且特別是有關於一種利用魚眼影像計算所攝物體座標的方法及影像擷取裝置。The present invention relates to an image capturing device and method, and more particularly to a method for calculating a coordinate of a captured object using a fisheye image and an image capturing device.
遊戲機是現代人家庭娛樂不可或缺的電子產品之一。為了增加玩家與遊戲內容的互動，許多遊戲機已摒除傳統手把的控制模式，加入了體感偵測的元素。藉由紅外線等感測器偵測使用者於空間中的移動或動作，並反應於遊戲內容的操控，而大幅增加遊戲的娛樂性。The game console is one of the indispensable electronic products for modern family entertainment. In order to increase the interaction between the player and the game content, many game consoles have removed the control mode of the traditional handlebar and added the elements of somatosensory detection. The sensor or the like detects the movement or motion of the user in the space, and responds to the manipulation of the game content, thereby greatly increasing the entertainment of the game.
除了早期的紅外線感測定位技術之外，近期的遊戲機更導入了光球偵測技術，當使用者手持光球遙桿在空間中揮舞時，遊戲機會藉由配置其上的雙鏡頭拍攝光球的影像，並根據光球在影像中的位置計算光球在空間中的位置。In addition to the early infrared sensation measurement technology, the recent game console has introduced a light ball detection technology. When the user holds the ray ball to swing in the space, the game opportunity is configured by the dual lens shooting light. The image of the ball, and the position of the ball in space is calculated based on the position of the ball in the image.
然而，由於傳統鏡頭的視野（field of view，FOV）有限，此將使得上述雙鏡頭的拍攝範圍受到限制，連帶使得使用者的移動空間也受到限制。而若將此鏡頭替換成視野較廣的魚眼鏡頭，則因為魚眼鏡頭所擷取的影像會有變形，需要進行魚眼校正後才能用來定位，但校正過程需要經過幾何轉換（geometry transformation），此將降低視訊影像的管線期間（pipeline duration），而有可能需要透過增加圖框率（frame rate）來解決。However, due to the limited field of view (FOV) of the conventional lens, this will limit the shooting range of the above dual lens, and the user's moving space is also limited. If the lens is replaced with a fisheye lens with a wide field of view, the image captured by the fisheye lens will be deformed and need to be corrected by fisheye before it can be used for positioning. However, the calibration process needs to undergo geometric transformation. ), this will reduce the pipeline duration of the video image, and may need to be solved by increasing the frame rate.
本發明提供一種利用魚眼影像計算所攝物體座標的方法及影像擷取裝置，可在不進行幾何轉換的情況下，使用魚眼影像計算出物體於空間中的三維座標。The invention provides a method for calculating a coordinate of an object taken by using a fisheye image and an image capturing device, which can calculate a three-dimensional coordinate of an object in a space by using a fisheye image without geometric conversion.
本發明的利用魚眼影像計算所攝物體座標的方法，適用於具有魚眼鏡頭的影像擷取裝置。所述方法是利用魚眼鏡頭擷取包括一物體的魚眼影像，接著偵測此物體在魚眼影像中的座標，而根據此座標，計算物體在魚眼鏡頭的影像感測器平面上相對於魚眼影像的魚眼中心的方位角，之後則利用魚眼鏡頭的鏡頭曲線，將所述座標與魚眼影像的魚眼中心的距離轉換為入射角，並根據所述物體在魚眼影像中朝向所述魚眼中心的方向上的寬度，計算物體與魚眼鏡頭之間的距離，最後則根據所述方位角、入射角以及距離，利用三角函數計算物體的三維座標。The method for calculating the coordinates of a subject using the fisheye image of the present invention is applicable to an image capturing device having a fisheye lens. The method uses a fisheye lens to capture a fisheye image including an object, and then detects a coordinate of the object in the fisheye image, and according to the coordinate, the object is calculated on the image sensor plane of the fisheye lens. The azimuth of the fisheye center of the fisheye image, and then the lens curve of the fisheye lens is used to convert the distance between the coordinate and the fisheye center of the fisheye image into an incident angle, and according to the object in the fisheye image The width in the direction toward the center of the fisheye is calculated, and the distance between the object and the fisheye lens is calculated. Finally, the three-dimensional coordinates of the object are calculated using a trigonometric function according to the azimuth angle, the incident angle, and the distance.
在本發明的一實施例中，所述物體包括發光裝置，而偵測物體在魚眼影像中的座標的步驟包括偵測魚眼影像中亮度或顏色分量大於預設值的多個像素，而以這些像素所形成區域的中心或重心在魚眼影像中的座標作為所述座標。In an embodiment of the invention, the object includes a light emitting device, and the step of detecting a coordinate of the object in the fisheye image includes detecting a plurality of pixels in the fisheye image that have a brightness or color component greater than a preset value, and The coordinates of the center or center of gravity of the area formed by these pixels in the fisheye image are taken as the coordinates.
本發明的影像擷取裝置包括魚眼鏡頭、儲存裝置及處理器。其中，儲存裝置是用以儲存多個模組。處理器耦接魚眼鏡頭及儲存裝置，用以存取並執行儲存於儲存裝置中的多個模組。這些模組包括影像擷取模組、物體偵測模組、方位角計算模組、入射角計算模組、距離計算模組及座標計算模組。影像擷取模組是利用魚眼鏡頭擷取包括一物體的魚眼影像。物體偵測模組是偵測所述物體在魚眼影像中的座標。方位角計算模組是根據所述座標，計算物體在魚眼鏡頭的影像感測器平面上相對於魚眼影像的魚眼中心的方位角。入射角計算模組是利用魚眼鏡頭的鏡頭曲線，將所述座標與魚眼影像的魚眼中心的距離轉換為入射角。距離計算模組是根據所述物體在魚眼影像中朝向魚眼中心的方向上的寬度，計算此物體與魚眼鏡頭之間的距離。座標計算模組是根據所述的方位角、入射角以及距離，利用三角函數計算所述物體的三維座標。The image capturing device of the present invention comprises a fisheye lens, a storage device and a processor. The storage device is used to store a plurality of modules. The processor is coupled to the fisheye lens and the storage device for accessing and executing a plurality of modules stored in the storage device. The modules include an image capture module, an object detection module, an azimuth calculation module, an incident angle calculation module, a distance calculation module, and a coordinate calculation module. The image capturing module uses a fisheye lens to capture a fisheye image including an object. The object detection module is a coordinate for detecting the object in the fisheye image. The azimuth calculation module calculates an azimuth of the object relative to the fisheye center of the fisheye image on the image sensor plane of the fisheye lens according to the coordinates. The incident angle calculation module converts the distance between the coordinates and the fisheye center of the fisheye image into an incident angle by using a lens curve of the fisheye lens. The distance calculation module calculates the distance between the object and the fisheye lens according to the width of the object in the direction of the fisheye image toward the center of the fisheye. The coordinate calculation module calculates a three-dimensional coordinate of the object by using a trigonometric function according to the azimuth angle, the incident angle, and the distance.
在本發明的一實施例中，所述的物體包括發光裝置，而所述物體偵測模組包括偵測魚眼影像中亮度或顏色分量大於預設值的多個像素，而以這些像素所形成區域的中心或重心在魚眼影像中的座標作為所述座標。In an embodiment of the invention, the object includes a light emitting device, and the object detecting module includes a plurality of pixels for detecting a brightness or color component of the fisheye image greater than a preset value, and the pixels are The coordinates of the center of the formation area or the center of gravity in the fisheye image are taken as the coordinates.
在本發明的一實施例中，所述物體所發出的光在魚眼鏡頭上的入射角與此光在魚眼鏡頭的影像感測器上的投影半徑呈正比。In an embodiment of the invention, the angle of incidence of the light emitted by the object on the fisheye lens is proportional to the projected radius of the light on the image sensor of the fisheye lens.
在本發明的一實施例中，假設所述物體在魚眼影像中的寬度為wT 、此物體的實際寬度為WT 、魚眼影像的影像寬度為WI 、魚眼鏡頭的視野（Field of View，FOV）為F ，則所述物體與魚眼鏡頭之間的距離r 為： 。In an embodiment of the invention, it is assumed that the width of the object in the fisheye image is w T , the actual width of the object is W T , the image width of the fisheye image is W I , and the field of view of the fisheye lens (Field) Of View, FOV) is F , then the distance r between the object and the fisheye lens is: .
在本發明的一實施例中，假設所述方位角為φ 、所述入射角為θ 以及所述距離為r ，所述物體的三維座標為(x ,y ,z )，其中 ； ；以及 。In an embodiment of the invention, assuming that the azimuth angle is φ , the incident angle is θ, and the distance is r , the three-dimensional coordinates of the object are ( x , y , z ), wherein;
基於上述，本發明的利用魚眼影像計算所攝物體座標的方法及影像擷取裝置利用雙魚眼鏡頭擷取物體影像，在不進行幾何轉換的情況，直接使用物體在魚眼影像中的座標計算由物體發射之光在魚眼鏡頭上的入射角以及在鏡頭平面上的方位角，並根據物體在魚眼影像中的寬度估測物體與魚眼鏡頭的距離，最後利用三角函數計算該物體在空間中的三維座標。Based on the above, the method for calculating the coordinates of the object by using the fisheye image and the image capturing device use the fisheye lens to capture the image of the object, and directly calculate the coordinate of the object in the fisheye image without geometric conversion. The angle of incidence of the light emitted by the object on the fisheye lens and the azimuth angle on the plane of the lens, and estimate the distance between the object and the fisheye lens according to the width of the object in the fisheye image. Finally, the object is calculated in space by using a trigonometric function. The three-dimensional coordinates in the middle.
為讓本發明的上述特徵和優點能更明顯易懂，下文特舉實施例，並配合所附圖式作詳細說明如下。The above described features and advantages of the invention will be apparent from the following description.
由於魚眼鏡頭採用等距鏡頭（Equi-Distance lens），其自物體（例如光球）接收的光的入射角會與此光在影像感測器上的投影半徑呈近乎線性的關係。據此，本發明即藉由偵測物體在魚眼影像中的位置，利用上述關係反推物體發光的入射角，並取得物體在鏡頭平面上的方位角。另一方面，本發明還依據物體在魚眼影像中的寬度，進一步推算物體與魚眼鏡頭之間的距離，最終依據所計算的距離及入射角與方位角，即可利用三角函數計算出物體於空間中的三維座標。藉此，本發明可在增加拍攝視野且不進行幾何轉換的情況下，實現所攝物體的三維座標計算。Since the fisheye lens adopts an Equi-Distance lens, the incident angle of light received from an object (such as a light ball) is nearly linear with the projection radius of the light on the image sensor. Accordingly, the present invention detects the position of the object in the fisheye image, uses the above relationship to reverse the incident angle of the object, and obtains the azimuth of the object on the lens plane. On the other hand, the invention further estimates the distance between the object and the fisheye lens according to the width of the object in the fisheye image, and finally calculates the object by using a trigonometric function according to the calculated distance and the incident angle and the azimuth angle. The three-dimensional coordinates in space. Thereby, the present invention can realize the three-dimensional coordinate calculation of the object to be taken while increasing the field of view and without performing geometric conversion.
圖1是依照本發明一實施例所繪示的影像擷取裝置的方塊圖。請參照圖1，本實施例的影像擷取裝置10例如是手機、平板電腦、筆記型電腦、導航裝置、行車紀錄器、數位相機、數位攝影機（Digital video camcorder，DVC）等具備攝像功能的電子裝置。影像擷取裝置10中包括魚眼鏡頭12、儲存裝置14及處理器16，其功能分述如下：FIG. 1 is a block diagram of an image capture device according to an embodiment of the invention. Referring to FIG. 1 , the image capturing device 10 of the embodiment is, for example, a mobile phone, a tablet computer, a notebook computer, a navigation device, a driving recorder, a digital camera, a digital video camcorder (DVC), and the like. Device. The image capturing device 10 includes a fisheye lens 12, a storage device 14, and a processor 16, and its functions are as follows:
魚眼鏡頭12包括鏡頭及影像感測器，其中所述鏡頭是採用視角接近、等於或超過180度的定焦或變焦鏡頭，其可使得位於其視野（Field of View，FOV）內的被攝物體成像在影像感測器上。影像感測器中配置有電荷耦合元件（Charge coupled device，CCD）、互補性氧化金屬半導體（Complementary metal-oxide semiconductor，CMOS）元件或其他種類的感光元件，而可感測進入鏡頭的光線強度，從而擷取影像訊號以產生魚眼影像。The fisheye lens 12 includes a lens and an image sensor, wherein the lens is a fixed focus or zoom lens that is close to, equal to, or exceeds 180 degrees with a viewing angle, which allows a shot to be taken within its Field of View (FOV) The object is imaged on the image sensor. The image sensor is provided with a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) component or other kinds of photosensitive elements, and can sense the intensity of light entering the lens. Thereby capturing the image signal to generate a fisheye image.
儲存裝置14例如是任何型態的固定式或可移動式隨機存取記憶體（random access memory，RAM）、唯讀記憶體（read-only memory，ROM）、快閃記憶體（flash memory）或類似元件或上述元件的組合。在本實施例中，儲存裝置14用以記錄影像擷取模組141、物體偵測模組142、方位角計算模組143、入射角計算模組144、距離計算模組145及座標計算模組146，這些模組例如是儲存在儲存裝置14中的程式。The storage device 14 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory or Similar elements or combinations of the above elements. In this embodiment, the storage device 14 is configured to record the image capturing module 141, the object detecting module 142, the azimuth calculating module 143, the incident angle calculating module 144, the distance calculating module 145, and the coordinate calculating module. 146. These modules are, for example, programs stored in the storage device 14.
處理器16例如是中央處理單元（Central Processing Unit，CPU），或是其他可程式化之一般用途或特殊用途的微處理器（Microprocessor）、數位訊號處理器（Digital Signal Processor，DSP）、可程式化控制器、特殊應用積體電路（Application Specific Integrated Circuits，ASIC）、可程式化邏輯裝置（Programmable Logic Device，PLD）或其他類似裝置或這些裝置的組合。處理器16連接第一魚眼鏡頭12、第二魚眼鏡頭14及儲存裝置14，而配置用以從儲存裝置14載入影像擷取模組141、物體偵測模組142、方位角計算模組143、入射角計算模組144、距離計算模組145及座標計算模組146的程式，據以執行本申請利用魚眼影像計算所攝物體座標的方法。The processor 16 is, for example, a central processing unit (CPU), or other programmable general purpose or special purpose microprocessor (Microprocessor), digital signal processor (DSP), programmable Controllers, Application Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), or other similar devices or combinations of these devices. The processor 16 is connected to the first fisheye lens 12, the second fisheye lens 14 and the storage device 14, and is configured to load the image capturing module 141, the object detecting module 142, and the azimuth calculation module from the storage device 14. The group 143, the angle of incidence calculation module 144, the distance calculation module 145, and the coordinate calculation module 146 are used to perform the method of calculating the object coordinates of the object using the fisheye image.
詳言之，圖2是依照本發明一實施例所繪示的利用魚眼影像計算所攝物體座標的方法的流程圖。請同時參照圖1及圖2，本實施例的方法適用於圖1的影像擷取裝置10，以下即搭配影像擷取裝置10中的各項元件說明本案之利用魚眼影像計算所攝物體座標的方法的詳細步驟。In more detail, FIG. 2 is a flow chart of a method for calculating a coordinate of a captured object using a fisheye image according to an embodiment of the invention. Referring to FIG. 1 and FIG. 2 simultaneously, the method of the embodiment is applicable to the image capturing device 10 of FIG. 1 , and the following describes the coordinates of the object in the case of using the fisheye image in the image capturing device 10 . Detailed steps of the method.
首先，處理器16執行影像擷取模組141，以利用魚眼鏡頭12擷取包括一物體的魚眼影像（步驟S202）。其中，所述物體例如是白色、藍色、紅色、綠色或其他易辦識顏色的光球，其例如是配置在遙桿、遙控器、虛擬實境的頭戴式顯示器、頭盔或是手環、手錶等可穿戴式裝置上，而可藉由發出白光或其他顏色的光，讓影像擷取裝置10可辨識出手持或配戴此物體之使用者的動作。First, the processor 16 executes the image capturing module 141 to capture the fisheye image including an object by using the fisheye lens 12 (step S202). Wherein, the object is, for example, white, blue, red, green or other easy-to-know light ball, which is, for example, a head mounted display, a helmet or a wristband disposed on a joystick, a remote controller, a virtual reality On a wearable device such as a watch, the image capturing device 10 can recognize the motion of the user holding the object or wearing the object by emitting white light or other colors of light.
接著，處理器16執行物體偵測模組142，以偵測物體在魚眼影像中的座標（步驟S204）。其中，物體偵測模組142例如會偵測魚眼影像中亮度或某一顏色分量大於預設值的多個像素，而以這些像素所形成區域的中心或重心在魚眼影像中的座標作為所述的座標。Next, the processor 16 executes the object detection module 142 to detect the coordinates of the object in the fisheye image (step S204). The object detecting module 142 detects, for example, a plurality of pixels in the fisheye image whose brightness or a certain color component is greater than a preset value, and uses the coordinates of the center or center of gravity of the area formed by the pixels in the fisheye image as The coordinates described.
詳言之，若所述物體為光球，則此物體在魚眼影像中將以亮度較高或某一顏色分量較高的圓形（或橢圓形）區域呈現，因此，物體偵測模組142即藉由將魚眼影像中各個像素的亮度值與預設值比較，以偵測出物體（白色光球），或是將魚眼影像中各個像素的某個顏色分量（例如R、G、B）的像素值（例如藍色分量的像素值）與預設值比較，以偵測出具有該顏色的物體（例如藍色光球）。另一方面，魚眼鏡頭所拍攝的物體，會依其偏離魚眼中心的距離而產生不同程度的形變（例如圓形的光球會變成橢圓形），因此，在確定物體於魚眼影像中的座標時，物體偵測模組142例如會計算此物體的中心（例如範圍可涵括此物體的最小矩形的中心）或重心，而以此中心或重心在魚眼影像中的座標作為物體的座標。In detail, if the object is a light sphere, the object will be presented in a fisheye image with a circular (or elliptical) region with a higher brightness or a higher color component. Therefore, the object detection module 142 is to detect an object (white light ball) by comparing the brightness value of each pixel in the fisheye image with a preset value, or to a certain color component of each pixel in the fisheye image (for example, R, G) The pixel value of B) (for example, the pixel value of the blue component) is compared with a preset value to detect an object having the color (for example, a blue light sphere). On the other hand, the object photographed by the fisheye lens will be deformed to different degrees according to the distance from the center of the fisheye (for example, a round photosphere will become elliptical), therefore, in determining the object in the fisheye image The object detection module 142, for example, calculates the center of the object (for example, the center of the smallest rectangle that can encompass the object) or the center of gravity, and the coordinates of the center or center of gravity in the fisheye image are used as objects. coordinate.
然後，處理器16執行方位角計算模組143，以根據所述座標，計算物體在魚眼鏡頭的影像感測器平面上相對於魚眼影像的魚眼中心的方位角（步驟S206）。此外，處理器16還會執行入射角計算模組144，以利用魚眼鏡頭12的鏡頭曲線，將前述座標與魚眼影像的魚眼中心的距離轉換為入射角（步驟S208）。其中，處理器16執行的順序並不限於上述的步驟S206、步驟S208，亦可先執行步驟S208、再執行步驟S206，或同時執行。Then, the processor 16 executes the azimuth calculation module 143 to calculate an azimuth of the object on the image sensor plane of the fisheye lens with respect to the fisheye center of the fisheye image according to the coordinates (step S206). In addition, the processor 16 also executes the incident angle calculation module 144 to convert the distance between the coordinates and the fisheye center of the fisheye image into an incident angle by using the lens curve of the fisheye lens 12 (step S208). The sequence executed by the processor 16 is not limited to the foregoing steps S206 and S208, and may be performed first in step S208, then in step S206, or simultaneously.
魚眼鏡頭12所拍攝物體在其影像感測器平面上的投影半徑，會依據其鏡頭曲線而有不同。在一實施例中，若魚眼鏡頭12是採用等距鏡頭，則其所接收光的入射角會與此光在影像感測器上的投影半徑呈近乎線性的關係。意即，由物體所發出的光在魚眼鏡頭12上的入射角與此光在魚眼鏡頭12的影像感測器上的投影半徑呈正比。而在另一實施例中，魚眼鏡頭12所接收光的入射角與此光在影像感測器上的投影半徑彼此間也可具有多項式函數的關係，此關係可藉由預先取得鏡頭曲線，或預先測定不同角度入射光在影像感測器上的投影半徑，而預先取得。The projection radius of the object photographed by the fisheye lens 12 on its image sensor plane will vary depending on its lens curve. In an embodiment, if the fisheye lens 12 is an equidistant lens, the incident angle of the received light will be in a nearly linear relationship with the projection radius of the light on the image sensor. That is, the angle of incidence of the light emitted by the object on the fisheye lens 12 is proportional to the projected radius of the light on the image sensor of the fisheye lens 12. In another embodiment, the incident angle of the light received by the fisheye lens 12 and the projection radius of the light on the image sensor may also have a polynomial function relationship, and the relationship may be obtained by preliminarily obtaining the lens curve. Or pre-measure the projection radius of the incident light at different angles on the image sensor, and obtain it in advance.
舉例來說，圖3A是依照本發明一實施例所繪示之物體在魚眼鏡頭的投影半徑與入射角的關係示意圖，圖3B是依照本發明一實施例所繪示之物體在魚眼影像中的位置與方位角的關係示意圖。請先參照圖3A，由物體T所發出的光例如是以入射角θ 入射魚眼鏡頭32，並經由魚眼鏡頭32的折射，在影像感測器34所在的影像感測器平面上與魚眼中心O的距離為投影平徑r的位置上成像。其中，魚眼鏡頭32在影像感測器平面上的有效投影平徑為R。根據魚眼鏡頭32的鏡頭曲線，前述的入射角θ 與投影平徑r的關係例如為，其中k 為常數且可預先測得。藉此，若使用此魚眼鏡頭32拍攝任意物體，即可依據該物體在所拍攝魚眼影像中的位置，利用上述關係反推由此物體發光的入射角θ 。另一方面，請參照圖3B，依據物體T’在魚眼影像36中的位置，即可以貫穿魚眼影像36的魚眼中心O的x軸為基準，計算出物體T’相對於魚眼中心O的方位角φ 。For example, FIG. 3A is a schematic diagram showing the relationship between the projection radius of the object and the incident angle of the fisheye lens according to an embodiment of the invention, and FIG. 3B is a view of the object in the fisheye according to an embodiment of the invention. Schematic diagram of the relationship between position and azimuth. Referring first to FIG. 3A, the light emitted by the object T is incident on the fisheye lens 32, for example, at an incident angle θ , and is refracted by the fisheye lens 32 on the image sensor plane where the image sensor 34 is located. The distance from the center of the eye O is imaged at a position where the projection is flat. The effective projection plane diameter of the fisheye lens 32 on the image sensor plane is R. According to the lens curve of the fisheye lens 32, the relationship between the aforementioned incident angle θ and the projection flat diameter r is, for example, , where k is a constant and can be measured in advance. Therefore, if any object is photographed using the fisheye lens 32, the incident angle θ of the light emitted by the object can be reversed by the above relationship according to the position of the object in the fisheye image. On the other hand, referring to FIG. 3B, the object T' is calculated relative to the center of the fish eye based on the position of the object T' in the fisheye image 36, that is, the x-axis of the fisheye center O of the fisheye image 36. The azimuth angle of O is φ .
此外，處理器16執行距離計算模組145，以根據物體在魚眼影像中朝向魚眼中心的方向上的寬度，計算物體與魚眼鏡頭之間的距離（步驟S210）。詳言之，如前所述，魚眼鏡頭122所拍攝的物體，會依其偏離魚眼中心的距離不同而產生不同程度的形變（例如圓形的光球會變成橢圓形），偏離魚眼中心的距離愈遠，形變的程度也愈嚴重。本發明實施例即依據此特性，根據魚眼影像中所攝物體形變後的寬度，推算該物體與魚眼鏡頭122之間的距離。Further, the processor 16 executes the distance calculation module 145 to calculate the distance between the object and the fisheye lens according to the width of the object in the direction of the fisheye image toward the center of the fisheye (step S210). In detail, as mentioned above, the object photographed by the fisheye lens 122 will be deformed to different degrees depending on the distance from the center of the fisheye (for example, a round photosphere will become an ellipse), which deviates from the fisheye. The farther the distance is from the center, the more severe the deformation. According to this characteristic, the distance between the object and the fisheye lens 122 is estimated according to the width of the object in the fisheye image.
舉例來說，圖4A及圖4B是依照本發明一實施例所繪示之所攝物體透過魚眼鏡頭成像的範例。請參照圖4A，由物體T發出的光在經由魚眼鏡頭42折射後，會在影像感測器44上成像。其中，假設物體T的原始寬度為W，經由魚眼鏡頭42折射，在影像感測器44上成像的寬度將變成w。圖4B即繪示由影像感測器44所擷取的物體T的魚眼影像40，其中物體T’即是物體T投射在影像感測器44上的成像，而由於魚眼鏡頭42的特性是，偏離魚眼中心O的距離愈遠，物體形變的程度也愈高，因此物體T’的形狀已變成圖中所示的橢圓形，且物體T’在魚眼影像40中朝向魚眼中心O的方向上的寬度w可用以計算物體T與魚眼鏡頭42之間的距離。For example, FIG. 4A and FIG. 4B illustrate an example of imaging an object through a fisheye lens according to an embodiment of the invention. Referring to FIG. 4A, the light emitted by the object T is imaged on the image sensor 44 after being refracted by the fisheye lens 42. Here, assuming that the original width T of the object T is W, refracted by the fisheye lens 42, the width imaged on the image sensor 44 will become w. 4B is a fisheye image 40 of the object T captured by the image sensor 44, wherein the object T' is an image of the object T projected on the image sensor 44, and due to the characteristics of the fisheye lens 42. Yes, the farther away from the center O of the fisheye, the higher the degree of deformation of the object, so the shape of the object T' has become an ellipse as shown in the figure, and the object T' is oriented toward the center of the fisheye in the fisheye image 40. The width w in the direction of O can be used to calculate the distance between the object T and the fisheye lens 42.
詳言之，在一實施例中，假設物體在魚眼影像中的的寬度為wT 、物體的實際寬度為WT 、魚眼影像的影像寬度為WI 、魚眼鏡頭的視野（Field of View，FOV）為F ，則物體與魚眼鏡頭之間的距離r 為： (1)In detail, in one embodiment, it is assumed that the width of the object in the fisheye image is w T , the actual width of the object is W T , the image width of the fisheye image is W I , and the field of view of the fisheye lens (Field of View, FOV) is F , then the distance r between the object and the fisheye lens is: (1)
舉例來說，表1列示利用魚眼鏡頭拍攝放置在不同距離的物體的魚眼影像時，該物體在魚眼影像中的寬度（像素）與距離的關係。其中，物體1為直徑1公分的光球、物體2為直徑5公分的光球，魚眼鏡頭所拍攝的魚眼影像的影像寬度為864像素，且魚眼鏡頭的視野為187度。
以拍攝距離100公分的物體2為例，此物體2在所拍攝魚眼影像中的寬度為13.23624425像素。將上述參數導入公式(1)後，可算出物體2與魚眼鏡頭之間的距離r為：Taking the object 2 of a distance of 100 cm as an example, the width of the object 2 in the fisheye image is 13.23624425 pixels. After introducing the above parameters into the formula (1), the distance r between the object 2 and the fisheye lens can be calculated as:
在計算出物體的方位角、入射角以及與魚眼鏡頭的距離之後，處理器16即執行座標計算模組146，而根據前述的方位角、入射角以及距離，利用三角函數計算物體的三維座標（步驟S212）。After calculating the azimuth angle of the object, the incident angle, and the distance from the fisheye lens, the processor 16 executes the coordinate calculation module 146, and calculates the three-dimensional coordinates of the object by using a trigonometric function according to the azimuth angle, the incident angle, and the distance. (Step S212).
詳言之，圖5是依照本發明一實施例所繪示之計算物體三維座標的示意圖。請參照圖5，假設O是魚眼鏡頭的魚眼中心，P點是物體T在鏡頭平面上的投影點，物體T與魚眼中心O之間的距離為r 。其中，線段為線段在鏡頭平面上的投影線，線段與線段之間的夾角θ 可視為物體T的發光入射於魚眼鏡頭的入射角；線段為線段在水平軸線上的投影線，而線段與線段之間的夾角φ 則可視為物體T相對於魚眼中心O的方位角。In detail, FIG. 5 is a schematic diagram of calculating a three-dimensional coordinate of an object according to an embodiment of the invention. Referring to FIG. 5, it is assumed that O is the fisheye center of the fisheye lens, P is the projection point of the object T on the lens plane, and the distance between the object T and the fisheye center O is r . Among them, the line segment Line segment Projection line on the plane of the lens, line segment With line segments The angle θ between can be regarded as the incident angle of the incident light of the object T on the fisheye lens; the line segment Line segment Projection line on the horizontal axis, while the line segment With line segments The angle φ between them can be regarded as the azimuth of the object T with respect to the center O of the fisheye.
基於前述的方位角φ 、入射角θ 以及距離r ，物體T的三維座標為(x ,y ,z )可利用下列公式推得：(2)(3)(4)Based on the azimuth angle φ , the incident angle θ, and the distance r described above, the three-dimensional coordinates of the object T are ( x , y , z ) which can be derived by the following formula: (2) (3) (4)
藉由上述方法，本實施例的影像擷取裝置10即可在不進行魚眼校正的情況下，實現所攝物體的三維座標計算，且其所拍攝範圍相較於傳統鏡頭更廣。By the above method, the image capturing device 10 of the present embodiment can realize the three-dimensional coordinate calculation of the object to be taken without performing fisheye correction, and the shooting range thereof is wider than that of the conventional lens.
綜上所述，本發明的利用魚眼影像計算所攝物體座標的方法及影像擷取裝置依據物體在雙魚眼鏡頭所拍攝影像中的位置，分別計算出物體發光進入魚眼鏡頭的入射角以及物體在鏡頭平面上的方位角，並根據物體在魚眼影像中的寬度估算物體與魚眼鏡頭的距離，最終利用三角函數計算出物體於空間中的三維座標。藉此，本發明實施例可在不進行幾何轉換的情況下，實現所攝物體的三維座標計算，且可在增加偵測範圍的同時，減少偵測物體所需的運算量。In summary, the method for calculating the coordinates of the object by using the fisheye image and the image capturing device respectively calculate the incident angle of the object into the fisheye lens according to the position of the object in the image taken by the fisheye lens and The azimuth of the object on the plane of the lens, and the distance between the object and the fisheye lens is estimated according to the width of the object in the fisheye image. Finally, the trigonometric function is used to calculate the three-dimensional coordinates of the object in space. Thereby, the embodiment of the invention can realize the three-dimensional coordinate calculation of the object to be taken without geometric conversion, and can reduce the calculation amount required for detecting the object while increasing the detection range.
雖然本發明已以實施例揭露如上，然其並非用以限定本發明，任何所屬技術領域中具有通常知識者，在不脫離本發明的精神和範圍內，當可作些許的更動與潤飾，故本發明的保護範圍當視後附的申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.
10‧‧‧影像擷取裝置10‧‧‧Image capture device
12‧‧‧魚眼鏡頭12‧‧‧ fisheye lens
14‧‧‧儲存裝置14‧‧‧Storage device
141‧‧‧影像擷取模組141‧‧‧Image capture module
142‧‧‧物體偵測模組142‧‧‧ Object Detection Module
143‧‧‧方位角計算模組143‧‧‧Azimuth calculation module
144‧‧‧入射角計算模組144‧‧‧incident angle calculation module
145‧‧‧距離計算模組145‧‧‧ Distance calculation module
146‧‧‧座標計算模組146‧‧‧ coordinate calculation module
16‧‧‧處理器16‧‧‧ Processor
32、42‧‧‧魚眼鏡頭32, 42‧‧‧ fisheye lens
34、44‧‧‧影像感測器34, 44‧‧‧ image sensor
36、40‧‧‧魚眼影像36, 40‧‧‧ fisheye images
W‧‧‧物體T的原始寬度W‧‧‧ Original width of object T
w‧‧‧物體T’的寬度w‧‧‧The width of the object T’
θ‧‧‧入射角 θ ‧‧‧ incident angle
φ‧‧‧方位角 φ ‧‧‧ azimuth
O‧‧‧魚眼中心O‧‧‧ Fisheye Center
P‧‧‧投影點P‧‧‧projection point
R‧‧‧有效投影平徑R‧‧‧effective projection flat diameter
r‧‧‧投影平徑R‧‧‧projection path
T‧‧‧物體T‧‧‧ objects
T’‧‧‧魚眼影像中的物體Objects in the fisheye image of T’‧‧‧
M‧‧‧P點在水平軸線上的投影點Projection point of M‧‧‧P point on the horizontal axis
S202~S212‧‧‧本發明一實施例之利用魚眼影像計算所攝物體座標的方法步驟S202~S212‧‧‧ method steps for calculating the coordinates of the object taken by using the fisheye image according to an embodiment of the present invention
圖1是依照本發明一實施例所繪示的影像擷取裝置的方塊圖。 圖2是依照本發明一實施例所繪示的利用魚眼影像計算所攝物體座標的方法的流程圖。 圖3A是依照本發明一實施例所繪示之物體在魚眼鏡頭的投影半徑與入射角的關係示意圖。 圖3B是依照本發明一實施例所繪示之物體在魚眼影像中的位置與方位角的關係示意圖。 圖4是依照本發明一實施例所繪示之計算物體三維座標的示意圖。FIG. 1 is a block diagram of an image capture device according to an embodiment of the invention. 2 is a flow chart of a method for calculating a coordinate of a captured object using a fisheye image according to an embodiment of the invention. FIG. 3A is a schematic diagram showing the relationship between the projection radius of the object and the incident angle of the fisheye lens according to an embodiment of the invention. FIG. 3B is a schematic diagram showing the relationship between the position and the azimuth angle of an object in a fisheye image according to an embodiment of the invention. 4 is a schematic diagram of calculating a three-dimensional coordinate of an object according to an embodiment of the invention.
Claims (10)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW106136425A TWI646506B (en) | 2017-10-24 | 2017-10-24 | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images |
| US16/164,810 US10762658B2 (en) | 2017-10-24 | 2018-10-19 | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW106136425A TWI646506B (en) | 2017-10-24 | 2017-10-24 | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TWI646506B TWI646506B (en) | 2019-01-01 |
| TW201917697A true TW201917697A (en) | 2019-05-01 |
Family
ID=65803635
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW106136425A TWI646506B (en) | 2017-10-24 | 2017-10-24 | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI646506B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11004232B2 (en) | 2019-07-12 | 2021-05-11 | Mediatek Inc. | Method and apparatus for performing object detection according to bounding box distribution that is determined by considering lens configuration |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5796426A (en) * | 1994-05-27 | 1998-08-18 | Warp, Ltd. | Wide-angle image dewarping method and apparatus |
| TW594453B (en) * | 2002-07-26 | 2004-06-21 | Gwo-Jen Jan | Method for presenting fisheye-camera images |
| TW200528945A (en) * | 2004-01-20 | 2005-09-01 | Chuang-Jan Chang | 3D visual measurement system using fish-eye cameras as visual detectors and method for constructing same |
| TWI423659B (en) * | 2010-11-09 | 2014-01-11 | Avisonic Technology Corp | Image corretion method and related image corretion system thereof |
| TW201239807A (en) * | 2011-03-24 | 2012-10-01 | Hon Hai Prec Ind Co Ltd | Image capture device and method for monitoring specified scene using the image capture device |
| CN103839227B (en) * | 2012-11-23 | 2017-06-09 | 富士通株式会社 | Fisheye image correcting method and device |
| CN104240236B (en) * | 2014-08-26 | 2017-10-24 | 中山大学 | A kind of method of correcting fisheye image after fish eye lens demarcation |
-
2017
- 2017-10-24 TW TW106136425A patent/TWI646506B/en active
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11004232B2 (en) | 2019-07-12 | 2021-05-11 | Mediatek Inc. | Method and apparatus for performing object detection according to bounding box distribution that is determined by considering lens configuration |
| TWI738406B (en) * | 2019-07-12 | 2021-09-01 | 聯發科技股份有限公司 | Method and apparatus for object detection |
Also Published As
| Publication number | Publication date |
|---|---|
| TWI646506B (en) | 2019-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101657039B1 (en) | Image processing apparatus, image processing method, and imaging system | |
| US20170330337A1 (en) | Image processing device, image processing method, and recording medium storing program | |
| US20140071227A1 (en) | Image processor, image processing method and program, and imaging system | |
| JP6556013B2 (en) | PROCESSING DEVICE, PROCESSING SYSTEM, IMAGING DEVICE, PROCESSING METHOD, PROGRAM, AND RECORDING MEDIUM | |
| US20150292884A1 (en) | Motion-sensor device having multiple light sources | |
| WO2016184131A1 (en) | Image photographing method and apparatus based on dual cameras and computer storage medium | |
| TWI599809B (en) | Lens module array, image sensing device and fusing method for digital zoomed images | |
| JPWO2014174779A1 (en) | Motion sensor device having a plurality of light sources | |
| US20140152772A1 (en) | Methods to combine radiation-based temperature sensor and inertial sensor and/or camera output in a handheld/mobile device | |
| US20170155889A1 (en) | Image capturing device, depth information generation method and auto-calibration method thereof | |
| WO2016029465A1 (en) | Image processing method and apparatus and electronic device | |
| TWI595444B (en) | Image capturing device, depth information generation method and auto-calibration method thereof | |
| JP2010276433A (en) | Imaging device, image processor, and distance measuring device | |
| US20190122379A1 (en) | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images | |
| TWI646506B (en) | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images | |
| TWI725279B (en) | Method and image pick-up apparatus for calculating coordinates of object being captured using dual fisheye images | |
| TWI635256B (en) | Phase detection auto-focus-based positioning method and system thereof | |
| JP2019009643A (en) | Image processing apparatus, image processing method and program | |
| CN110021044B (en) | Method for calculating coordinates of shot object by using double-fisheye image and image acquisition device | |
| US9743007B2 (en) | Lens module array, image sensing device and fusing method for digital zoomed images | |
| TWI693577B (en) | Image generating method and electronic apparatus | |
| TW202001792A (en) | A method for detecting motion in a video sequence | |
| CN109696122A (en) | The method and video capturing device of taken the photograph object coordinates are calculated using flake image | |
| TWI576648B (en) | Image capturing apparatus and image capturing method | |
| TWI672950B (en) | Image device capable of compensating image variation |