201210342 ^ 六、發明說明： .【發明所屬之技術領域】 本發明係關於一種多攝影機平順場景切換之方法，特別為 一種利用人員追蹤之資訊，透過影像形變視訊轉換技術，達 成影像監控系統在不同攝影機間，鏡頭畫面平順切換的顯示 效果。 【先前技術】 近年來，多重攝影機視訊系統廣泛受到研究與推廣，大多 • 應用於影像監控系統與運動球赛轉播上；傳統的系統，前端 之多攝影機的拍攝晝面以視訊串流（video streaming)或以類比 訊號傳送至控制中心，並以多分割晝面方式顯示於監控螢幕 上’再經由監控人員選擇欲播放的攝影機畫面放大觀看；然 而’此方法當監控人員需做攝影機切換時，瞬間切換監視畫 面的視覺效果會使得監控人員無法確切掌控追蹤人員之移動 方向或須以人眼費力的來切換追蹤，追蹤人員不易。本方法 透過人員追蹤資訊，提供一種即時的平順場景切換方法，藉 著將多台不同方向攝影機所拍攝的監控影像，利用影像形變 _ .的視訊轉換技術，達成不同攝影機間鏡頭平順切換的顯示效 果。 另外也有人提出自動偵測與平滑換景的方式，如論文 “Real-Time View Transition between Cameras Having Different Perspectives55, L.W. Chang, K.W. Chen, W.T. Peng, Y.P. Hung,19th IPPR Conference on Computer Vision, Graphics, and Image Processing, August 2006.中，作者提出平順場景切換之 方法，其流程為將兩台真實攝影機拍攝到的背景影像當作材 質，貼於事先建立好的三維模型以建立不同角度之背景虛擬 影像，透過找出真實攝影機影像中人物的二維影像位置，製 201210342 ;一個透明度遮罩，利用此遮翠得到前景人物影像，接下來， ::人物相對於背景三維模型的位置，將前景人物影 背景虛擬影像合成，隨著前景人物的移動呈現平 =切換之顯示效果。此方法主要為合成前景人物影像和 责景虛擬影像之技術，因其冰 〜旦s A… 事先建立之背景虛擬影像， 環㈣Μ在因素而改變時，該方法所呈現 ==場景影像會有所不同，導致監控人員無法掌握： m 訊’且其方法加人陰難理技術以擷取前景 =二r的變化必須調整間值參數才會得到較好 蚕= 的環境；而本方法未分成前景人物影 場旦的二、影像做處理，採用即時的影像做兩台攝影機間 可呈現最新實際場景的影像，且計算複雜 度較低，可適用於任何即時系統。 計，=二習用方式仍有諸多缺失，實非-良善之設 加二::人::==^生，缺點，乃亟思 發完成本❹攝影機平順場景^之;;=後’終於成功研 【發明目的】 方法本ΓΓ目：:在於提供一種多攝影機平順場景切換之 追蹤皆: 之影像形變視訊轉換技術，透過人員 【發明内容】 係利 達成上述發明目的之多攝影機平順場景切換之方法， 201210342 用人員追蹤資訊，藉著將多台且兩兩不同方向攝影機所拍攝 -的監控料’透過影像形變的視訊轉換技術，達成兩兩攝影 機間鏡頭平順切換的顯示效果。在某一場所或樓層架設多’： 攝影機，這些攝影機所拍攝的晝面兩兩涵蓋部份的區域^ 圍，而攝影機所拍攝的原始影像串流傳回到中心端影像擷取 設備，傳送的方式可以透過有線或無線的傳輸設備，若攝影 機使用的是廣角攝影機，利用影像校正模組（參考文獻〇，將 原本因攝影機廣角鏡頭而變形的影像，校正回真實的線條長 度與比例。平順場景切換區域分為直線型與穿牆型平順場景 •切換區域兩種，並事先規劃這兩種切換區域之攝影機間人員 進出之動線關連與平順場景切換之區域線，當人員進入此區 ，線後，程式自動執行人員追蹤，其方法為透過目前影像與 高斯背景模型建立模組（參考文獻2)相減或相除而得到精確的 ^員前景位置，高斯背景模型建立模組係針對攝影機畫面中 母個像素點的像素值在一段時間内的變化做統計分析，以判 斷每個像素點為前景或背景，由於會隨著時間的改變而不斷 更新統計結果，因此能適應環境的變化，再以多重人員追蹤 _才莫組紀錄晝面中每一人員的行經路線。當人員進入平順場景 7換區域時，透過平順場景切換模組運算，若為直線型平順 場景切換區域，使用透視變換（Hom〇graphy)技術（參考文獻 3)，藉由兩台攝影機畫面中相同的特徵點（至少四個）求得形變 矩陣，透視變換（Homography)是一種數學上的觀念，它定義 一種兩張影像間之關係性，在任一張影像上的一點，可對應 至另一張影像上的一點，經由計算完透視變換（H〇m〇graphy) 形變矩陣後，即可將任一張影像像素位置轉換成另一張影像 的像素位置’而將兩台攝影機的影像連接，以人員追蹤資訊 將人員顯示於畫面中心輸出連接影像；若為穿牆型平順場景 切換區域，透過兩台攝影機相同特徵點的資訊如位置、長度， E S3 5 201210342 做為兩台攝影機畫面正規化的參數計算，包含了位移和垂 直杯水小縮放的參數，以垂直、水平大小縮放參數加上 内插决算法計算’將兩台攝影機畫面影像轉換到相同的場景 離2=算利用相鄰四點的像素值’依照與相鄰座 根據距離比例求得取樣值，最後以位移參數 面影像連接’且透過人員追縱資訊將人員顯 ' 。輸出連接影像。經由此影像形變的視訊鐮拖姑 術，系統即時運算且替益鹿m 變的視訊轉換技 算且螢幕顯不攝影機間平順場景切換之畫面。 【實施方式】 請參閱圖一及圖二戶斤+ ^ 切換環境架設示意圖，圖：:穿一多：影機平順場景 環境牟机干音阁Λ 馬穿牆4•多攝影機平順場景切換 衣境I又不意圖。假設在某一場所或樓層裝設兩 分別是一號攝影機和-號摄影嬙 ° ~機 景切換之區域(如圖一: = 兩，刚平順場 後，即自動進行人員追縱：人昌!域)，當人員進入此場所 員追蹤當人員進入1號區域時，罄葸6 動顯示二號攝影機所拍攝到的畫面21 、螢幕自 時’螢幕顯示兩台攝影機間平順場田二η ，域時，榮幕則顯示一號攝影機所拍二書進入3 法並可在某一場所中串接延伸多台攝影 ：： 兩兩間之平順場景切換。 規_〗夕σ攝影機 圖三為本發明之實施流程圖，首先，一 拍攝到的原始影像串流31傳送到影像處二=機所 :的是廣角攝影機，透過影像校正模组右:機使 Cahbration)，將原本因攝影機廣角鏡頭而變形的（’era 真實二始： 影像的線條長度與比例如昭片-， 真實衫像的線條長度是以像t (Pixel 二片- 界的長度（以公分計算）做# 可以和真實世 算）做等比例的轉換。將影像利用影像處理 201210342 技術中的背景相減法，透過影像與所建立的高斯背景模型模 組33相減或相除可得到精確的移動人員前景影像。經過背景 相減法處理之每個前景目標人員以—長方形標定，長方形的 置即是目標人貝在攝影機畫面上的前景位置，透過此 ^訊’利用人員追縱模組34即可記錄每—人員在攝影機 畫面中的行經路線。 透過事先將場所分成的三個區域做為攝影機切換的觸 2分別是-號攝影機拍攝區域、二號攝影機拍攝區域和兩 口攝影機間平順切換區域’由人員追蹤模組34產 資訊後，當人員進人-號攝影機拍攝區域時，螢 ==所：攝到的畫面；當人員進入二號攝影機拍攝區 域時，榮幕即時顯示二號攝影機所拍攝到的畫面；t 入，台攝影機間平順切換區域時，經由平順場景切換模组35 運异’螢幕即時顯示兩台攝影機間平順切換畫面。平順 :換模組35中’分為直線型與穿牆型平順場景切換模組兩 :’直線型平順場景切換模組主要是透過透視變換 H_graphy)技術，藉由兩台攝影機畫面中相㈣特徵點（至201210342 ^ VI. Description of the invention: [Technical field of the invention] The present invention relates to a method for switching a multi-camera smooth scene, in particular to use a person tracking information to achieve an image monitoring system through image deformation video conversion technology. Between the cameras, the lens screen is smoothly switched to display. [Prior Art] In recent years, multi-camera video systems have been widely researched and promoted, most of which are used in video surveillance systems and sports ball broadcasts; traditional systems, front-end multi-cameras are captured by video streaming. Or send it to the control center with analog signal, and display it on the monitor screen in multiple partitions. Then select the camera screen to be played by the monitor to zoom in and see; however, this method is used when the monitor needs to switch the camera and switch instantly. The visual effect of the surveillance screen will make it impossible for the monitor to control the direction of the tracking person or to switch the tracking with the eye and the eye. The method provides an instant smooth scene switching method through the personnel tracking information, and realizes the smooth switching effect of different cameras by using the image conversion technology of the image deformation _ by the plurality of cameras in different directions. . In addition, some people have proposed automatic detection and smoothing of the scene, such as the paper "Real-Time View Transition between Cameras Having Different Perspectives 55, LW Chang, KW Chen, WT Peng, YP Hung, 19th IPPR Conference on Computer Vision, Graphics, and In Image Processing, August 2006., the author proposes a method for smoothing scene switching. The process is to use the background image captured by two real cameras as a material and paste it on a previously established 3D model to create a background virtual image at different angles. By finding the 2D image position of the person in the real camera image, 201210342; a transparency mask, using this occultation to get the foreground character image, then, :: the position of the character relative to the background 3D model, the foreground character shadow background The virtual image is synthesized, and the display effect of the foreground character is changed with the movement of the foreground character. This method is mainly for the technique of synthesizing the foreground character image and the scene virtual image, because the ice is s A... The background virtual image established in advance, the ring (4) When the factor changes, the method presents == scene image It will be different, so that the monitoring personnel can not grasp: m News 'and its method adds a sinister technique to capture the foreground = two r changes must adjust the inter-value parameter to get a better silkworm = environment; and this method The image is not processed into the foreground image. The image is processed by using the instant image to display the latest actual scene between the two cameras, and the calculation complexity is low, which can be applied to any real-time system. There are still many shortcomings in the way, the real is not - the good set plus two:: people::==^sheng, shortcomings, is the thought of the completion of this camera smooth scene ^;; = after 'finally succeeded research [invention purpose] Method: This is to provide a multi-camera smooth scene switching tracking: Image deformation video conversion technology, through the personnel [invention content] To Lee to achieve the above-mentioned invention of the multi-camera smooth scene switching method, 201210342 staff tracking Information, through the monitoring of multiple cameras in two different directions - the video conversion technology through image deformation, to achieve a smooth lens between the two cameras The display effect is changed. In a certain place or floor, there are many ': cameras, the two sides of the camera are covered by the camera, and the original image captured by the camera is transmitted back to the center. The device can be transmitted through a wired or wireless transmission device. If the camera uses a wide-angle camera, use the image correction module (Reference 〇 to correct the image that was originally deformed by the camera wide-angle lens to the true line length and scale. . The smooth scene switching area is divided into two types: linear type and through-wall type smooth scenes and switching areas, and the area lines of the moving line related and smooth scene switching between the camera in and out of the two switching areas are planned in advance, when the personnel enters this area. After the line, the program automatically performs personnel tracking by subtracting or dividing the current image from the Gaussian background model (Reference 2) to obtain an accurate foreground position. The Gaussian background model is built for the module. The pixel value of the parent pixel in the camera screen is statistically analyzed over a period of time to determine whether each pixel is a foreground or a background. Since the statistical result is continuously updated as time changes, it can adapt to changes in the environment. Then, the multi-person tracking _ _ _ group records the course of each person in the face. When the person enters the smooth scene 7 change area, the module is switched by the smooth scene, and if it is a straight-line smooth scene switching area, the perspective transform (Hom〇graphy) technique (Reference 3) is used, and the same is used in the two camera screens. The feature points (at least four) are used to obtain the deformation matrix. Homography is a mathematical concept that defines the relationship between two images. One point on any image can correspond to another. A point on the image, after calculating the transformation matrix (H〇m〇graphy) deformation matrix, you can convert any image pixel position into the pixel position of another image and connect the images of the two cameras to Personnel tracking information will be displayed on the center of the screen to output the connected image; if it is a smooth-type scene switching area, the information of the same feature points of the two cameras, such as position and length, E S3 5 201210342 is normalized as two camera pictures. Parameter calculation, including parameters for displacement and vertical cup water scaling, scaling parameters in vertical and horizontal sizes plus interpolation algorithm Calculate 'convert two camera screen images to the same scene from 2=calculate the pixel values of the adjacent four points' according to the ratio of distance from the adjacent seat according to the distance, and finally connect with the displacement parameter surface image' and pass the personnel Tracking information will show people'. Output connection image. Through the video image deformation of this image, the system instantly calculates the video conversion technology for the benefit deer and changes the screen to the smooth transition between the cameras. [Embodiment] Please refer to Figure 1 and Figure 2 for the jin + ^ switch environment erection diagram, picture:: wear more than one: the camera smooth scene environment 牟 machine dry sound cabinet Λ horse through the wall 4 • multi-camera smooth scene switch clothing I Not intended. Assume that two cameras and a number of cameras are installed in a certain place or floor. The area of the camera is switched to the area (Figure 1: 1: Two, just after the field is finished, the personnel are automatically chased: Man Chang! Field), when the person enters the venue to track when the person enters the No. 1 area, the 罄葸6 moves to display the picture 21 taken by the second camera, and the screen shows the screen between the two cameras. At the time, the screen shows that the two books taken by the No. 1 camera entered the 3 method and can be connected in series to extend multiple photography:: The smooth scene switching between the two. Fig. 3 shows the flow chart of the implementation of the present invention. First, a captured original image stream 31 is transmitted to the image station. The camera is rotated by a wide angle camera. Cahbration), which was originally deformed by the wide-angle lens of the camera ('era's true two-start: the length of the line of the image and the ratio of the line-like image, for example, the length of the line of the real shirt is like the length of the t-Pixel - the length of the circle (in centimeters) Calculate) Do # can be converted to the real world. The image can be subtracted or divided by the background Gaussian background model 33 by using the background subtraction method in the image processing 201210342 technique. Moving the foreground image of the person. Each foreground target person processed by the background subtraction method is calibrated by a rectangle, and the rectangular position is the foreground position of the target person on the camera screen, and the user tracking module 34 is It can record the route of each person in the camera screen. The three regions that are divided into the camera in advance are used as the camera switch. The shooting area, the second camera shooting area, and the two camera smooth switching areas. After the personnel tracking module 34 produces information, when the person enters the camera area of the camera, the firefly ==: the captured picture; when the person enters When the No. 2 camera captures the area, the screen immediately shows the picture captured by the No. 2 camera; when the camera is in the smooth switching area between the cameras, the screen is switched between the two cameras via the smooth scene switching module 35. Picture: Smooth: In the replacement module 35, there are two types of linear and wall-type smooth scene switching modules: 'Linear smooth scene switching module is mainly through perspective transformation H_graphy) technology, with two cameras in the picture phase (4) Feature points (to
Hi固ί徵點）計算兩台攝影機㈣變料，利用此矩陣將兩 口攝衫機的影像連接（image stiiching)，且加入影像混合方法 (^unage blendlng)(參考文獻4)，使得影像做連接時不會受到产 擾’畫面顯示更為自然。而穿牆型平順場景切換模： 主要疋利用找出兩台攝影機畫面中相同的特徵點，二 中門框的上下兩個端點，透過特徵點的資訊如位置、長、 做為兩台攝影機畫面正規化的參數計算，包含了位移和^ 參Ϊ於門框的兩端點資訊僅能做垂直大小縮 ° 會使传畫面有拉長的視覺效果，在此運用其垂 直大小縮放的比例做水平大小縮放，確保縮放的—致性，、以 垂直、水平大小縮放參數利用内插演算法計算，將兩台攝影 201210342 機畫面影像轉換到相同的埸熹 播畲^伯、击社」 例，以位移參數將兩台攝影 機畫面如像連接。兩台攝影機間 縱模組34產生之人員追縱資訊，：;=畫：:利用人員追 時透過平順場景切換模組建立左右場景畫面。—巾即 ，照片四為本發明多攝影機平順場景切換方法之 影機平順場景切換方法執行結果。 、·’ 攝 照片五為本發明多攝影機平順場景切換方法之 攝影機平順場景切換方法執行結果。 ®夕 【特點及功效】 本發明所提供之多攝影機平順場景切換 用技術相互比較時，更具備下列優點： ，、其他、 炎播pi發月透過#像形變的視訊轉換技術，達成攝影機和攝 =順切換之效果，提供給監控人員更清楚且更方便的 2=發'可降低人事成本，監控人員可輕易的了解被監控 金而她^境中的係’不需以人卫做額外的攝影機 畫面切換。 〜 3.本發明提供一簡易且運算複雜度低之影像形變轉換技 術，可適用於任何即時系統。 、 上歹丨詳細說明乃針對本發明之一可行實施例進行具體說 明，惟該實施例並非用以限制本發明之專利範圍，凡未脫離 本發明技藝精神所為之等效實施或變更，均應包含於本案之 專利範圍中。 〃 ’’示上所述，本案不僅於技術思想上確屬創新，並具備習用 之傳統方法所不及之上述多項功效，已充分符合新穎性及進 步性之法定發明專利要件，爰依法提出申請，懇請貴局核 准本件發明專利申請案，以勵發明，至感德便。 201210342 【圖式簡單說明】 .請參閱有關本發明之詳細說明、附圖及其照片，將可進一 步瞭解本發明之技術内容及其目的功效；有關附圖及照片為： 圖一為直線型多攝影機平順場景切換環境架設示意圖； 圖二為穿牆型多攝影機平順場景切換環境架設示意圖； 圖三為多攝影機平順場景切換之實施流程圖。 【參考文獻】 Z. S. Wang, Z. G. Wang and X. H. Xu, κExtraction of the Corner of Checkerboard image，” World Congress on Intelligent Control and Automation, china, pp. 6789-6792, 2008. D.S. Lee. “Effective Gaussian Mixture Learning for Video Background Subtraction，” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 827-832, 2005. R. Hartley, A. Zisserman. “Multiple view geometry, Cambridge University Press, 1 edition, 2000. M. Brown and D. G. Lowe, “Recognising Panoramas,” Proceedings of International Conference on Computer Vision, Nice, France, Vol. 2, pp. 1218-1225, 2003. 【附件】 畫面21為二號攝影機所拍攝到的畫面； 畫面23為一號攝影機所拍攝到的畫面； 照片一為廣角鏡頭校正前的影像； 照片二為廣角鏡頭校正後的影像； 照片三穿牆型平順場景切換模組實施方法； 照片四直線型多攝影機平順場景切換方法執行結果； 201210342 照片五穿牆型多攝影機平順場景切換方法執行結果。 【主要元件符號說明】 31 原始影像串流 32 影像校正模組 33 高斯背景模型建立模組 34 人員追縱模組 平順場景切換模組 35Hi ί ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) The screen will not be affected by the production disturbance. The wall-type smooth scene switching mode: Mainly uses to find the same feature points in the two camera screens, the upper and lower ends of the two middle door frames, through the feature point information such as position, length, as two camera screens The normalized parameter calculation includes the displacement and ^. The information at both ends of the door frame can only be used to reduce the vertical size. The image will have a long visual effect. Scaling, ensuring the scaling, and scaling the parameters in vertical and horizontal sizes using the interpolation algorithm to convert the two images of the 201210342 screen image to the same 畲 畲 伯 、 击 击 击 击 击The parameters connect the two camera screens like images. The information generated by the vertical module 34 between the two cameras is: •==Picture:: Use the personnel to chase and establish the left and right scenes through the smooth scene switching module. The towel is the execution result of the camera smoothing scene switching method of the multi-camera smooth scene switching method of the present invention. ,·· Photograph 5 is the execution result of the camera smoothing scene switching method of the multi-camera smooth scene switching method of the present invention. ® 夕 [Features and Efficacy] The multi-camera smooth scene switching technology provided by the present invention has the following advantages when compared with each other: ,, other, inflammatory broadcasts, through the image-changing video conversion technology, to achieve the camera and the camera = The effect of switching, providing the monitoring personnel with a clearer and more convenient 2 = send 'can reduce personnel costs, the monitoring staff can easily understand the monitored gold and her system in the environment 'do not need to do additional Camera screen switching. ~ 3. The present invention provides a simple and computationally intensive image deformation conversion technique that can be applied to any real-time system. The detailed description of the present invention is intended to be illustrative of a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. It is included in the patent scope of this case. 〃 ''Showing that the case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progress, and applied for it according to law. I urge you to approve the application for this invention patent to encourage the invention. 201210342 [Simplified description of the drawings] Please refer to the detailed description, the drawings and the photographs of the present invention, and the technical contents of the present invention and the effects thereof can be further understood; the related drawings and photographs are as follows: Schematic diagram of the environment setting of the camera smoothing scene switching; Figure 2 is a schematic diagram of the environment setting of the smoothing scene switching of the multi-camera through the wall; Figure 3 is a flow chart of the implementation of the smoothing of the multi-camera smooth scene. [References] ZS Wang, ZG Wang and XH Xu, κ Extraction of the Corner of Checkerboard image,” World Congress on Intelligent Control and Automation, china, pp. 6789-6792, 2008. DS Lee. “Effective Gaussian Mixture Learning for Video Background Subtraction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 827-832, 2005. R. Hartley, A. Zisserman. “Multiple view geometry, Cambridge University Press, 1 edition, 2000. M. Brown and DG Lowe, “Recognising Panoramas,” Proceedings of International Conference on Computer Vision, Nice, France, Vol. 2, pp. 1218-1225, 2003. [Attachment] Picture 21 is the picture taken by camera No. 2; picture 23 is the number one camera. Photograph taken; Photo 1 is the image before the wide-angle lens correction; Photo 2 is the wide-angle lens corrected image; Photo 3 through-wall type smooth scene switching module implementation method; Photo 4 linear multi-camera smooth scene switching method execution result; 201210342 Photo five wall-mounted multi-camera smooth The scene switching method execution result. [Main component symbol description] 31 Original image stream 32 Image correction module 33 Gaussian background model building module 34 Person tracking module Smoothing scene switching module 35