CN106204595A - A kind of airdrome scene three-dimensional panorama based on binocular camera monitors method - Google Patents
A kind of airdrome scene three-dimensional panorama based on binocular camera monitors method Download PDFInfo
- Publication number
- CN106204595A CN106204595A CN201610549128.2A CN201610549128A CN106204595A CN 106204595 A CN106204595 A CN 106204595A CN 201610549128 A CN201610549128 A CN 201610549128A CN 106204595 A CN106204595 A CN 106204595A
- Authority
- CN
- China
- Prior art keywords
- background
- scene
- dimensional
- camera
- airport
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/30—Polynomial surface description
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20068—Projection on vertical or horizontal image axis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Physics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Algebra (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
A kind of airdrome scene three-dimensional panorama based on binocular camera monitors method, it monitors scene by the three-dimensional overall situation airdrome scene situation installing binocular camera generation one on airdrome scene unified, makes airdrome scene controlling officer that airdrome scene situation is had more visual and clear assurance.First have to cover, with binocular camera, the scene scope needing to monitor, and these video cameras are carried out pre-demarcation, monitor according to Binocular Vision Principle and recover the three-dimensional dense point cloud that the video frame images of each binocular camera is corresponding, go to unify under three-dimensional world coordinate by these points again, merge and form panorama supervision scene, finally can be processed there being a cloud to be redeveloped into surface three dimension model, i.e. grid (Mesh) change.In implementing, this method uses some special to arrange process, to improve computational efficiency and the practical function of the present invention.
Description
Technical field
The present invention relates to computer vision technique and the video of airdrome scene (Airport Surface) video monitoring system
Monitor key technology area, be specifically related to a kind of airdrome scene three-dimensional panorama based on binocular camera and monitor method.
Background technology
The background technology of the present invention has two aspects: one, computer vision technique；Two, airport scene monitoring technology.Below
Discuss respectively.
One, computer vision technique
Computer vision is a kind of simulation using computer and relevant device to biological vision.Its main task is through
Picture or video to gathering process to obtain the three-dimensional information of corresponding scene.Figuratively, it is simply that install to computer
Upper eyes (photographing unit) and brain (algorithm), allow the computer can perception environment.Computer vision is engineering field, Ye Shi section
One in field is rich in challenge important research field.Computer vision is a comprehensive subject, including meter
Calculation machine science and engineering, signal processing, physics, applied mathematics and statistics, neuro physiology and Cognitive Science etc..
Binocular stereo vision measurement technology in the computer vision used in the present invention.Its principle is with two video cameras
Each position is to Same Scene capture, and this is referred to as binocular imaging.Now between two width images, produced parallax can be used to count
Calculate video camera and each pixel distance in world coordinates in video.One complete stereo visual system is segmented into 6
Individual step, is divided into 6 modules in other words, they respectively: 1) camera calibration, 2) video acquisition, 3) image characteristics extraction, 4)
Stereo matching, 5) 3D reconstruction, 6) post processing.The present invention is also based on this framework and uses these steps to be described.The present invention
Utilize is also above-mentioned Binocular Vision Principle, but further the result of multiple binocular vision systems is melted and collectively form
The supervision platform of one wider range.
Two, airport scene monitoring technology
Along with developing rapidly of air-transport industry, airdrome scene is required to the scene traffic automation management system of specialty.These
Control system is based primarily upon on airport the control tower set up, the information source such as scene surveillance radar, ADS-B, multipoint positioning, it is achieved pipe
Automation function processed.
That most represent at present in airport scene monitoring technology is A-SMGCS, and the motion of the most senior scene guides and controls system
System (Advanced Surface Movement Guidance and Control Systems, A-SMGCS) comprehensively have employed
Scene surveillance radar, ADS-B, the multiple scene monitoring technology of multipoint positioning, be a kind of safety that can strengthen land operations
Total system solution, is the key equipment ensureing enough traffic flow under low visibility.It achieve large airport tower
Platform airport surface traffic control function, cover departure from port aircraft transfer after completing to let pass ground, release from aircraft gate, slide, queue up, ground
Between transfer, transfer control tower, enter runway, take off these critical process, also contemplated incoming flight from land, transfer ground,
Slide, pass through runway, these processes of entrance aircraft gate.Manage by means of multi-sensor information fusion and sensor synergism and
The key technologies such as Optimization deployment, it is achieved that the fusion of the heterogeneous information such as radar, ADS-B and MDS is integrated.This technology is reserved
Carrying out, with video monitor technology, the interface that merges, an object of the present invention is also the present invention to be integrated in A-SMGCS
Go, promote the surveillance coverage of A-SMGCS, be further ensured that the safety and efficiency that airdrome scene is runed.
Summary of the invention
(two video camera overlaps monitor same based on binocular Pan/Tilt/Zoom camera to it is an object of the invention to propose and design one
One scene) airdrome scene three-dimensional panorama monitor new method.The input data of this method are: 1) be arranged on binocular on airport and take the photograph
The video of camera；2) video camera nominal data in advance.After the inventive method processes, panorama three dimension on airport can be obtained
Word model, thus realize the effect that airdrome scene is monitored from any angle position.
The object of the present invention is achieved like this:
Step one: camera calibration, demarcates in advance for each video camera in every a pair binocular camera, is imaged
Machine intrinsic parameterK i1 WithK i2 And distortion parameter.
Step 2: video acquisition, is installed to video camera airport and monitors on a position accordingly, it is thus achieved that corresponding video.Peace
During dress, the distance between two video cameras need to be measuredd i 。
Step 3: image characteristics extraction, the image that every pair of video camera is obtainedI i1 WithI i2 Calculate its corresponding characteristic point,
Computational methods use BRISK (Binary Robust Invariant Scalable Key-points), it is thus achieved that respective image is special
Levy corresponding pointI i1 WithI i2 ., and calculate corresponding fundamental matrix F (Fundamental according to the feature correspondence of two width images
Matrix), thus extrapolate the essential matrix that two width images are corresponding, thus calculate the projection matrix of two width images
(Projection Matrix)P i1 WithP i2 。
Step 4: Stereo matching, the characteristic point that step 3) obtains is sparse features point correspondence, wants dense characteristic here
The corresponding relation of point could be used for airdrome scene panorama and monitor.First have to carry out image rectification, by image polar curve (Epipolar)
Furnishing image level line, obtainsI’ i1 WithI’ i2 .Meanwhile, it is also performed to the detection of moving target, if examined after completing to initialize
Measure this pixel do not change and move, the most no longer carry out Stereo matching and follow-up three-dimensional reconstruction, but directly enter
Enter the resurfacing stage to post processing.The method that the detection of video frequency motion target uses is the matching detection of mixed Gauss model
Method.
Then forI’ i1 In a pixelI’ i1 (x, y), along this pointI’ i2 Corresponding corresponding horizontal polar curve
The pixel that upper search is most likeI’ i2 (x’, y).Here the similarity criterion used is the normalized crosscorrelation NCC of standard
(Normalized Cross Correlation), matching strategy is LRC(Left-right Checking), i.e. findI’ i2
(x’, yAfter), also will be byI’ i2 (x’, y)I’ i1 In corresponding polar curve on scan for concurrently that existing corresponding corresponding point are stillI’ i1 (x, y), just confirm that the two is corresponding match point.
Step 5: three-dimensional reconstruction, according to projection formula have [x 1 , y 1 , 1]=P i1 X j [x 2 , y 2 , 1]=P i2 X j Two public affairs
Formula can solveI i1 (x 1 , y 1 ) andI i2 (x 2 , y 2 ) this to matched pixel to common corresponding three-dimensional coordinate pointX j Value.Solve
Go out point corresponding to all pixel matching and i.e. can obtain the three-dimensional coordinate of all pixelsX,I.e. dense point cloud.
Step 6: unified some cloud coordinate, after all of video camera is to obtaining dense point cloud, should be placed on all of which
A unified global point cloud is formed under one world coordinate system.
Step 7: resurfacing based on a cloud (Surface Reconstruction), i.e. grid (Mesh) change process.
Meanwhile, the purpose of the present invention additionally uses when realizing and arranges as follows:
1) using binocular camera that airdrome scene is carried out maximized covering, between binocular camera, lap requires to be more than
65%, the shooting direction of binocular camera is close to parallel；
2) as far as possible binocular camera between there is also overlap, lap requires more than 30%；For there being the binocular of overlap to take the photograph
Camera pair, can add a boundling adjustment and optimize each camera intrinsic parameter therein and outer parameter, to improve precision；
3) if two set binocular cameras between there is no overlap, then every suit video camera is carried out in advance be registrated to unified machine
Under the world coordinate system of field；
4) binocular camera is fixed form, it is impossible to enough arbitrary mobile, changes observed direction and picture zoom scaling；
5) airport scope is big, monitor content dispersion, so the video camera of the present invention is required to use the high definition more than 2,000,000 pixels
Video camera；
6) because the feature of airport monitoring, the requirement monitored for background is low, and the supervision for moving target requires higher；So
The present invention monitors that strategy is that different content is used different refreshing frequencys, specifically carries out the target back of the body with mixed Gauss model
The detection of scape, refreshes 5 times for moving target, and background refreshes 1 time for only one minute for one second；
7) present invention can manually arrange typical airfield background 3D model, or refreshes preservation airport background 3D mould by certain frequency
Type is to fixed disk file, such as every day；The airport background 3D model preserved calls display when being available for needing；
8) present invention can select suitable background 3D airport model to replace the airport background currently automatically generated by hand；
9) synchronization/frame losing processes, and the present invention is not required for strict time synchronized, but uses and receive data with regard to instant computing brush
The scheme of new overall situation world coordinates, because video camera is not constantly in motion, and the time difference between binocular camera also may be used
To ignore；
10) dim light of night filling background process, whether the present invention can come into night with automatic decision scene, in the condition being night
Under, if arranging of user can abandon background at night, then show by the current airport background model on daytime preserved, to promote
The effect that airport three-dimensional panorama monitors；
11) greasy weather processes, and during the greasy weather, the present invention is by the size of automatic decision mist, after mistiness degree reaches to a certain degree, equally may be used
To show by the current airport background model on daytime preserved；
12) background area in frame of video is manually set, before by mixed Gauss model detection moving target step, Ke Yijian
Look into user's whether hand labeled background area, if any, directly invoke labelling result and be set to background, without again with mixing height
This model carries out detecting whether this partial pixel is background, because some video cameras are fixed, and so can be greatly
Reduce amount of calculation；
13) rainy day scene process, for the rainy day, the present invention will increase smooth pre-treatment step upon receiving the image；
14) with the interface of weather system, if there being the interface with weather system, can according to the Weather information received replace night from
Dynamic judgement, rainy day automatic decision, greasy weather automatic decision；
15) this method can support that GPU parallel computation is to increase computational efficiency.
Accompanying drawing explanation
Fig. 1 is the system hardware that a kind of airdrome scene three-dimensional panorama based on binocular camera of the present invention monitors that method is corresponding
Schematic diagram.
Fig. 2 is the schematic diagram of the radial distortion of the scaling method used by the present invention.
Detailed description of the invention
In order to make the purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, right
The present invention is further elaborated.
A kind of airdrome scene three-dimensional panorama based on binocular camera of the present invention monitors that the basic thought of method is: use double
Lens camera carries out maximized covering to airdrome scene, and the Main Function of binocular camera is to be clapped this binocular camera
Take the photograph scene and carry out measurement in space recovery, it is thus achieved that dense point cloud.Then registration Algorithm is used to be joined by the some cloud of all binocular cameras
Under the accurate world coordinate system arriving unified airport.Finally use the Surface Creation method of some cloud, generate three-dimensional panorama mould further
Type, carries out scene video intuitive monitoring in video monitor client for user.
The occupation mode of the algorithm of the present invention is: use binocular camera that airdrome scene is carried out maximized covering, by
Stereo restoration server obtains the two-path video of every suit binocular camera, carries out stereo restoration, obtains current camera coordinate
Under three-dimensional some cloud, a stereo restoration server can process the 8 road videos that 4 set binocular cameras obtain.Will a cloud transmission
Parameter to the registration pre-registration of server recycling carries out unifying to be registrated to airport world coordinate system.Registration server again will registration
After data be sent to Surface Creation server and carry out the surface recovery of threedimensional model, render and form final Three-dimensional monitor mould
Type.The hardware composition diagram of system is shown in Fig. 1.
Step one video camera mark described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention
Determining specifically: preferably, the method used is to consider radial constraint and the calibration algorithm of distortion.
The relation of world coordinate system and camera coordinate system is:
, wherein K with t is taken as:,
The schematic diagram of radial distortion is as shown in Figure 2.Model used by distortion is as follows:
(u, v) it is the actual coordinate of the pixel of image, (x, y) it is preferable pixel coordinate, (u c, v c) it is center of distortion.
Radial constraint refers on the image plane, center of distortion point (u c, v c), ideal image pixel (x, y), real image pixel
Point (u, v) conllinear.Demarcate the key step solved as follows:
1) radially consistent constraint is utilized to solveRWitht 1,t 2
2) effective focal length is solvedf , translation in Z-directiont 3And distortion parameterk 1。
Step 2 video acquisition described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention
Specifically: by the video data after compression by internal network transport to corresponding stereo restoration server, it is preferred that decompression
H.264 contracting institute accepted standard is.
Step 3 described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention is preferred
, image characteristics extraction concrete grammar is: BRISK feature is to specifically refer to Binary Robust Invariant Scalable
Key-points.It should be noted that find here is only sparse Feature point correspondence relation, is only used for external parameters of camerasR i,T i Calculating.
Step 4 described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention is three-dimensional
Mate specifically: preferably, the method for image rectification uses the projective transformation method that limit is mapped to infinite point here,
Its step is as follows:
1) Feature point correspondence utilizing step 3 to obtain obtains corresponding pixel corresponding relation to be asked between two width images
Fundamental matrixFWith the limit in each imageeWithe’, at least need 7 pairs of match points.
2) wille’It is mapped to infinite point (1,0,0), obtains corresponding projective transformationH’。
3) ask byeIt is mapped to the projective transformation that infinite point (1,0,0) is correspondingH, can ask for by asking to minimize.
4) projective transformation is then usedHWithH’Resampling the first width, the second width image respectivelyI i1 WithI i2 。
After once panorama has mated, after global context generates, can mate just for moving target.Moving target
The detection mixed Gauss model of video frequency motion target employs 5 gaussian kernel to each pixel, when each gaussian kernel is not matched
Weight coefficient become the 0.97 of initial value.The thresholding of background gaussian kernel weight sum is set as 0.95.To the value of pixel of newly arriving with
Each is considered as that the average in the gaussian kernel of background compares, if its difference is between the variance of 2 times, then recognizes
For being background, otherwise it is assumed that be prospect.
Step 5 described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention is three-dimensional
Rebuild specifically: because the present invention only be binocular camera, three-dimensional reconstruction also relies on the projection square of two video cameras
Battle arrayP i1 WithP i2 , needing exist for must be consistent with the distance between binocular camera by the rescaling of projection matrix.According to projection public affairs
Formula have [x, y, 1]=P i1 X j [x’, y, 1]=P i2 X j Two formula can solveI’ i1 (x, y) andI’ i2 (x’,y) common corresponding three-dimensional coordinate pointX j Value.The point solving all pixel matching corresponding i.e. can get single binocular camera
The three-dimensional coordinate of all pixels of dense point cloudX。
Step 6 described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention is unified
Point cloud coordinate, the longitude and latitude transformation approach of employing, some key points in airport can be measured its longitude and latitude, and it is changed
Becoming and form unified world coordinates under the Euclidean coordinate system of a standard, this coordinate origin coordinate is typically at the central point on airport
Position.Longitude and latitude transformation approach is divided into 2 steps:
1) coordinate latitude and longitude coordinates of measurement being converted in model coordinate systems, uses Gauss Kru&4&ger projection (Gauss-
Kruger projection)；
2) set the normalized relative dimensional coordinate of camera position asb i , the Euclidean coordinate after longitude and latitude conversion isB i , then it
And transition matrixGRelation be that this formula method of least square is sought transition matrixGInitial value, then use Levenberg-
Marquardt is non-linear tries to achieve more excellent transition matrixG,GAfter need to keeping changing, the orthogonality of coordinate axes is consistent with ratio
Property, i.e. add rotation, translation, equal proportion scaling constraint.May then pass throughGBy the three-dimensional coordinate of pixelXUnified to the world
Under coordinate system.
Step 7 described in a kind of airdrome scene three-dimensional panorama supervision method based on binocular camera of the present invention based on
The resurfacing of some cloud, gridding processes the method used and is:
1) walk the world coordinates provided in utilization, check and make all of point the most all be evenly distributed on the former of world coordinates
Around, the average distance variance at Ji Diandao center is in the range of a setting for point；
2) reset with the parameter on multinomial model surface so that it is the Least-squares estimator arriving a little surface is minimum；
3) again by up-sampling and the iterative process optimizing surface multinomial model of down-sampling.
4) finally complete to render and form final background model.
Owing in airport monitoring, background range is relatively big, the ratio accounting for image is big, and target is typically small and accounts for current frame image
Area ratio compares the least, so general setting of the present invention is, carries out three-dimensional reconstruction for background at the beginning and runs this step
All resurfacings, for the target of high frequency motion, the most not running surface reconstruction procedures, only observe its dense three-dimensional point cloud,
So can greatly reduce the operand of system.
Meanwhile, further the most what time technical scheme also has:
1) using binocular camera that airdrome scene is carried out maximized covering, between binocular camera, lap requires to be more than
65%, the shooting direction of binocular camera is close to parallel；
2) as far as possible binocular camera between there is also overlap, lap requires more than 30%；For there being the binocular of overlap to take the photograph
Camera pair, can add a boundling adjustment and optimize each camera intrinsic parameter therein and outer parameter, to improve precision；
3) if two set binocular cameras between there is no overlap, then every suit video camera is carried out in advance be registrated to unified machine
Under the world coordinate system of field；
4) binocular camera is fixed form, it is impossible to enough arbitrary mobile, changes observed direction and picture zoom scaling；
5) airport scope is big, monitor content dispersion, so the video camera of the present invention is required to use the high definition more than 2,000,000 pixels
Video camera；
6) because the feature of airport monitoring, the requirement monitored for background is low, and the supervision for moving target requires higher；So
The present invention monitors that strategy is that different content is used different refreshing frequencys, specifically carries out the target back of the body with mixed Gauss model
The detection of scape, refreshes 5 times for moving target, and background refreshes 1 time for only one minute for one second；
7) present invention can manually arrange typical airfield background 3D model, or refreshes preservation airport background 3D mould by certain frequency
Type is to fixed disk file, such as every day；The airport background 3D model preserved calls display when being available for needing；
8) present invention can select suitable background 3D airport model to replace the airport background currently automatically generated by hand；
9) synchronization/frame losing processes, and the present invention is not required for strict time synchronized, but uses and receive data with regard to instant computing brush
The scheme of new overall situation world coordinates, because video camera is not constantly in motion, and the time difference between binocular camera also may be used
To ignore；
10) dim light of night filling background process, whether the present invention can come into night with automatic decision scene, in the condition being night
Under, if arranging of user can abandon background at night, then show by the current airport background model on daytime preserved, to promote
The effect that airport three-dimensional panorama monitors；
11) greasy weather processes, and during the greasy weather, the present invention is by the size of automatic decision mist, after mistiness degree reaches to a certain degree, equally may be used
To show by the current airport background model on daytime preserved；
12) background area in frame of video is manually set, before by mixed Gauss model detection moving target step, Ke Yijian
Look into user's whether hand labeled background area, if any, directly invoke labelling result and be set to background, without again with mixing height
This model carries out detecting whether this partial pixel is background, because some video cameras are fixed, and so can be greatly
Reduce amount of calculation；
13) rainy day scene process, for the rainy day, the present invention will increase smooth pre-treatment step upon receiving the image；
14) with the interface of weather system, if there being the interface with weather system, can according to the Weather information received replace night from
Dynamic judgement, rainy day automatic decision, greasy weather automatic decision；
15) this method can support that GPU parallel computation is to increase computational efficiency.
With the above-mentioned desirable embodiment according to the present invention for enlightenment, by above-mentioned description, relevant staff completely may be used
With in the range of without departing from this invention technological thought, carry out various change and amendment.The technical model of this invention
Enclose the content being not limited in description, it is necessary to determine its technical scope according to right.
Claims (4)
1. airdrome scene three-dimensional panorama based on binocular camera monitors a method, it is characterized in that specifically comprising the following steps that
Step one: camera calibration, demarcates in advance for each video camera in every a pair binocular camera, is imaged
Machine intrinsic parameterK i1 WithK i2 And distortion parameter；
Step 2: video acquisition, is installed to video camera airport and monitors on a position accordingly, it is thus achieved that corresponding video；During installation,
The distance between two video cameras need to be measuredd i ;
Step 3: image characteristics extraction, the image that every pair of video camera is obtainedI i1 WithI i2 Calculate its corresponding characteristic point, calculating side
Method uses BRISK (Binary Robust Invariant Scalable Key-points), it is thus achieved that respective image feature is corresponding
PointI i1 WithI i2 ；, and calculate corresponding fundamental matrix F (Fundamental Matrix) according to the feature correspondence of two width images,
Thus extrapolate the essential matrix that two width images are corresponding, thus calculate the projection matrix (Projection of two width images
Matrix)P i1 WithP i2 ；
Step 4: Stereo matching, the characteristic point that step 3) obtains is sparse features point correspondence, wants dense characteristic point here
Corresponding relation could be used for airdrome scene panorama and monitor；First have to carry out image rectification, by image polar curve (Epipolar) furnishing
Image level line, obtainsI’ i1 WithI’ i2 ；Meanwhile, it is also performed to the detection of moving target, if detected after completing to initialize
Carry out this pixel do not change and move, the most no longer carry out Stereo matching and follow-up three-dimensional reconstruction, but enter directly into
The resurfacing stage of post processing；The method that the detection of video frequency motion target uses is the matching detection side of mixed Gauss model
Method；
Then forI’ i1 In a pixelI’ i1 (x, y), along this pointI’ i2 Search on corresponding corresponding horizontal polar curve
The pixel that rope is most likeI’ i2 (x’, y)；Here the similarity criterion used is the normalized crosscorrelation NCC of standard
(Normalized Cross Correlation), matching strategy is LRC(Left-right Checking), i.e. findI’ i2
(x’, yAfter), also will be byI’ i2 (x’, y)I’ i1 In corresponding polar curve on scan for concurrently that existing corresponding corresponding point are stillI’ i1 (x, y), just confirm that the two is corresponding match point；
Step 5: three-dimensional reconstruction, according to projection formula have [x 1 , y 1 , 1]=P i1 X j [x 2 , y 2 , 1]=P i2 X j Two formula can
To solveI i1 (x 1 , y 1 ) andI i2 (x 2 , y 2 ) this to matched pixel to common corresponding three-dimensional coordinate pointX j Value；Solve institute
The point having pixel matching corresponding i.e. can obtain the three-dimensional coordinate of all pixelsX,I.e. dense point cloud；
Step 6: unified some cloud coordinate, after all of video camera is to obtaining dense point cloud, should be placed on one all of which
A unified global point cloud is formed under world coordinate system；
Step 7: resurfacing based on a cloud (Surface Reconstruction), i.e. grid (Mesh) change process.
2. three-dimensional panorama as claimed in claim 1 monitors method, it is characterised in that: after once panorama has mated, the overall situation back of the body
After scape generates, can mate just for moving target；The detection mixed Gauss model pair of Moving Targets Based on Video Streams moving target
Each pixel employs 5 gaussian kernel, and weight coefficient when each gaussian kernel is not matched becomes the 0.97 of initial value；Background Gauss
The thresholding of core weight sum is set as 0.95；It to the value of pixel of newly arriving it is considered as the average in the gaussian kernel of background with each
Compare, if its difference is between the variance of 2 times, then it is assumed that be background, otherwise it is assumed that be prospect.
3. three-dimensional panorama as claimed in claim 1 monitors method, it is characterised in that: three-dimensional reconstruction is carried out at the beginning for background
And run this all resurfacing walked, for the target of high frequency motion, the most not running surface reconstruction procedures, only observe it dense
Three-dimensional point cloud, so can greatly reduce the operand of system.
4. three-dimensional panorama as claimed in claim 1 monitors method, it is characterised in that: further technical scheme applies
Following setting:
1) using binocular camera that airdrome scene is carried out maximized covering, between binocular camera, lap requires to be more than
65%, the shooting direction of binocular camera is close to parallel；
2) as far as possible binocular camera between there is also overlap, lap requires more than 30%；For there being the binocular of overlap to take the photograph
Camera pair, can add a boundling adjustment and optimize each camera intrinsic parameter therein and outer parameter, to improve precision；
3) if two set binocular cameras between there is no overlap, then every suit video camera is carried out in advance be registrated to unified machine
Under the world coordinate system of field；
4) binocular camera is fixed form, it is impossible to enough arbitrary mobile, changes observed direction and picture zoom scaling；
5) airport scope is big, monitor content dispersion, so the video camera of the present invention is required to use the high definition more than 2,000,000 pixels
Video camera；
6) because the feature of airport monitoring, the requirement monitored for background is low, and the supervision for moving target requires higher；So
The present invention monitors that strategy is that different content is used different refreshing frequencys, specifically carries out the target back of the body with mixed Gauss model
The detection of scape, refreshes 5 times for moving target, and background refreshes 1 time for only one minute for one second；
7) present invention can manually arrange typical airfield background 3D model, or refreshes preservation airport background 3D mould by certain frequency
Type is to fixed disk file, such as every day；The airport background 3D model preserved calls display when being available for needing；
8) present invention can select suitable background 3D airport model to replace the airport background currently automatically generated by hand；
9) synchronization/frame losing processes, and the present invention is not required for strict time synchronized, but uses and receive data with regard to instant computing brush
The scheme of new overall situation world coordinates, because video camera is not constantly in motion, and the time difference between binocular camera also may be used
To ignore；
10) dim light of night filling background process, whether the present invention can come into night with automatic decision scene, in the condition being night
Under, if arranging of user can abandon background at night, then show by the current airport background model on daytime preserved, to promote
The effect that airport three-dimensional panorama monitors；
11) greasy weather processes, and during the greasy weather, the present invention is by the size of automatic decision mist, after mistiness degree reaches to a certain degree, equally may be used
To show by the current airport background model on daytime preserved；
12) background area in frame of video is manually set, before by mixed Gauss model detection moving target step, Ke Yijian
Look into user's whether hand labeled background area, if any, directly invoke labelling result and be set to background, without again with mixing height
This model carries out detecting whether this partial pixel is background, because some video cameras are fixed, and so can be greatly
Reduce amount of calculation；
13) rainy day scene process, for the rainy day, the present invention will increase smooth pre-treatment step upon receiving the image；
14) with the interface of weather system, if there being the interface with weather system, can according to the Weather information received replace night from
Dynamic judgement, rainy day automatic decision, greasy weather automatic decision；
15) this method can support that GPU parallel computation is to increase computational efficiency.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610549128.2A CN106204595B (en) | 2016-07-13 | 2016-07-13 | A kind of airdrome scene three-dimensional panorama monitoring method based on binocular camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610549128.2A CN106204595B (en) | 2016-07-13 | 2016-07-13 | A kind of airdrome scene three-dimensional panorama monitoring method based on binocular camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106204595A true CN106204595A (en) | 2016-12-07 |
| CN106204595B CN106204595B (en) | 2019-05-10 |
Family
ID=57477166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610549128.2A Active CN106204595B (en) | 2016-07-13 | 2016-07-13 | A kind of airdrome scene three-dimensional panorama monitoring method based on binocular camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106204595B (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106604016A (en) * | 2017-01-26 | 2017-04-26 | 上海图漾信息科技有限公司 | Stereoscopic video capture system |
| CN106803271A (en) * | 2016-12-23 | 2017-06-06 | 成都通甲优博科技有限责任公司 | A kind of camera marking method and device of vision guided navigation unmanned plane |
| CN106950985A (en) * | 2017-03-20 | 2017-07-14 | 成都通甲优博科技有限责任公司 | A kind of automatic delivery method and device |
| CN107240303A (en) * | 2017-05-02 | 2017-10-10 | 成都艾尔伯特科技有限责任公司 | Airport cognitive teaching system and method |
| CN107845130A (en) * | 2017-12-20 | 2018-03-27 | 浙江煮艺文化科技有限公司 | A kind of surrounding three-dimensional reconstructing method |
| CN108462838A (en) * | 2018-03-16 | 2018-08-28 | 深圳岚锋创视网络科技有限公司 | A kind of panoramic video anti-fluttering method, device and portable terminal |
| CN108469254A (en) * | 2018-03-21 | 2018-08-31 | 南昌航空大学 | A kind of more visual measuring system overall calibration methods of big visual field being suitable for looking up and overlooking pose |
| CN108592919A (en) * | 2018-04-27 | 2018-09-28 | 百度在线网络技术（北京）有限公司 | The drawing of opposite edges and localization method, device, storage medium and terminal device |
| CN108734739A (en) * | 2017-04-25 | 2018-11-02 | 北京三星通信技术研究有限公司 | The method and device generated for time unifying calibration, event mark, database |
| CN108876899A (en) * | 2018-05-03 | 2018-11-23 | 中国船舶重工集团公司第七�三研究所 | A kind of airfield runway foreign object detection binocular solid system and detection method |
| CN109188355A (en) * | 2018-09-14 | 2019-01-11 | 四川大学 | A kind of optimization of multipoint location system receiving antenna and Optimal Station method |
| CN110176032A (en) * | 2019-04-28 | 2019-08-27 | 暗物智能科技(广州)有限公司 | A kind of three-dimensional rebuilding method and device |
| CN111696162A (en) * | 2020-06-11 | 2020-09-22 | 中国科学院地理科学与资源研究所 | Binocular stereo vision fine terrain measurement system and method |
| CN111723830A (en) * | 2019-03-20 | 2020-09-29 | 杭州海康威视数字技术股份有限公司 | Image mapping method, device and equipment and storage medium |
| CN111798515A (en) * | 2020-06-30 | 2020-10-20 | 大连亚泰华光电技术有限公司 | Stereoscopic vision monitoring method for burning condition identification |
| CN111862179A (en) * | 2019-04-12 | 2020-10-30 | 北京城市网邻信息技术有限公司 | Three-dimensional object modeling method and apparatus, image processing device, and medium |
| CN112001963A (en) * | 2020-07-31 | 2020-11-27 | 浙江大华技术股份有限公司 | Fire fighting channel investigation method, system and computer equipment |
| CN113660509A (en) * | 2021-10-18 | 2021-11-16 | 上海飞机制造有限公司 | Three-dimensional model processing system and method based on cloud rendering |
| CN115713565A (en) * | 2022-12-16 | 2023-02-24 | 盐城睿算电子科技有限公司 | Target positioning method for binocular servo camera |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100239122A1 (en) * | 2007-10-11 | 2010-09-23 | Hans-Juergen Busch | Method for creating and/or updating textures of background object models, video monitoring system for carrying out the method, and computer program |
| CN102129708A (en) * | 2010-12-10 | 2011-07-20 | 北京邮电大学 | Fast multilevel imagination and reality occlusion method at actuality enhancement environment |
| CN102646275A (en) * | 2012-02-22 | 2012-08-22 | 西安华旅电子科技有限公司 | Method for realizing virtual three-dimensional superposition through tracking and positioning algorithms |
| CN103198488A (en) * | 2013-04-16 | 2013-07-10 | 北京天睿空间科技有限公司 | PTZ surveillance camera realtime posture rapid estimation method |
| CN103297798A (en) * | 2012-03-02 | 2013-09-11 | 包崇兵 | Three-dimensional reconstruction method of scene spots in binocular stereo visual system |
| CN105678748A (en) * | 2015-12-30 | 2016-06-15 | 清华大学 | Interactive calibration method and apparatus based on three dimensional reconstruction in three dimensional monitoring system |
-
2016
- 2016-07-13 CN CN201610549128.2A patent/CN106204595B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100239122A1 (en) * | 2007-10-11 | 2010-09-23 | Hans-Juergen Busch | Method for creating and/or updating textures of background object models, video monitoring system for carrying out the method, and computer program |
| CN102129708A (en) * | 2010-12-10 | 2011-07-20 | 北京邮电大学 | Fast multilevel imagination and reality occlusion method at actuality enhancement environment |
| CN102646275A (en) * | 2012-02-22 | 2012-08-22 | 西安华旅电子科技有限公司 | Method for realizing virtual three-dimensional superposition through tracking and positioning algorithms |
| CN103297798A (en) * | 2012-03-02 | 2013-09-11 | 包崇兵 | Three-dimensional reconstruction method of scene spots in binocular stereo visual system |
| CN103198488A (en) * | 2013-04-16 | 2013-07-10 | 北京天睿空间科技有限公司 | PTZ surveillance camera realtime posture rapid estimation method |
| CN105678748A (en) * | 2015-12-30 | 2016-06-15 | 清华大学 | Interactive calibration method and apparatus based on three dimensional reconstruction in three dimensional monitoring system |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106803271B (en) * | 2016-12-23 | 2020-04-28 | 成都通甲优博科技有限责任公司 | Camera calibration method and device for visual navigation unmanned aerial vehicle |
| CN106803271A (en) * | 2016-12-23 | 2017-06-06 | 成都通甲优博科技有限责任公司 | A kind of camera marking method and device of vision guided navigation unmanned plane |
| CN106604016A (en) * | 2017-01-26 | 2017-04-26 | 上海图漾信息科技有限公司 | Stereoscopic video capture system |
| CN106950985A (en) * | 2017-03-20 | 2017-07-14 | 成都通甲优博科技有限责任公司 | A kind of automatic delivery method and device |
| CN108734739A (en) * | 2017-04-25 | 2018-11-02 | 北京三星通信技术研究有限公司 | The method and device generated for time unifying calibration, event mark, database |
| CN107240303A (en) * | 2017-05-02 | 2017-10-10 | 成都艾尔伯特科技有限责任公司 | Airport cognitive teaching system and method |
| CN107845130A (en) * | 2017-12-20 | 2018-03-27 | 浙江煮艺文化科技有限公司 | A kind of surrounding three-dimensional reconstructing method |
| CN108462838A (en) * | 2018-03-16 | 2018-08-28 | 深圳岚锋创视网络科技有限公司 | A kind of panoramic video anti-fluttering method, device and portable terminal |
| CN108462838B (en) * | 2018-03-16 | 2020-10-02 | 影石创新科技股份有限公司 | Panoramic video anti-shake method and device and portable terminal |
| CN108469254A (en) * | 2018-03-21 | 2018-08-31 | 南昌航空大学 | A kind of more visual measuring system overall calibration methods of big visual field being suitable for looking up and overlooking pose |
| CN108592919A (en) * | 2018-04-27 | 2018-09-28 | 百度在线网络技术（北京）有限公司 | The drawing of opposite edges and localization method, device, storage medium and terminal device |
| CN108876899A (en) * | 2018-05-03 | 2018-11-23 | 中国船舶重工集团公司第七�三研究所 | A kind of airfield runway foreign object detection binocular solid system and detection method |
| CN109188355A (en) * | 2018-09-14 | 2019-01-11 | 四川大学 | A kind of optimization of multipoint location system receiving antenna and Optimal Station method |
| CN111723830B (en) * | 2019-03-20 | 2023-08-29 | 杭州海康威视数字技术股份有限公司 | Image mapping method, device and equipment and storage medium |
| CN111723830A (en) * | 2019-03-20 | 2020-09-29 | 杭州海康威视数字技术股份有限公司 | Image mapping method, device and equipment and storage medium |
| CN111862179A (en) * | 2019-04-12 | 2020-10-30 | 北京城市网邻信息技术有限公司 | Three-dimensional object modeling method and apparatus, image processing device, and medium |
| CN111862179B (en) * | 2019-04-12 | 2022-02-18 | 北京城市网邻信息技术有限公司 | Three-dimensional object modeling method and apparatus, image processing device, and medium |
| CN110176032B (en) * | 2019-04-28 | 2021-02-26 | 暗物智能科技(广州)有限公司 | Three-dimensional reconstruction method and device |
| CN110176032A (en) * | 2019-04-28 | 2019-08-27 | 暗物智能科技(广州)有限公司 | A kind of three-dimensional rebuilding method and device |
| CN111696162B (en) * | 2020-06-11 | 2022-02-22 | 中国科学院地理科学与资源研究所 | Binocular stereo vision fine terrain measurement system and method |
| CN111696162A (en) * | 2020-06-11 | 2020-09-22 | 中国科学院地理科学与资源研究所 | Binocular stereo vision fine terrain measurement system and method |
| CN111798515A (en) * | 2020-06-30 | 2020-10-20 | 大连亚泰华光电技术有限公司 | Stereoscopic vision monitoring method for burning condition identification |
| CN111798515B (en) * | 2020-06-30 | 2024-01-12 | 大连亚泰华光电技术有限公司 | Stereoscopic vision monitoring method for recognizing incineration condition |
| CN112001963A (en) * | 2020-07-31 | 2020-11-27 | 浙江大华技术股份有限公司 | Fire fighting channel investigation method, system and computer equipment |
| CN113660509A (en) * | 2021-10-18 | 2021-11-16 | 上海飞机制造有限公司 | Three-dimensional model processing system and method based on cloud rendering |
| CN115713565A (en) * | 2022-12-16 | 2023-02-24 | 盐城睿算电子科技有限公司 | Target positioning method for binocular servo camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106204595B (en) | 2019-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106204595B (en) | A kind of airdrome scene three-dimensional panorama monitoring method based on binocular camera | |
| US8180107B2 (en) | Active coordinated tracking for multi-camera systems | |
| CN103236160B (en) | Road network traffic condition monitoring system based on video image processing technology | |
| US20080074494A1 (en) | Video Surveillance System Providing Tracking of a Moving Object in a Geospatial Model and Related Methods | |
| CN106920276B (en) | A kind of three-dimensional rebuilding method and system | |
| CN111899282B (en) | Pedestrian track tracking method and device based on binocular camera calibration | |
| CN111435081B (en) | Sea surface measuring system, sea surface measuring method and storage medium | |
| CN106033614B (en) | A kind of mobile camera motion object detection method under strong parallax | |
| CN110889873A (en) | Target positioning method and device, electronic equipment and storage medium | |
| CN104318604A (en) | 3D image stitching method and apparatus | |
| CA3133537A1 (en) | Homography through satellite image matching | |
| CN111383204A (en) | Video image fusion method, fusion device, panoramic monitoring system and storage medium | |
| Takacs et al. | 3D mobile augmented reality in urban scenes | |
| Ke et al. | Roadway surveillance video camera calibration using standard shipping container | |
| Junejo et al. | Autoconfiguration of a dynamic nonoverlapping camera network | |
| Koppanyi et al. | Experiences with acquiring highly redundant spatial data to support driverless vehicle technologies | |
| Wang et al. | Automated mosaicking of UAV images based on SFM method | |
| CN116151320A (en) | Visual odometer method and device for resisting dynamic target interference | |
| Tu et al. | Method of Using RealSense Camera to Estimate the Depth Map of Any Monocular Camera | |
| US20200302641A1 (en) | Methods and systems for silent object positioning with image sensors | |
| CN110884672A (en) | Auxiliary landing device of panoramic imaging helicopter | |
| CN111486820B (en) | Measurement system, measurement method, and storage medium | |
| Zhou et al. | WLTOG: An optimization approach for wild large‐range target omnidirectional geolocation based on monocular PTZ camera | |
| CN109214984B (en) | Image acquisition method and device, autonomous positioning navigation system and computing equipment | |
| Shi et al. | Quick calibration of massive urban outdoor surveillance cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |