CN106373088B - The quick joining method of low Duplication aerial image is tilted greatly - Google Patents
The quick joining method of low Duplication aerial image is tilted greatly Download PDFInfo
- Publication number
- CN106373088B CN106373088B CN201610727071.0A CN201610727071A CN106373088B CN 106373088 B CN106373088 B CN 106373088B CN 201610727071 A CN201610727071 A CN 201610727071A CN 106373088 B CN106373088 B CN 106373088B
- Authority
- CN
- China
- Prior art keywords
- image
- point
- region
- feature
- greatly
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 239000011159 matrix material Substances 0.000 claims abstract description 43
- 238000003384 imaging method Methods 0.000 claims abstract description 26
- 238000006243 chemical reaction Methods 0.000 claims abstract description 21
- 239000000284 extract Substances 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 10
- 239000002131 composite material Substances 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 239000013256 coordination polymer Substances 0.000 claims description 4
- 230000007704 transition Effects 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 2
- 230000001419 dependent effect Effects 0.000 claims description 2
- 210000001525 retina Anatomy 0.000 claims description 2
- 239000012141 concentrate Substances 0.000 claims 1
- 239000000203 mixture Substances 0.000 claims 1
- 230000004304 visual acuity Effects 0.000 claims 1
- 230000009466 transformation Effects 0.000 abstract description 16
- 238000005516 engineering process Methods 0.000 description 13
- 230000015572 biosynthetic process Effects 0.000 description 7
- 238000013507 mapping Methods 0.000 description 6
- 238000003786 synthesis reaction Methods 0.000 description 6
- 238000011160 research Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010304 firing Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000003716 rejuvenation Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention proposes the quick joining methods that one kind tilts greatly low Duplication aerial image, can be improved using the present invention and tilt low Duplication image registration robustness greatly, reduce stitching error, improve wide format images combined coefficient.The technical scheme is that: the position distribution relationship of image is calculated first with the imaging auxiliary data of aerial image, establishes image positional relationship matrix and calculating benchmark figure；Then by the adjacent map path between Shortest Path Searching image and reference map, the overlapping region on neighborhood paths between adjacent image is calculated, and extracts the significant characteristics point of overlapping region.Secondly characteristic point or region are normalized to vertex, construct multiple dimensioned figure using different adjacency point sets, and carry out matching primitives space conversion matrices with figure matching；Last use space transformation matrix calculates the pixel value that respective coordinates and use bilinear interpolation of the composograph pixel in source images calculate composograph, obtains final stitching image.
Description
Technical field
The present invention relates to a kind of field of image processings about photograph cartography, computer vision, image procossing and computer
The image split-joint method of graphics, and in particular to one kind is based on the low Duplication aerial chart of the matched big inclination of conspicuousness key point diagram
As quick joining method.
Background technique
Image mosaic (image mosaic) technology is exactly the image for having lap for several, it may be that different time,
The image that different perspectives obtains, is combined into the technology of the seamless high-definition picture of width large size, is by one group of mutual overlapping portion
The image sequence divided carries out spatial match alignment, and the wide viewing angle that a width includes each image sequence information is formed after resampling synthesizes
The technology of scene, complete, high-resolution new images.The image mosaic research of early stage is always for cartography of taking a picture, mainly
It is to largely taking photo by plane or the integration of the image of satellite.In order to solve the limitation of camera lens shooting angle, with computer technology and
The exhibition of digital image processing techniques, automatic Image Stitching technology be increasingly becoming photogrammetry, computer vision, image procossing and
The research hotspot of computer graphics, and be widely used in deep space exploration, remote sensing image processing (multispectral classification, environmental monitoring,
Change detection, image mosaic, weather forecast, especially big image in different resolution synthesis, Earth Information emerging system (GIS)), medical image
Analysis (CT and NMR data integration etc.), cartography map rejuvenation, the positioning of computer vision target, automated quality control, video pressure
Reduce the staff the fields such as code, virtual reality technology, super-resolution reconstruction.There are many method of image mosaic, and different algorithm steps have
Different, but rough process is identical.In general, image mosaic mainly comprises the steps that
A) image preprocessing.Basic operation including Digital Image Processing, such as denoising, edge extracting, histogram treatment,
It establishes the matching template of image and certain transformation, such as the operation of Fourier transformation, wavelet transformation is carried out to image.
B) image registration.It is exactly using certain matching strategy, the template or characteristic point found out in image to be spliced are being joined
Corresponding position in image is examined, and then determines the transformation relation between two images.
C) transformation model is established.According to the corresponding relationship between template or characteristics of image, calculate in mathematical model
Each parameter value, to establish the mathematical transformation model of two images.
D) uniform coordinate converts.According to the mathematics transformation model of foundation, image to be spliced is transformed into the seat of reference picture
In mark system, uniform coordinate transformation is completed.
E) fusion reconstruct.Overlapping region with stitching image is merged to obtain the smooth and seamless panorama sketch of splicing reconstruct
Picture.
The registration of adjacent image and splicing are the key that panorama picture formation technologies, and the research in relation to image registration techniques is so far
Existing very long history, main method have following two: the smallest method of luminance difference based on two images and based on spy
The method of sign.Using the joining method based on feature templates matching characteristic point.It is certain that this method allows image to be spliced to have
Inclination and deformation, overcome the problem that axle center must be consistent when obtaining image, while allowing to have certain color difference between adjacent image.
The splicing of panorama sketch mainly includes following 4 steps: pre-splicing, i.e., the more accurate position that determining two width adjacent images are overlapped of image
It sets, the search being characterized a little lays the foundation.The extraction of characteristic point finds spy to be matched that is, after essentially coinciding position and determining
Sign point.Image matrix transformation and splicing establish the transformation matrix of image according to match point and realize the splicing of image.It is finally
The smoothing processing of image.
It, can be to from the same area using the image registration techniques in image mosaic technology in remote sensing technology field
Two width or multiple image are compared, and also can use image mosaic technology for what remote sensing satellite took has distortion ground image
It is spliced into more accurately complete image, as the foundation further studied.The research of image mosaic technology is most of at this stage
Be based on satellite remote sensing images, aerial mapping image, digital camera pan-shot image, image-forming condition usually has special want
It asks, such as the firing angle degree imaging that is positive substantially of satellite remote sensing images, imaging platform are relatively stable；Aerial mapping image is essentially close
It is imaged like positive firing angle degree, image-forming range is closer, and the image Duplication for surveying and drawing requirement is also higher；The image-forming range of pan-shot image
It is relatively close, it is desirable that image Duplication it is also higher, pattern distortion is smaller under above-mentioned image-forming condition, the visual angle between image overlapping region
Difference is small.But in image scouting, due to the limitation of many shooting conditions, farther out such as image-forming range, imaging angle is larger, imaging
Platform mobility strong, image Duplication is low, can not obtain the image of high quality.Therefore, traditional joining method solve it is remote,
The problem of still having many difficult points when image mosaic problem under big inclination angle, low Duplication image-forming condition, specifically exist has:
(1) remote, big inclination angle image-forming condition bring image mosaic problem
Needs of the contemporary aircraft due to executing task at a distance, it is necessary to shoot image under remote big inclination angle, far
Apart from big inclination angle, shooting inevitably causes pattern distortion.Reconnaissance flight device is since task needs to carry out fast reserve, no
But image Duplication is lower, and there is also visual angle differences between the sequence image of continuous imaging, and shooting distance farther out, inclination angle
In biggish situation, pattern distortion difference caused by small visual angle difference is also bigger.Therefore, compared to shooting is just penetrated, at a distance
The two images shot under big inclination angle, the difference of image overlapping region due to caused by the difference of visual angle become much larger.Tradition figure
The research of picture splicing is mostly based on satellite application, and joining method is suitable for just penetrating or the splicing of approximate orthograph picture, for
There is no fine considerations for distortion caused by big inclination angle, and it is dfficult to apply to the splicings of aerial reconnaissance image.
The key technology of splicing is image registration.The image registration techniques of early stage mainly use point match method, such methods
Speed is slow, precision is low, and usually needs manually to choose initial matching point, can not adapt to large data volume.Image registration is figure
As the basis of fusion, and the calculation amount of image registration algorithm is generally very big.Traditional can resist affine, scale, brightness to become
The method for registering of change is the matching process based on local feature region or characteristic area, the characteristic point of extraction or the feature of characteristic area
Vector is the statistical nature based on characteristic point surrounding neighbors pixel, matching when by calculate feature vector between similarity away from
From, think when similarity distance meets threshold requirement Feature Points Matching success.Successful match or not therefore, feature is depended on
The similarity of point regional area, but in the splicing of big tilted image, inclination causes changing greatly for image local area, only relies on
The similarity mode of regional area is difficult to success.
(2) bring image mosaic problem is imaged in low Duplication
Due to the needs of aircraft fast reserve, the Duplication being imaged between the image of acquisition is lower, low Duplication meaning
Image overlapping region it is smaller, the effective coverage for proposing characteristic point is small, and the characteristic point quantity of extraction is necessarily less, and conventional method is special
The matching rate for levying point is smaller, causes the characteristic point obtained to less, there is the risk that can not resolve space conversion matrices.
(3) wide cut aerial image splices speed issue
Image snoop requests generate Image Intelligence in time, higher to the splicing rate request of image.But the picture of aerial image
Width is usually larger, and amount of images is more, consumes more time in feature extraction, matching search, wide format images synthesis, and
With increasing for stitching image quantity, the time of consumption is dramatically increased.Therefore, the time of feature extraction how is reduced, is accelerated special
The speed of sign matching search, improves the efficiency of wide format images synthesis, is the difficult point for solving aerial image and quickly splicing.
Summary of the invention
It is an object of the invention to the image split-joint methods for solving at this stage not to consider remote, big inclination angle, low overlapping
The distortion of aerial image caused by rate image-forming condition is serious, and the local feature of overlapping region differs greatly, characteristic point to be matched compared with
Image mosaic time longer problem, provides one kind and draws caused by few problem and aerial image picture is larger, quantity is more
Enter to be imaged auxiliary data calculating effective overlapping region and pass is utilized by the salient region Detection and Extraction key point of overlapping region
Key point constructs multiple dimensioned figure and based on matched method is schemed, and improves the probability of success and splicing of image mosaic under special image-forming condition
Speed realizes the quick joining method for tilting low Duplication aerial image greatly.
In order to obtain above-mentioned technical effect, the technical solution adopted by the present invention is that: one kind tilting greatly low Duplication aerial chart
The quick joining method of picture, it is characterised in that include the following steps:
The sequence aerial image of sensor front end camera acquisition and the imaging auxiliary data of each image, root are read first
The position distribution relationship that each image is calculated according to imaging auxiliary data, establishes image positional relationship matrix；According to image positional relationship
Matrix calculates the corresponding image in center of all image overlay areas, and the reference map as composite diagram coordinate system；Then pass through
Shortest path first searches for the adjacent map path between image and reference map, calculates the position on neighborhood paths between adjacent image
Relationship calculates the overlapping region between adjacent image according to positional relationship and Duplication, extracts overlay region in two images respectively
Significant characteristics region collection is established in the significant characteristics point in domain and region, the characteristic point or region normalizing that characteristic area is concentrated
Vertex is turned to, the non-directed graph of different scale is established with the point set of different adjacencies；It is carried out again with figure feature special in two images
The figure matching of sign point, obtains matching double points collection, is registrated adjacent two images one by one, calculate all adjacent two-by-two on neighborhood paths
Space conversion matrices between image transmit the space conversion matrices calculated between image and reference map using neighborhood paths；Fortune
Respective coordinates and use bilinear interpolation of the composograph pixel in source images, which are calculated, with space conversion matrices calculates composite diagram
The pixel value of picture obtains final stitching image.
The present invention has the advantages that compared with the prior art.
The present invention calculates the position distribution relationship between image using the imaging auxiliary data of aerial image, reduces image
The number being registrated two-by-two；By calculating reference map of the suitable image as the frame of reference, image and reference coordinate are reduced
Average viewing angle difference between system overcomes low Duplication imaging bring figure to reduce the averaged deformation error of composite diagram
As Bonding Problem.
The present invention calculates the adjacent map path between image and reference map using Shortest Path Searching, by extracting image
Conspicuousness key point establishes the multiple dimensioned figure of key point, reduces the time of feature extraction, accelerates the speed of characteristic matching search
Degree improves the efficiency of wide format images synthesis, reduces wide cut aerial image splicing speed difficulty.
In view of the problems of the existing technology, the imaging auxiliary data for introducing aerial image shortens big image mosaic to the present invention
Time reduces the spelling for tilting low Duplication image greatly using Stable distritation characteristic of the conspicuousness key point when multi-angle is imaged
Connect error；Introduce the robustness that figure matching technique improves image registration.By salient region Detection and Extraction key point, figure is solved
The few problem of the characteristic point as caused by low Duplication；By constructing large-scale structure feature using key point and being based on scheming matched
Method solves the problems, such as that the matching that differs greatly of local feature caused by remote, big inclination angle is imaged is more difficult；It is imaged by introducing
Auxiliary data calculates image overlapping region to reduce the feature extraction and matching time, solves the quick Bonding Problem of aerial image.Make
The space conversion matrices on adjacent map path between adjacent image are calculated one by one with figure matching process, are transmitted and are counted using neighborhood paths
Space conversion matrices between nomogram picture and reference map；Last use space transformation matrix calculates composograph pixel in source images
In respective coordinates and using bilinear interpolation calculate composograph pixel value, obtain final stitching image, solve long distance
From, big inclination angle image-forming condition bring image mosaic problem.
Detailed description of the invention
For a clearer understanding of the present invention, existing embodiment through the invention, referring concurrently to attached drawing, to describe this hair
It is bright, in which:
Fig. 1 is the image mosaic flow chart that the present invention tilts greatly low Duplication aerial image.
Fig. 2 is that the present invention calculates image positional relationship matrix schematic diagram.
Fig. 3 is the space conversion matrices flow chart between present invention calculating image and reference map.
Fig. 4 is the schematic diagram of present invention description graph structure feature.
Specific embodiment
Refering to fig. 1.According to the present invention, the sequence aerial image and every width figure of the acquisition of sensor front end camera are read first
The imaging auxiliary data of picture calculates the position distribution relationship of each image according to imaging auxiliary data, establishes image positional relationship square
Battle array；The corresponding image in center of all image overlay areas is calculated according to image positional relationship matrix, and as composite diagram coordinate
The reference map of system；Then the adjacent map path between image and reference map is searched for by shortest path first, calculates neighborhood paths
Positional relationship between upper adjacent image calculates the overlapping region between adjacent image according to positional relationship and Duplication, respectively
The significant characteristics point of overlapping region and region in two images are extracted, significant characteristics region collection is established, by characteristic area collection
In characteristic point or region be normalized to vertex, the non-directed graph of different scale is established with the point set of different adjacencies；Again with figure
Feature carries out the figure matching of characteristic point in two images, obtains matching double points collection, is registrated adjacent two images one by one, calculates neighbour
Connect the space conversion matrices on path between all adjacent images two-by-two, transmitted using neighborhood paths calculate image and reference map it
Between space conversion matrices；It is double that respective coordinates and use of the composograph pixel in source images are calculated with space conversion matrices
Linear interpolation calculates the pixel value of composograph, obtains final stitching image.
Refering to Fig. 2.Using camera primary optical axis and ground intersection point P as picture centre, establishing C-XYZ is indicated with aircraft center
C is expressed as the northeast day coordinate system of origin, and according to the spatial relationship that image I is imaged, auxiliary data when using imaging calculates figure
As the geographical coordinate at the center of I, the positional relationship matrix of image is then calculated according to auxiliary data, by each picture centre coordinate meter
The position distribution relationship of nomogram picture, establishes positional relationship matrix, wherein and o-xyz is earth axes, θ,Middle expression image I at
Pitch angle and azimuth as phase owner's optical axis CP, (X, Y, H) indicate that aircraft center is in ground o-xyz when image I imaging
Position in coordinate system.
Refering to Fig. 3.In calculating the space conversion matrix between each image and reference map, first on calculating neighborhood paths
Positional relationship between adjacent image；Then, the overlapping region between adjacent image is calculated according to positional relationship and Duplication；It connects
Respectively extract two images in overlapping region significant characteristics point and region, establish significant characteristics region collection；By feature
The characteristic point or region that region is concentrated are normalized to vertex, establish the undirected of different scale using the point set of different adjacencies
Figure, constructs the multiple dimensioned figure of significant characteristics；Then the structure feature of figure is described；It reuses figure feature and carries out spy in two images
The figure matching of sign point, obtains matching double points collection；The spatial alternation square between adjacent two images is calculated followed by matching double points
Battle array；The space conversion matrices on neighborhood paths between all adjacent images two-by-two are finally calculated, the transmitting of space reflection is passed through
Calculate the space conversion matrices between image and reference map.
Refering to Fig. 4.In the structure feature of description figure, with characteristic point p point in image and apart from nearest d adjacent features
Point k1,k2,…kd-1,kdBetween line constitute diagram p non-directed graph, using p Neighbor Points and p form side between it is suitable
Hour hands angle theta1,θ2,…θd-1,θdThe structure feature of figure is described.
In the quick splicing for tilting low Duplication aerial image greatly,
(1) sequence image and auxiliary data S1: are read
Read N width sequence image { In, n=1,2 ..., N and image imaging auxiliary dataAssuming that image center is overlapped with aircraft center.
(2) S2: image positional relationship matrix R is calculated
S21: according to image InSpatial relationship when imaging utilizes auxiliary data MenCalculate image InGeographical coordinate
S22: the positional relationship matrix of N width image is calculated, wherein θn,It indicates using aircraft center C as origin
The pitch angle of n-th width image imaging phase owner optical axis CP and azimuth, X in the C-XYZ coordinate system of northeast dayn,Yn,HnTable respectively
Show aircraft center position coordinates when the n-th width image is imaged in the o-xyz coordinate system of ground.
S221: initially setting up the positional relationship matrix R of a N × N, and be initialized as 0,
S222: the longitudinal center's distance calculated between image is DH=d × H/2, wherein image picture elements resolution ratio is d, figure
Picture size is W × H, and wherein W is picture traverse, and H is picture altitude.
S223: the y according to imagenCoordinate is sorted N width image from small to large, then according to distance interval DHIt divides
Gather at L, i.e. Cl={ In, l=1,2 ... L.
S224: the index of N width image is mapped into positional relationship matrix R
It is first depending on image latitude y-coordinate to map to image index in same a line of positional relationship matrix: to image In,
N=1,2 ..., N, if In∈Cl, then the l row that n is mapped to positional relationship matrix R is indexed, when multiple image belongs to together
One set ClWhen, multiple image is both mapped in l row, specific algorithm is as follows.
For the multiple image in l row, it is ranked up according to image longitude x coordinate, and be mapped to each of l row
In column: to corresponding image collection C in the l row of position relational matrix Rl, the x of foundation imagenCoordinate is arranged from small to large
Sequence, and from left to right map in the l row of matrix.
(3) S3: reference map I of the suitable figure as the frame of reference is calculatedc
According to the positional relationship R of imageijWith image InLatitude coordinates (xn,yn), calculate suitable reference map Ic.It examines
Consider position relational matrix R and represents the spatial relationship of image, when the mass center of chosen position relational matrix is as reference map, other
Average viewing angle difference between image and reference map is smaller, and the anamorphose for splicing synthesis is smaller, circular
get_row_index(Rij≠ 0) it indicates to meet RijCorresponding element line index in matrix R, get_ when ≠ 0
colum_index(Rij≠ 0) it indicates to meet RijColumn index of the corresponding element in matrix R when ≠ 0.Then reference map isCorresponding image, i.e.,
(4) S4: image I is calculatednWith reference map IcBetween adjacent map path
Calculate image InWith reference map IcBetween adjacent shortest path, the image passed through is followed successively byUsing the index of these images as image InTo reference map IcBetween transition matrix transmission path ln,
I.e.
ln=(n0,n1,n2,…,nM,c),nm∈{1,2,…,N}
Wherein M indicates path lnLength, index n0Correspondence image In, index c and correspond to benchmark image Ic, index nmIt is corresponding
Image and index nm+1Corresponding image is adjacent.
(5) S5: image I is calculatednWith reference map IcBetween space conversion matrix Hn。
1) S51: path l is calculatednOn adjacent imageWithBetween positional relationship
The direction the y overlapping region Shang You is indicated in positional relationship matrix R between the corresponding image of adjacent rows, is remembered between them
Positional relationship is right_left or left_right, indicates the direction the x overlapping region Shang You between the corresponding image of adjacent column,
Remember that the positional relationship between them is top_bottom or bottom_top, computation rule is as follows: assuming that imageWith
Corresponding element is respectively in RWith
If i2=i1+ 1, indicate imageWithBetween positional relationship be
If i1=i2+ 1, indicate imageWithBetween positional relationship be
If j2=j1+ 1, indicate imageWithBetween positional relationship be
If j1=j2+ 1, indicate imageWithBetween positional relationship be
2) S52: l is calculatednUpper adjacent imageWithBetween overlapping imageWith
FoundationWith adjacent imageBetween positional relationshipWith Duplication ρ, calculate in image
Overlapping region, calculation method is as follows:
IfThen
IfThen
IfThen
IfThen
3) S53: the significant characteristics region collection of overlapping image is extracted
Respectively to overlapping imageWithThe significant characteristics of extraction as follows region collectionWith
S531: using filter F to overlapping area imageIt carries out convolution and extracts feature fi,
fi=O*F, i ∈ O
S532: to each of overlapping region O point i, the significance measure S of i is calculated(i)
Wherein N (i) is 8 Neighbourhood sets of point i, DijFor the feature difference of point i and point j
D(i,j)=| fi-fj|2
Wherein e(i,j)Indicate the irrelevance of human eye retina, the distance dependent with block i and j, wherein C (f, e) is that vision is quick
Acutance.
C (f, e)=1/T (f, e)
T (f, e) is contrast threshold in formula
F is the spatial frequency of image, and α is spatial frequency constant, and e is Eccentricity, e2It is the half-resolution degree of eccentricity,
T0It is minimum contrast threshold value, empirical valueα=0.106, e2=2.3.
S533: the significant characteristics S of each point i is calculated(i)Afterwards, following threshold process is done:
Work as S(i)< threshold, then S(i)=0.
Work as S(i)> threshold, then S(i)=S(i)。
By the significant characteristics point S of connection(i)It merges, non-interconnected conspicuousness feature is rejected, and salient region is obtained
Collect A={ S(p), p=1,2 ..., P.
Respectively to overlapping regionWithThe step of carrying out S441 to S443 extracts salient region collection
4) S54: in the multiple dimensioned figure of building significant characteristics,
Respectively to overlapping imageWithThe salient region collection of middle extractionFollowing processing is done, building is significant
The multiple dimensioned figure of property feature
S541: the salient region S in the notable feature region collection A of the overlapping region O of image I is taken(p)∈ A, p=1,
2 ..., vertex c of the center of P as figure(p)。
MpIndicate characteristic area S(p)The number at midpoint,Indicate coordinate of the pixel i in image I.It obtains
Overlapping region O vertex set C={ c(p), p=1,2 ..., P }.
S542: multiple dimensioned neighbour is constructed using vertex set C and schemes G.
S5421: vertex c is calculated using k nearest neighbor algorithm(p)The kd of ∈ CthNeighbor vertices set(k-1) dthNeighbour top
Point setThen vertex c(p)DkNeighbour's point setFor
Neighbour's point setIn have d fixed point.
S5422: it utilizesConstruct vertex c(p)Neighbour's scale be dkRadiation diagram
S5423: scale set D={ d is definedk, k=1,2,3 ..., K }, wherein dk=[(k-1) d, kd].Difference is taken respectively
Scale dk∈ D repeats step S4521 to S4522, constructs vertex c(p)Multiple dimensioned neighbour figure
S5424: all vertex c in opposite vertexes collection C(p)∈ C, p=1,2,3 ..., P do S4521 to S4523 processing, building
The neighbour of vertex set C schemes G={ G(p), p=1,2,3 ..., P }.
Respectively to overlapping imageWithStep S451 to S452 processing is done, the multiple dimensioned figure of significant characteristics is obtained
5) structure feature of figure S55: is described
Respectively to overlapping imageWithSignificant characteristics building figureWithFollowing method is done to retouch
It states, obtains the structure feature of figureWith
S551: point c is utilized(p)DkThe neighbour of scale schemesIt calculates by point c(p)With vertex kd-1The side of connection with
Point c(p)With vertex kdThe angle on the side of connection isWhereinIn order to enable feature has rotational invariance, by d
A angleIt carries out forming point c after sorting from small to large(p)Feature vector.
S552: to point c(p)Multiple dimensioned neighbour's figureS461 processing is done, point c is obtained(p)It is more
Scalogram structure feature
S553: to all vertex c(p)∈ C, p=1,2 ..., the neighbour of P schemes G(p)∈ G does S461 to S462 processing, obtains
Structure feature F={ the F of the figure G of significant characteristics(p), p=1,2 ..., P.
Respectively to overlapping imageWithSignificant characteristics building figureWithIt is at S461 to S463
Reason, obtains the graph structure feature of significant characteristicsWith
6) S56: characteristic matching
S561: feature point set is utilizedIt establishes
Kd-tree search tree；
S5611: d dimensional feature collection is calculatedIn have it is most generous
The intrinsic dimensionality k of difference*, as left and right subtree divide dimension,WhereinIt indicatesKthiWei Te
Levy the variance of component.
S5612: by the kth of point all in set*Dimensional feature componentBe ranked up, take medianMake
For kth*Divide value in dimension
S5613: k is utilized*WithIt establishes root node to divide feature set, by kth*Dimensional feature componentCharacteristic point as right subtree, by kth*Dimensional feature component
Characteristic point as left subtree.
S5614: carrying out S4711, S4712, S4713 recursive operation to the point set in left subtree, right subtree respectively, until left
Right subtree is all regardless of subdivided.
S562: characteristic point is found in kd-tree search tree using BBF methodNearest neighbor point
With secondary Neighbor Points
S5621: for pointIt searches for and compares since the root node in kd-tree search tree, and tied in search tree
PointCompare, ifThen select to the right sub- rightChild tree search
Otherwise rope selects to search for left subtree leftChild, and by not selected subtree subtree={ rightChild or then
LeftChild } and its root node withBetween characteristic distanceIt is saved in minimum
In Priority Queues Q, queue Q is arranged according to the sequence of distance D from small to large.
S5622: recurrence executes S5621 operation, until then search calculates a little to leaf nodeIn leaf node
The characteristic point of preservationThe distance between
The possible more than one of the characteristic point that wherein leaf node saves, saves wherein apart from the smallest two characteristic points As nearest neighbor point and time Neighbor Points and corresponding two distances
With
S5623: the corresponding subtree subtree=Q (0) of minimum range in selection queue Q executes S5621 and S5622 behaviour
Make carry out retrospective search, when calculated new nearest neighbor pointDistance than original nearest neighbor distance dis ' hour,
Original nearest neighbor point is replaced with into time Neighbor Points i.e.Nearest neighbor distance originally replace with time neighbour away from
From i.e. dis "=dis ', new nearest neighbor point is replaced into original nearest neighbor point i.e.New nearest neighbor distance
Replace original nearest neighbor distance i.e. dis '=dis " '；When calculated new secondary Neighbor PointsDistance it is closer than original time
Neighborhood distance dis " hour, new secondary Neighbor Points, which are replaced original secondary Neighbor Points, isNew secondary nearest neighbor distance
The original secondary nearest neighbor distance, that is, dis "=ds " " of replacement.The subtree Q (0) recalled in queue is deleted after the completion of searching.
S5624: repeating step S5623 up to backtracking step is greater than threshold value or queue Q is sky, is finally obtained a littleNearest neighbor pointWith secondary Neighbor Points
S563: judgement is when satisfaction
When, it is believed that pointFor pointMatch point, note matching double points be
S564: rightIn all characteristic pointsRepeat step S472 and
S473 obtains two width adjacent imagesWithThe matching double points set of image
7) S57: matching double points set is utilizedIt is calculated using random sampling consistency (RANCAC) method adjacent
ImageWithBetween space conversion matrices
S571: from matching double points collectionIn randomly select four points to collection The transformation matrix H between four points pair is calculated, and calculates the interior point set of transformation matrices H.
Overdetermined equation is solved using least square method, obtains transformation matrix H
S572: executing step S571N times, and the interior point set for recording the interior point maximum single sample of number is matched as consistency
Point is to collection
S573: consistency matching double points collection is utilizedCalculate the space conversion matrices between matching double points
8) S58: image I is calculatednWith reference map IcBetween transition matrix
To path ln=(n0,n1,n2,…,nM, c) on adjacent image two-by-twoWithS51 to S57 processing is executed,
Calculate separately out the transformation matrix between all adjacent imagesThen image InWith reference map IcIt
Between transition matrix HnFor
WhereinFor unit matrix.
To each image InThe processing of step S4, S5 is done respectively, calculates image InWith reference map IcBetween space conversion matrices
Hn, n=1,2 ..., N.
(6) S6: image synthesis
S61: each image I is calculatednMapping area in the composite image four angles coordinate Mapn={ (tlxn,
tlyn),(trxn,tryn),(brxn,bryn),(blxn,blyn), wherein tlxn=c, tlyn=f
Wherein WnFor the width of ground n width image, HnFor the height of the n-th figure.
S62: according to all mapping area { Mapn, calculate the size of composograph
Lx=min { tlxn,blxn}
Ty=min { tlyn,tryn}
Rx=max { trxn,brxn}
By=max { blyn,bryn}
Then the size of composograph is
Width=rx-lx
Height=by-ty
S63: H is calculatednInverse matrix Hinvn
S64: mapping area Map in composograph is calculated using bilinear interpolationnIn pixel value
For mapping area MapnIn pixel (x, y) ∈ Mapn, corresponding original image InMidpoint is
Xint=floor (x '), x "=x '-xint, yint=floor (y '), y "=y '-yint, wherein floor ()
It indicates to be rounded downwards, then the corresponding pixel value of pixel (x, y) is in composograph
F (x, y)=x " * y " * I (xint, yint)+(1-x ") * y " * I (xint+1, yint)+x " * (1-y ") * I (xint,
yint+1)+(1-x″)*(1-y″)*I(xint+1,yint+1)。
Claims (9)
1. the quick joining method that one kind tilts greatly low Duplication aerial image, it is characterised in that include the following steps:
First read sensor front end camera acquisition sequence aerial image and each image imaging auxiliary data, according at
As auxiliary data calculates the position distribution relationship of each image, image positional relationship matrix is established；According to image positional relationship matrix
Calculate the corresponding image in center of all image overlay areas, and the reference map as composite diagram coordinate system；Then by most short
Routing algorithm searches for the adjacent map path between image and reference map, calculates the position on neighborhood paths between adjacent image and closes
System calculates the overlapping region between adjacent image according to positional relationship and Duplication, extracts overlapping region in two images respectively
Significant characteristics point and region, establish significant characteristics region collection, by characteristic area concentrate characteristic point or region normalize
For vertex, the non-directed graph of different scale is established with the point set of different adjacencies；Feature in two images is carried out with figure feature again
The figure matching of point, obtains matching double points collection, is registrated adjacent two images one by one, calculate all neighbor maps two-by-two on neighborhood paths
Space conversion matrices as between transmit the space conversion matrices calculated between image and reference map using neighborhood paths；With
Space conversion matrices calculate respective coordinates and use bilinear interpolation of the composograph pixel in source images and calculate composograph
Pixel value, obtain final stitching image.
2. tilting the quick joining method of low Duplication aerial image greatly as described in claim 1, it is characterised in that: with camera
The focus of primary optical axis is the center P of image, establishes the northeast day coordinate system C-XYZ that origin is expressed as with aircraft center C, according to
The spatial relationship of image I imaging, auxiliary data when using imaging calculates the geographical coordinate at the center of image I, then by each figure
Inconocenter coordinate calculates the position distribution relationship of image, establishes positional relationship matrix, wherein and o-xyz is earth axes, θ,
Indicate pitch angle and the azimuth of image I imaging phase owner optical axis CP, (X, Y, H) indicates aircraft center when image I imaging
Position in the o-xyz coordinate system of ground.
3. tilting the quick joining method of low Duplication aerial image greatly as claimed in claim 2, it is characterised in that: with image
Middle characteristic point p point with apart from nearest d adjacent characteristic point k1, k2... kd-1, kdBetween line form p point non-directed graph, make
With the angle theta clockwise between the side of Neighbor Points and the p composition of p point1, θ2... θd-1, θdThe structure feature of figure is described.
4. tilting the quick joining method of low Duplication aerial image greatly as described in claim 1, it is characterised in that: inclining greatly
In the quick splicing of oblique low Duplication aerial image, N width sequence image { I is readn, n=1,2 ..., N } and the imaging of image it is auxiliary
Help dataAccording to image InSpatial relationship when imaging utilizes auxiliary data MenCalculate figure
As InGeographical coordinate
Calculate the positional relationship matrix R of N width image, wherein θn、It indicates using aircraft center C as the northeast of origin day
The pitch angle of n-th width image imaging phase owner optical axis CP and azimuth, X in C-XYZ coordinate systemn, Yn, HnIt is illustrated respectively in ground
Aircraft center position coordinates when the n-th width image is imaged in the o-xyz coordinate system of face, N indicate sequence image quantity to be spliced.
5. tilting the quick joining method of low Duplication aerial image greatly as claimed in claim 4, it is characterised in that: establish one
The positional relationship matrix R of a N × N, and it is initialized as 0, the longitudinal center's distance calculated between image is DH=d × H/2,
In, image picture elements resolution ratio is d, and image size is W × H, wherein W is image pixel width, H image pixel height；According to figure
The latitude y of picturenCoordinate is sorted N width image from small to large, then according to distance interval DHIt is divided into L set Cl=
{In, l=1,2 ... L, to image In, n=1,2 ..., N, if In∈Cl, then indexed n and be mapped to relational matrix R's
L row, when multiple image belongs to the same set ClWhen, multiple image is both mapped in l row；For more in l row
It is ranked up by width image according to image longitude x coordinate, and is mapped in each column of l row: to the l row of relational matrix R
In corresponding image collection Cl, the x of foundation imagenCoordinate is sorted from small to large, and from left to right maps to the l of matrix
In row.
6. tilting the quick joining method of low Duplication aerial image greatly as claimed in claim 5, it is characterised in that: calculate figure
As InWith reference map IcBetween adjacent shortest path, the image of process is followed successively byBy the rope of these images
Draw as image InTo reference map IcBetween transition matrix transmission path ln, ln=(n0, n1, n2..., nM, c), nm∈ 1,
2 ..., N }, wherein M indicates path lnLength, index n0Correspondence image In, index c and correspond to benchmark image Ic, index nmIt is corresponding
Image and index nm+1Corresponding image is adjacent.
7. tilting the quick joining method of low Duplication aerial image greatly as described in claim 1, it is characterised in that: extracting
When being overlapped the significant characteristics region collection of image, respectively to overlapping imageWithExtraction conspicuousness as follows
Characteristic area collectionWithUsing filter F to overlapping area imageIt carries out convolution and extracts spy
Levy fi, fi=O*F, i ∈ O calculate the significance measure S of i then to each of overlapping region O point i(i), S(i)=
∑I ≠ j, j ∈ N (i)C (f, e(i, j))D(i, j)；Wherein, N (i) is 8 Neighbourhood sets of point i, DijFor the feature difference D of point i and point j(i, j)
=| fi-fj|2, e(i, j)Indicate that the irrelevance of human eye retina, the distance dependent with block i and j, C (f, e) are visual acuity, e
It is Eccentricity.
8. tilting the quick joining method of low Duplication aerial image greatly as described in claim 1, it is characterised in that: constructing
In the multiple dimensioned figure of significant characteristics, respectively to overlapping imageWithThe salient region collection of middle extractionFollowing processing is done, building is significant
The multiple dimensioned figure of property featureTake the salient region in the notable feature region collection A of the overlapping region O of image I
S(p)Vertex c of the center of ∈ A, p=1,2 ..., P as figure(p),?
To overlapping region O vertex set C={ c(p), p=1,2 ..., P }, multiple dimensioned neighbour, which is constructed, using vertex set C schemes G, MpIndicate special
Levy region S(p)The number at midpoint,Indicate coordinate of the pixel i in image I.
9. tilting the quick joining method of low Duplication aerial image greatly as described in claim 1, it is characterised in that: in figure
In the feature description matched, respectively to overlapping imageWithSignificant characteristics building figureWithIt does as follows
Processing, obtains the structure feature of figureWithTake point c(p)With vertex kd-1The side of connection and point c(p)With vertex kdConnection
The angle on side isAnd it carries out forming point c after sorting from small to large(p)Feature vector
Obtain point c(p)Multiple dimensioned graph structure featureTo all vertex c(p)∈ C, p=1,
The neighbour of 2 ..., P schemes G(p)∈ G processing obtains the structure feature F={ F of the figure G of significant characteristics(p), p=1,2 ..., P.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610727071.0A CN106373088B (en) | 2016-08-25 | 2016-08-25 | The quick joining method of low Duplication aerial image is tilted greatly |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610727071.0A CN106373088B (en) | 2016-08-25 | 2016-08-25 | The quick joining method of low Duplication aerial image is tilted greatly |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106373088A CN106373088A (en) | 2017-02-01 |
| CN106373088B true CN106373088B (en) | 2019-08-06 |
Family
ID=57878445
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610727071.0A Active CN106373088B (en) | 2016-08-25 | 2016-08-25 | The quick joining method of low Duplication aerial image is tilted greatly |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106373088B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106920207B (en) * | 2017-03-13 | 2020-05-19 | 哈尔滨工程大学 | Method for real-time calculation of projection point of image plane of space platform based on hardware implementation |
| CN107563964B (en) * | 2017-08-22 | 2020-09-04 | 长光卫星技术有限公司 | Rapid splicing method for large-area-array sub-meter-level night scene remote sensing images |
| JP6962812B2 (en) * | 2017-12-26 | 2021-11-05 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co., Ltd | Information processing equipment, flight control instruction method, program, and recording medium |
| CN108537732B (en) * | 2018-04-10 | 2021-11-02 | 福州大学 | PCA-SIFT-based rapid image splicing method |
| CN109061049A (en) * | 2018-06-21 | 2018-12-21 | 河南天腾测绘科技有限公司 | A kind of gas data monitoring method of all region covering |
| CN109660732B (en) * | 2019-01-04 | 2021-07-23 | Oppo广东移动通信有限公司 | Electronic equipment and mobile platform |
| CN109934093B (en) * | 2019-01-21 | 2021-03-30 | 创新奇智(南京)科技有限公司 | Method for identifying goods on shelf, computer readable medium and identification system |
| CN109948700B (en) * | 2019-03-19 | 2020-07-24 | 北京字节跳动网络技术有限公司 | Method and device for generating feature map |
| CN111815511A (en) * | 2019-04-11 | 2020-10-23 | 上海集森电器有限公司 | Panoramic image splicing method |
| CN111223133B (en) * | 2020-01-07 | 2022-10-11 | 上海交通大学 | Registration method of heterogeneous images |
| CN115266282B (en) * | 2021-04-30 | 2023-07-21 | 武汉兰丁云医学检验实验室有限公司 | Pathological image automatic identification method based on combination of HE (HE) dyeing, ki67 and P16 |
| CN114565516B (en) * | 2022-03-03 | 2024-05-14 | 上海核工程研究设计院股份有限公司 | Sensor data fusion containment surface area robust splicing method |
| CN118037999A (en) * | 2024-04-10 | 2024-05-14 | 时代新媒体出版社有限责任公司 | Interactive scene construction method and system based on VR thinking teaching |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090108495A (en) * | 2008-04-11 | 2009-10-15 | 주식회사 디오텍 | Panorama Image Generating Method for Portable Terminal |
| CN103927759A (en) * | 2014-04-30 | 2014-07-16 | 武汉大学 | Automatic cloud detection method of aerial images |
| CN105678687A (en) * | 2015-12-29 | 2016-06-15 | 天津大学 | Stereo image stitching method based on content of images |
| CN105844587A (en) * | 2016-03-17 | 2016-08-10 | 河南理工大学 | Low-altitude unmanned aerial vehicle-borne hyperspectral remote-sensing-image automatic splicing method |
| CN105869120A (en) * | 2016-06-16 | 2016-08-17 | 哈尔滨工程大学 | Image stitching real-time performance optimization method |
-
2016
- 2016-08-25 CN CN201610727071.0A patent/CN106373088B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090108495A (en) * | 2008-04-11 | 2009-10-15 | 주식회사 디오텍 | Panorama Image Generating Method for Portable Terminal |
| CN103927759A (en) * | 2014-04-30 | 2014-07-16 | 武汉大学 | Automatic cloud detection method of aerial images |
| CN105678687A (en) * | 2015-12-29 | 2016-06-15 | 天津大学 | Stereo image stitching method based on content of images |
| CN105844587A (en) * | 2016-03-17 | 2016-08-10 | 河南理工大学 | Low-altitude unmanned aerial vehicle-borne hyperspectral remote-sensing-image automatic splicing method |
| CN105869120A (en) * | 2016-06-16 | 2016-08-17 | 哈尔滨工程大学 | Image stitching real-time performance optimization method |
Non-Patent Citations (1)
| Title |
|---|
| 基于ASIFT算法的低重叠度无人机影像拼接方法;宋耀金 等;《遥感技术与应用》;20150831;第30卷(第4期);725-730 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106373088A (en) | 2017-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106373088B (en) | The quick joining method of low Duplication aerial image is tilted greatly | |
| CN108596101B (en) | Remote sensing image multi-target detection method based on convolutional neural network | |
| CN108765298A (en) | Unmanned plane image split-joint method based on three-dimensional reconstruction and system | |
| US9798928B2 (en) | System for collecting and processing aerial imagery with enhanced 3D and NIR imaging capability | |
| CN108596108B (en) | Aerial remote sensing image change detection method based on triple semantic relation learning | |
| CN111047695B (en) | Method for extracting height spatial information and contour line of urban group | |
| CN113222820B (en) | Pose information-assisted aerial remote sensing image stitching method | |
| CN109376641B (en) | Moving vehicle detection method based on unmanned aerial vehicle aerial video | |
| CN109900274B (en) | Image matching method and system | |
| CN108320304A (en) | A kind of automatic edit methods and system of unmanned plane video media | |
| CN112857356B (en) | Unmanned aerial vehicle water body environment investigation and air route generation method | |
| CN112946679B (en) | Unmanned aerial vehicle mapping jelly effect detection method and system based on artificial intelligence | |
| Zhang et al. | Aerial orthoimage generation for UAV remote sensing | |
| CN109671109A (en) | Point off density cloud generation method and system | |
| Nguyen et al. | Coarse-to-fine registration of airborne LiDAR data and optical imagery on urban scenes | |
| CN116740288A (en) | Three-dimensional reconstruction method integrating laser radar and oblique photography | |
| Comandur et al. | Semantic labeling of large-area geographic regions using multiview and multidate satellite images and noisy OSM training labels | |
| CN109544455B (en) | Seamless fusion method for ultralong high-definition live-action long rolls | |
| KR102587445B1 (en) | 3d mapping method with time series information using drone | |
| CN109883400A (en) | Fixed station Automatic Targets and space-location method based on YOLO-SITCOL | |
| Liu et al. | An improved APAP algorithm via line segment correction for UAV multispectral image stitching | |
| CN113554754A (en) | Indoor positioning method based on computer vision | |
| CN114199250A (en) | Scene matching navigation method and device based on convolutional neural network | |
| Afriansyah et al. | Image Mapping Detection of Green Areas Using Speed Up Robust Features | |
| CN110930510A (en) | Urban space three-dimensional reconstruction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |