CN109978760A

CN109978760A - A kind of image split-joint method and device

Info

Publication number: CN109978760A
Application number: CN201711445096.2A
Authority: CN
Inventors: 姚佳宝; 王莉
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2017-12-27
Filing date: 2017-12-27
Publication date: 2019-07-05
Anticipated expiration: 2037-12-27
Also published as: CN109978760B

Abstract

The embodiment of the present application provides a kind of image split-joint method and device, method includes: to obtain basic stitching image and image to be spliced, extract the characteristic point of basic stitching image and image to be spliced, determine multiple matching characteristic points pair, according to determining multiple matching characteristic points pair, calculate the homography matrix of each image to be spliced, according to the homography matrix of each image to be spliced, determine the overlapping region of each image to be spliced, according to the re-projection error of matching characteristic point pair, first grid foundation is carried out to the overlapping region of each image to be spliced, for every one first grid of each image to be spliced, according to the weight of first grid, adjust the homography matrix of the image to be spliced, obtain the homography matrix of first grid, according to the homography matrix of the homography matrix of each image to be spliced and every one first grid, splice basic spliced map Picture and at least an image to be spliced.Using the embodiment of the present application, solve the problems, such as the dislocation of image mosaic and ghost image.

Description

A kind of image split-joint method and device

Technical field

This application involves technical field of image processing, more particularly to a kind of image split-joint method and device.

Background technique

In order to solve the problems, such as that one-shot camera field angle is narrow, more camera lens panoramic cameras come into being.More camera lenses Panoramic camera can acquire simultaneously the multiway images of different directions, and be spliced, and the panoramic picture at big visual angle is obtained.

Currently, the joining method of image includes: to obtain an at least image to be spliced and basic stitching image, wherein to Stitching image is converted based on basic stitching image, such as by image to be spliced into the coordinate system of basic stitching image；Really The matching characteristic point pair of every two images in fixed image to be spliced and basic stitching image；According to the coordinate of matching characteristic point pair, Determine that each image to be spliced is directed to the homography matrix of basic stitching image；Splice image to be spliced and basis according to homography matrix Stitching image.

In practical application, an image is with depth information, and different objects are located at different planes, depth letter in image Breath is different, and if a people station is before a building in image, the depth information of people and building is different in this image, this is just Cause feature distribution in image uneven.In the picture in the non-uniform situation of feature distribution, spelled according to single homography matrix Map interlinking picture, it is likely that stitching image is caused the problem of dislocation and ghost image occur.

Summary of the invention

The embodiment of the present application is designed to provide a kind of image split-joint method and device, to solve the dislocation of image mosaic And ghost problems.Specific technical solution is as follows:

On the one hand, the embodiment of the present application provides a kind of image split-joint method, which comprises

Obtain basic stitching image and an at least image to be spliced；

Extract the characteristic point of the basic stitching image and at least one image to be spliced；

Characteristic point progress to every two images in an at least image to be spliced and the basic stitching image Match, determines multiple matching characteristic points pair；

According to the coordinate of determining multiple matching characteristic point centering feature points, calculates each image to be spliced and be directed to the base The homography matrix of plinth stitching image；

According to the homography matrix of each image to be spliced, the overlapping region of each image to be spliced is determined；

According to the re-projection error of matching characteristic point pair, the first grid is carried out to the overlapping region of each image to be spliced and is built It is vertical；

The figure to be spliced is adjusted according to the weight of first grid for every one first grid of each image to be spliced The homography matrix of picture obtains the homography matrix of first grid；

According to the homography matrix of the homography matrix of each image to be spliced and every one first grid, splice the basis splicing Image and an at least image to be spliced.

It is described to extract the basic stitching image and an at least figure to be spliced in one embodiment of the application The step of characteristic point of picture, comprising:

It is extracted according to SIFT (Scale-invariant feature transform, Scale invariant features transform) algorithm Characteristic point in an at least image to be spliced and the basic stitching image.

It is described to an at least image to be spliced and the basic stitching image in one embodiment of the application In the characteristic points of every two images matched, determine the step of multiple matching characteristic points pair, comprising:

For each characteristic point of each image in an at least image to be spliced and the basic stitching image, Calculate the Euclidean distance between the description operator of the characteristic point of other images and the description operator of this feature point；Other described images are Image in an at least image to be spliced and the basic stitching image in addition to the image；By this feature point and it is European away from Characteristic point corresponding from the smallest description operator constitutes matching characteristic point pair.

In one embodiment of the application, in an at least image to be spliced and the basic stitching image Before the characteristic point of every two images is matched, the method also includes:

For each characteristic point of extraction, 360 degree of gradient histograms are established using the gradient information of this feature vertex neighborhood pixel Figure obtains the description operator of this feature point spatially；Normalize the description operator of this feature point.

In one embodiment of the application, the homography matrix according to each image to be spliced is determined each wait spell The step of overlapping region of map interlinking picture, comprising:

For each image to be spliced, the overlapping region of the image to be spliced is determined using following steps:

Determine the abscissa and vertical seat of four vertex of the image to be spliced in the plane in the basic stitching image institute Mark；

Determine four vertex of matching image the basic stitching image abscissa and ordinate in the plane；Institute Stating matching image is the feature point group with the image to be spliced into the image where the characteristic point of matching characteristic point pair；

Determine the first abscissa, the second abscissa, the first ordinate and the second ordinate；First abscissa is should be to Maximum value in the minimum abscissa on four vertex of minimum abscissa and the matching image on four vertex of stitching image；It is described Second abscissa is the horizontal seat of maximum on four vertex of maximum abscissa and the matching image on four vertex of image to be spliced Minimum value in mark；First ordinate is the minimum ordinate and the matching image four on four vertex of image to be spliced Maximum value in the minimum ordinate on a vertex；Second ordinate is the maximum ordinate on four vertex of image to be spliced With the minimum value in the maximum ordinate on four vertex of matching image；

By first abscissa, second abscissa, first ordinate and second ordinate, determining should The overlapping region of image to be spliced and the matching image.

In one embodiment of the application, the re-projection error according to matching characteristic point pair, to each to be spliced The overlapping region of image carries out the step of the first grid is established, comprising:

From the re-projection error of the corresponding all matching characteristic points pair of each image to be spliced, obtains maximum re-projection and miss Difference and minimum re-projection error；

According to maximum re-projection error and minimum re-projection error, first is carried out to the overlapping region of each image to be spliced Grid is established.

It is described according to maximum re-projection error and minimum re-projection error in one embodiment of the application, to each The overlapping region of image to be spliced carries out the step of the first grid is established, comprising:

For each image to be spliced, according to the following formula, the first grid is carried out to the overlapping region of the image to be spliced It establishes:

Wherein, △ h_iFor the throwing again of matching characteristic point pair where characteristic point in i-th of first grids of the image to be spliced Shadow error, △ h_minFor the corresponding minimum re-projection error of the image to be spliced of acquisition, △ h_maxFor the figure to be spliced of acquisition As corresponding maximum re-projection error, N is positive integer；The value range of i is 0~N-1.

In one embodiment of the application, every one first grid for each image to be spliced, according to this The step of weight of one grid adjusts the homography matrix of the image to be spliced, obtains the homography matrix of first grid, comprising:

First grid is determined according to the weight of first grid for every one first grid of each image to be spliced The weight of matching characteristic point pair where interior characteristic point；

First grid is determined according to the weight of first grid for every one first grid of each image to be spliced The weight of matching characteristic point pair where outer characteristic point；

For every one first grid of each image to be spliced, according to matching characteristic point where characteristic point in first grid Pair the outer characteristic point of weight and first grid where matching characteristic point pair weight, adjust the image to be spliced singly answers square Battle array, using homography matrix adjusted as the homography matrix of first grid.

In one embodiment of the application, the weight according to first grid determines feature in first grid Where point the step of the weight of matching characteristic point pair, comprising:

For each characteristic point in first grid, matching characteristic point pair where determining this feature point by following formula Weight:

Wherein, w_*For the weight of matching characteristic point pair where this feature point, γ is the weight of first grid, the value of γ Range is the number of 0~1, the M characteristic point that be in first grid include in matching characteristic point pair.

In one embodiment of the application, the weight according to first grid determines the outer feature of first grid Where point the step of the weight of matching characteristic point pair, comprising:

For each characteristic point outside first grid, matching characteristic point pair where determining this feature point by following formula Weight:

Wherein, w'_*For the weight of matching characteristic point pair where this feature point, t_*For the coordinate of this feature point, t_iFor this I-th includes the coordinate in matching characteristic point centering feature point in one grid.

In one embodiment of the application, the homography matrix and every one first grid according to each image to be spliced Homography matrix, the step of splicing the basic stitching image and an at least image to be spliced, comprising:

According to parameter preset, the second grid foundation is carried out to the Non-overlapping Domain of each image to be spliced；The non-overlap Region is the region in each image to be spliced in addition to overlapping region；

For every one second grid of each image to be spliced, according to the power of other adjacent the second grids of second grid Value ratio adjusts the homography matrix of the image to be spliced, obtains the homography matrix of second grid；

According to the homography matrix of the homography matrix of every one first grid and every one second grid, splice the basic spliced map Picture and an at least image to be spliced.

In one embodiment of the application, every one second grid for each image to be spliced, according to this The weighting ratio of other adjacent the second grids of two grids, adjusts the homography matrix of the image to be spliced, obtains second grid Homography matrix the step of, comprising:

For every one second grid of each image to be spliced, according to the power of other adjacent the second grids of second grid Value ratio determines the weight of other adjacent the second grids of second grid；

For every one second grid of each image to be spliced, according to the power of other adjacent the second grids of second grid Value, adjusts the homography matrix of the image to be spliced, using homography matrix adjusted as the homography matrix of second grid.

In one embodiment of the application, the weight ratio of other second grids adjacent according to second grid Example, the step of determining the weight of other adjacent the second grids of second grid, comprising:

According to the following formula, the weight of other adjacent the second grids of second grid is determined:

Wherein, w_jFor the weight of j-th adjacent of other the second grid of second grid, Q_jIt is adjacent for second grid The weighting ratio of j-th of other the second grid, Q_iFor the weighting ratio of i-th adjacent of other the second grid of second grid, S For the total number of other second grids adjacent with second grid.

In one embodiment of the application, every one second network it is equal in magnitude.

Two aspects, the embodiment of the present application provide a kind of image splicing device, and described device includes:

Acquiring unit, for obtaining basic stitching image and at least an image to be spliced；

Extraction unit, for extracting the characteristic point of the basic stitching image and at least one image to be spliced；

Matching unit, for every two images in an at least image to be spliced and the basic stitching image Characteristic point is matched, and determines multiple matching characteristic points pair；

Computing unit calculates each to be spliced for the coordinate according to determining multiple matching characteristic point centering feature points Homography matrix of the image for the basic stitching image；

Determination unit determines the overlay region of each image to be spliced for the homography matrix according to each image to be spliced Domain；

Unit is established, for the re-projection error according to matching characteristic point pair, to the overlapping region of each image to be spliced Carry out the first grid foundation；

Adjustment unit is adjusted for every one first grid for each image to be spliced according to the weight of first grid The homography matrix of the whole image to be spliced, obtains the homography matrix of first grid；

Concatenation unit, for spelling according to the homography matrix of each image to be spliced and the homography matrix of every one first grid Connect the basic stitching image and an at least image to be spliced.

In one embodiment of the application, the extraction unit is specifically used for:

The characteristic point in an at least image to be spliced and the basic stitching image is extracted according to SIFT algorithm.

In one embodiment of the application, the matching unit is specifically used for:

In one embodiment of the application, described device further include:

Obtaining unit is established for each characteristic point for extraction using the gradient information of this feature vertex neighborhood pixel 360 degree of histogram of gradients obtain the description operator of this feature point spatially；

Normalization unit normalizes the description operator of this feature point for each characteristic point for extraction.

In one embodiment of the application, the determination unit is specifically used for for each image to be spliced, and determining should The overlapping region of image to be spliced, comprising:

First determines subelement, for determining that four vertex of the image to be spliced are put down where the basic stitching image Abscissa and ordinate on face；

Second determines subelement, for determine four vertex of matching image the basic stitching image institute in the plane Abscissa and ordinate；The matching image is the characteristic point with the feature point group of the image to be spliced at matching characteristic point pair The image at place；

Third determines subelement, for determining the first abscissa, the second abscissa, the first ordinate and the second ordinate； First abscissa is the minimum on four vertex of minimum abscissa and the matching image on four vertex of image to be spliced Maximum value in abscissa；Second abscissa is the maximum abscissa and matching figure on four vertex of image to be spliced Minimum value in the maximum abscissa on four vertex of picture；First ordinate is that the minimum on four vertex of image to be spliced is vertical Maximum value in the minimum ordinate on four vertex of coordinate and the matching image；Second ordinate is the image to be spliced Minimum value in the maximum ordinate on four vertex of maximum ordinate and the matching image on four vertex；

4th determines subelement, for by first abscissa, second abscissa, first ordinate and institute The second ordinate is stated, determines the overlapping region of the image to be spliced and the matching image.

It is described to establish unit in one embodiment of the application, comprising:

Subelement is obtained, for from the re-projection error of the corresponding all matching characteristic points pair of each image to be spliced, Obtain maximum re-projection error and minimum re-projection error；

First establishes subelement, is used for according to maximum re-projection error and minimum re-projection error, to each figure to be spliced The overlapping region of picture carries out the first grid foundation.

In one embodiment of the application, described first establishes subelement, is specifically used for:

In one embodiment of the application, the adjustment unit, comprising:

5th determines subelement, for every one first grid for each image to be spliced, according to first grid Weight determines the weight of matching characteristic point pair where characteristic point in first grid；

6th determines subelement, for every one first grid for each image to be spliced, according to first grid Weight, the weight of matching characteristic point pair where determining the outer characteristic point of first grid；

The first adjustment subelement, for every one first grid for each image to be spliced, according in first grid The weight of matching characteristic point pair where the weight of matching characteristic point pair where characteristic point and the outer characteristic point of first grid, adjustment should The homography matrix of image to be spliced, using homography matrix adjusted as the homography matrix of first grid.

In one embodiment of the application, the described 5th determines subelement, is specifically used for:

For each characteristic point in every one first grid of each image to be spliced, this feature is determined by following formula The weight of matching characteristic point pair where point:

In one embodiment of the application, the described 6th determines subelement, is specifically used for:

For each characteristic point outside every one first grid of each image to be spliced, this feature is determined by following formula The weight of matching characteristic point pair where point:

In one embodiment of the application, the concatenation unit, comprising:

Second establishes subelement, for carrying out second to the Non-overlapping Domain of each image to be spliced according to parameter preset Grid is established；The Non-overlapping Domain is the region in each image to be spliced in addition to overlapping region；

Second adjustment subelement, for every one second grid for each image to be spliced, according to the second grid phase The weighting ratio of other adjacent the second grids, adjusts the homography matrix of the image to be spliced, obtain second grid singly answers square Battle array；

Splice subelement, for spelling according to the homography matrix of every one first grid and the homography matrix of every one second grid Connect the basic stitching image and an at least image to be spliced.

In one embodiment of the application, the second adjustment subelement is specifically used for:

For every one second grid of each image to be spliced, according to the following formula, adjacent its of second grid is determined The weight of his the second grid:

Three aspects, the embodiment of the present application provide a kind of electronic equipment, including processor and memory；

The memory, for storing computer program；

The processor realizes that any of the above-described image is spelled for executing the computer program stored on the memory Connect method and step.

Four aspects, the embodiment of the present application provide a kind of machine readable storage medium, in the machine readable storage medium It is stored with computer program, the computer program realizes any of the above-described image split-joint method step when being executed by processor.

In the embodiment of the present application, after determining that each image to be spliced is directed to the homography matrix of basic stitching image, according to Re-projection error between matching characteristic point pair carries out the first grid foundation, re-projection to the overlapping region of each image to be spliced Error is an important parameter being characterized by, and can be established characteristic point similar in feature according to re-projection error at one the In one grid, in turn, for every one first grid of each image to be spliced, according to the weight of first grid, adjustment should be to The homography matrix of stitching image obtains the homography matrix of first grid, avoids the non-uniform image of feature distribution using single The problem of one homography matrix, splices basic stitching image and image to be spliced, is scheming according to the homography matrix of every one first grid As in the non-uniform situation of feature distribution, dislocation and the ghost image of image mosaic are solved the problems, such as.Certainly, implement this Shen Any product or method please must be not necessarily required to reach all the above advantage simultaneously.

Detailed description of the invention

Technical solution in ord to more clearly illustrate embodiments of the present application or in the related technology, below will be to embodiment or phase Attached drawing needed in technical description is closed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.

Fig. 1 is the first flow diagram of image split-joint method provided by the embodiments of the present application；

Fig. 2 is the first schematic diagram of image to be spliced provided by the embodiments of the present application；

Fig. 3 is second of flow diagram of image split-joint method provided by the embodiments of the present application；

Fig. 4 is a kind of distribution diagram of weighting ratio provided by the embodiments of the present application；

Fig. 5 is second of schematic diagram of image to be spliced provided by the embodiments of the present application；

Fig. 6 is the first structural schematic diagram of image splicing device provided by the embodiments of the present application；

Fig. 7 is second of structural schematic diagram of image splicing device provided by the embodiments of the present application；

Fig. 8 is the third structural schematic diagram of image splicing device provided by the embodiments of the present application；

Fig. 9 is the 4th kind of structural schematic diagram of image splicing device provided by the embodiments of the present application；

Figure 10 is the 5th kind of structural schematic diagram of image splicing device provided by the embodiments of the present application；

Figure 11 is a kind of structural schematic diagram of electronic equipment provided by the embodiments of the present application.

Specific embodiment

Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on Embodiment in the application, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall in the protection scope of this application.

Currently, when carrying out image mosaic using single homography matrix, in the picture in the non-uniform situation of feature distribution, very It may cause stitching image and the problem of dislocation and ghost image occur.

For dislocation and the ghost image for solving the problems, such as image mosaic, the embodiment of the present application provide a kind of image split-joint method and Device.With reference to Fig. 1, Fig. 1 is the first flow diagram of image split-joint method provided by the embodiments of the present application, this method packet It includes:

S101: basic stitching image and at least an image to be spliced are obtained.

In one embodiment of the application, image collecting device often collects an image, so that it may from Image Acquisition Device gets the image of the acquisition, and after getting multiple images, figure is opened in selection one that can be random from this multiple image As basic stitching image, other images are as image to be spliced.

In one embodiment of the application, image collecting device often collects an image, image can be stored to Database can obtain multiple images, from this multiple image when obtaining image to be spliced and basic stitching image from database In random selection one open image as basic stitching image, other images are as image to be spliced.

In the embodiment of the present application, image collecting device can be more camera lens panoramic cameras, one-shot camera etc..

S102: the characteristic point of basic stitching image and at least one image to be spliced is extracted.

In one embodiment of the application, in order to improve extract characteristic point reasonability, according to SIFT algorithm extract to Characteristic point in stitching image and basic stitching image.

For example, image to be spliced and basic stitching image can be extracted first to construct scale space using SIFT algorithm Characteristic point, SIFT algorithm robustness is good, has scaling invariance and certain noise immunity by the characteristic point that SIFT algorithm extracts.

In one embodiment of the application, for each characteristic point of extraction, this feature vertex neighborhood pixel can use Gradient information establish 360 degree of histogram of gradients, generate spatial description of higher-dimension, that is, obtain this feature point spatially Operator is described, this can make characteristic point have rotational invariance；Operator is described to the higher dimensional space of acquisition to be normalized, The description vectors of this feature point are obtained, this will be so that characteristic point be provided with comparison invariance.By with rotational invariance, comparison The characteristic point of invariance calculates homography matrix, effectively increases the accuracy that homography matrix is calculated.

S103: matching the characteristic point of every two images in an at least image to be spliced and basic stitching image, Determine multiple matching characteristic points pair.

In one embodiment of the application, each image in the image to be spliced and basic stitching image for acquisition Each characteristic point, calculate the characteristic point of other images description operator and this feature point description operator between Euclidean distance, Wherein, other images are the image in the image to be spliced obtained and basic stitching image in addition to the image；By this feature point with The corresponding characteristic point of the smallest description operator of Euclidean distance constitutes matching characteristic point pair.

For example, getting basic stitching image A and image B, C to be spliced, wherein the characteristic point extracted in image A has P01, p02, the characteristic point extracted in image B have p03, p04, and the characteristic point extracted in image C has p05.

The characteristic point of image A and the Feature Points Matching of image B: between the description operator and the description operator of p03 that calculate p01 Euclidean distance d1；Calculate the Euclidean distance d2 between the description operator of p01 and the description operator of p04；Calculate p02 description operator with Euclidean distance d3 between the description operator of p03；Calculate the Euclidean distance d4 between the description operator of p02 and the description operator of p04.

The characteristic point of image A and the Feature Points Matching of image C: the description of the description operator of p01 and the p05 of image C is calculated Euclidean distance d5 between operator；Calculate the Euclidean distance d6 between the description operator of p02 and the description operator of the p05 of image C.

The characteristic point of image B and the Feature Points Matching of image C: the description of the description operator of p03 and the p05 of image C is calculated Euclidean distance d7 between operator；Calculate the Euclidean distance d8 between the description operator of p04 and the description operator of the p05 of image C.

For the characteristic point p01 in image A, determining Euclidean distance has: d1, d2, d5.If d1 < d2 < d5, d1 are corresponding P03, then p01 and p03 constitutes matching characteristic point pair.

For the characteristic point p02 in image A, determining Euclidean distance has: d3, d4, d6.If d4 < d3 < d6, d4 are corresponding P04, then p02 and p04 constitutes matching characteristic point pair.

For the characteristic point p03 in image B, determining Euclidean distance has: d1, d3, d5.If d1 < d3 < d5, d1 are corresponding P01, then p03 and p01 constitutes matching characteristic point pair.

For the characteristic point p04 in image B, determining Euclidean distance has: d2, d4, d6.If d4 < d2 < d6, d2 are corresponding P02, then p04 and p02 constitutes matching characteristic point pair.

For the characteristic point p05 in image C, determining Euclidean distance has: d5, d6, d7, d8.If d7 < d5 < d6 < d8, d7 Corresponding p03, then p05 and p03 constitutes matching characteristic point pair.

In one embodiment of the application, for a characteristic point, determining between the description operator of this feature point After the smallest description operator of Euclidean distance, it can be determined that whether the smallest Euclidean distance is less than distance threshold；It, will if being less than This feature point characteristic point corresponding with the description operator of the smallest Euclidean distance constitutes matching characteristic point pair；Otherwise, it is determined that This feature point cannot constitute matching characteristic point pair with other characteristic points can not in subsequent calculating homography matrix, stitching image Consider this feature point.

In one embodiment of the application, k-dtree (k-dimensional tree, k dimension space tree) can be used The algorithm for calculating Euclidean distance is accelerated；RANSAC (Random Sample Consensus, with press proof can also be used This matching) algorithm and it is double select matching principle to optimize screening to algorithm, to improve the accuracy of determination matching characteristic point pair.

S104: it according to the coordinate of determining multiple matching characteristic point centering feature points, calculates each image to be spliced and is directed to The homography matrix of basic stitching image.

In embodiments herein, homography matrix are as follows:

If the coordinate of matching characteristic point two characteristic point of centering is respectively [x, y] and [x', y'],Square is singly answered in expression I-th column of battle array；

It can be according to formula

With

Positive definite equation is constructed, it is as follows

According to matrix A, construct optimal homography matrix seeks equation:

In formula (6), h_iIndicate i-th of matching characteristic point to corresponding homography matrix；SVD is carried out to matrix A (Singular Value Decomposition, singular value decomposition), available:

The corresponding feature vector v of C minimal eigenvalue λ in formula (7)_minAs optimal homography matrix H.

In embodiments herein, it is assumed that h₃₃It is 1, in this way, passing through 8 characteristic points by 4 matching characteristic points pair Construct 8 positive definite equations, so that it may the homography matrix of two images where determining matching characteristic point centering feature point.

In embodiments herein, if the characteristic point in an image to be spliced and the characteristic point in basic stitching image Matching characteristic point pair is constituted, then can directly determine out the homography matrix that the image to be spliced is directed to basic stitching image；

If the characteristic point in characteristic point and another image to be spliced in an image to be spliced constitutes matching characteristic point It is right, and the characteristic point in another image to be spliced constitutes matching characteristic point pair with the characteristic point in basic stitching image.Example Such as, image to be spliced has X, Y, and basic stitching image has Z, the characteristic point in image X to be spliced and the feature in image Y to be spliced Point constitutes matching characteristic point pair, and the characteristic point in characteristic point and basic stitching image Z in image Y to be spliced constitutes matching characteristic Point pair then needs to calculate the homography matrix 1 for being directed to image Y to be spliced from image X to be spliced, and calculates image Y needle to be spliced To the homography matrix 2 of basic stitching image Z；In conjunction with homography matrix 1 and homography matrix 2, image X to be spliced is calculated for basis The homography matrix 3 of stitching image Z.

S105: according to the homography matrix of each image to be spliced, the overlapping region of each image to be spliced is determined.

In one embodiment of the application, for each image to be spliced, the figure to be spliced is determined using following steps The overlapping region of picture:

S1051: determine four vertex of the image to be spliced basic stitching image abscissa in the plane and vertical Coordinate.

S1052: determine four vertex of matching image basic stitching image abscissa and ordinate in the plane.

Here, matching image is the feature point group with the image to be spliced into the figure where the characteristic point of matching characteristic point pair Picture.For example, the characteristic point in characteristic point in image B to be spliced and basic stitching image A constitutes matching characteristic point pair, then base Plinth stitching image A is the matching image of image B to be spliced, and image B to be spliced is the matching image of basic stitching image A.

S1053: the first abscissa, the second abscissa, the first ordinate and the second ordinate are determined.

Wherein, the first abscissa is four vertex of minimum abscissa and matching image on four vertex of image to be spliced Maximum value in minimum abscissa；Second abscissa is the maximum abscissa and matching image four on four vertex of image to be spliced Minimum value in the maximum abscissa on a vertex；First ordinate be four vertex of image to be spliced minimum ordinate and Maximum value in minimum ordinate with four vertex of image；Second ordinate is that the maximum on four vertex of image to be spliced is vertical Minimum value in the maximum ordinate on four vertex of coordinate and matching image.

S1054: by the first abscissa, the second abscissa, the first ordinate and the second ordinate, the image to be spliced is determined With the overlapping region of matching image.

The overlapping region of the above-mentioned image to be spliced and matching image is the overlapping region of the image to be spliced.

It is basic stitching image with image A for example, getting this 3 images of A, B, C, B and C are image to be spliced, scheme A Matching image, the matching image that image B is image C for image B；

The width of image A is W₁, it is highly L₁, the coordinate on four vertex of image A is t₁₁(0,0), t₁₂(0, L₁)、t₁₃ (W₁, 0), t₁₄(W₁, L₁)；The minimum abscissa of image A is 0, and maximum abscissa is W₁, minimum ordinate is 0, maximum ordinate For L₁；

For image B, according to the homography matrix of the image B acquired, when image B is projected to the plane of image A, image The coordinate on four vertex of B is t₂₁(x₂₁, y₂₁)、t₂₂(x₂₂, y₂₂)、t₂₃(x₂₃, y₂₃)、t₂₄(x₂₄, y₂₄)；

x_{min 1}=min (x₂₁, x₂₂, x₂₃, x₂₄)；

x_{max 1}=max (x₂₁, x₂₂, x₂₃, x₂₄)；

y_{min 1}=min (y₂₁, y₂₂, y₂₃, y₂₄)；

y_{max 1}=max (y₂₁, y₂₂, y₂₃, y₂₄)；

According to x_{min 1}、x_{max 1}、y_{min 1}、y_{max 1}And the minimum abscissa of image A is 0, maximum abscissa is W₁, minimum Ordinate is 0, maximum ordinate L₁, determine the lower-left angular coordinate (u of the overlapping region of image B_{min 1}, v_{min 1}) and upper right corner seat Mark (u_{max 1}, v_{max 1}):

u_{min 1}=max (x_{min 1}, 0)；

v_{min 1}=max (y_{min 1}, 0)；

u_{max 1}=max (x_{max 1}, W₁)；

v_{max 1}=max (y_{max 1}, L₁)；

By lower-left angular coordinate (u_{min 1}, v_{min 1}) and upper right angular coordinate (u_{max 1}, v_{max 1}) rectangular area determined can make For the overlapping region of image B.

For image C, according to the homography matrix of the image C acquired, when image C is projected to the plane of image A, image The coordinate on four vertex of C is t₃₁(x₃₁, y₃₁)、t₃₂(x₃₂, y₃₂)、t₃₃(x₃₃, y₃₃)、t₃₄(x₃₄, y₃₄)；

x_{min 2}=min (x₃₁, x₃₂, x₃₃, x₃₄)；

x_{max 2}=max (x₃₁, x₃₂, x₃₃, x₃₄)；

y_{min 2}=min (y₃₁, y₃₂, y₃₃, y₃₄)；

y_{max 2}=max (y₃₁, y₃₂, y₃₃, y₃₄)；

According to x_{min 2}、x_{max 2}、y_{min 2}、y_{max 2}And above-mentioned x_{min 1}、x_{max 1}、y_{min 1}、y_{max 1}, determine the weight of image C Lower-left angular coordinate (the u in folded region_{min 2}, v_{min 2}) and upper right angular coordinate (u_{max 2}, v_{max 2}):

u_{min 2}=max (x_{min 2}, x_{min 1})；

v_{min 2}=max (y_{min 2}, y_{min 1})；

u_{max 2}=max (x_{max 2}, x_{max 1})；

v_{max 2}=max (y_{max 2}, y_{max 1})；

By lower-left angular coordinate (u_{min 2}, v_{min 2}) and upper right angular coordinate (u_{max 2}, v_{max 2}) rectangular area determined can make For the overlapping region of image C.

In the embodiment of the present application, the overlapping region of image to be spliced can also be determined using other modes.For example, obtain to In stitching image matching characteristic point basic stitching image maximum abscissa in the plane, minimum abscissa, maximum vertical sit Mark and minimum ordinate, the rectangular area that 4 coordinates that will acquire are determined can be used as the overlapping region of the image to be spliced, The embodiment of the present application is to this without limiting.Here, matching characteristic point is with the feature point group in matching image into matching characteristic The characteristic point of point centering.

Behind the overlapping region that an image to be spliced has been determined, region in the image to be spliced in addition to overlapping region can be with It is determined as the Non-overlapping Domain of the image to be spliced.

S106: according to the re-projection error of matching characteristic point pair, first is carried out to the overlapping region of each image to be spliced Grid is established.

In one embodiment of the application, the re-projection error of matching characteristic point pair can be determined by following formula:

In formula (8), △ h is the homography matrix error of matching characteristic point pair, is referred to as matching characteristic point pair here Between re-projection error；For the homogeneous coordinates of matching characteristic point two characteristic points of centering, respectivelyFor H in formula (8), if not establishing grid to image, which is initial calculation Obtained homography matrix, determining re-projection error are as follows: the re-projection of matching characteristic point pair misses where the characteristic point in the image Difference；If establishing grid to image, which is the homography matrix of a grid, determining re-projection error are as follows: the spy in the grid The re-projection error of matching characteristic point pair where sign point.

For each image to be spliced, the weight of the matching characteristic point pair in the image to be spliced has been determined where characteristic point Projection error, the i.e. re-projection error of the corresponding matching characteristic point pair of the image to be spliced, for ease of description hereinafter referred to as should be to The corresponding re-projection error of stitching image obtains maximum re-projection from the corresponding all re-projection errors of the image to be spliced Error delta h_maxWith minimum re-projection error △ h_min, at this point, re-projection error region is △ h_min-△h_max, according to △ h_maxAnd △ h_min, the first grid foundation is carried out to the overlapping region of the image to be spliced.

It, can be according to according to △ h in a kind of implementation_maxWith △ h_min, by following formula, to the image to be spliced Overlapping region carries out the first grid foundation:

Wherein, △ h_iFor the throwing again of matching characteristic point pair where characteristic point in i-th of first grids of the image to be spliced Shadow error, N are positive integer；The value range of i is 0~N-1.

The problem of distortion of image mosaic effect is normally resulted in when feature distribution is uneven, especially for Image Acquisition When the field angle of device is larger, the edge image region of the Non-overlapping Domain of image to be spliced and the characteristic point in matching characteristic point Distance it is excessive, will lead to twisted phenomena.

It by re-projection error region division is N number of section according to formula (10), the characteristic point in each section is established same In one first grid, in this way, image to be spliced as shown in Figure 2, white area is overlapping region in Fig. 2, is built in white area Vertical grid is the first grid, and gray area is Non-overlapping Domain, establishes the first grid to overlapping region according to formula (10), The size of first grid may not wait, but in every one first grid matching characteristic point pair where characteristic point re-projection error one Determine in range, i.e. the feature of characteristic point is close in first grid, for every one first grid, ensure that in first grid Feature it is uniform, according to the homography matrix stitching image of first grid, image can be efficiently solved in first grid and spelled The problem of dislocation connect and ghost image.

S107: every one first grid of each image to be spliced is adjusted according to the weight of characteristic point in first grid The homography matrix of the whole image to be spliced, obtains the homography matrix of first grid.

In one embodiment of the application, for every one first grid of each image to be spliced, according to first net The weight of lattice, determines the weight of matching characteristic point pair where characteristic point in first grid, and determines that first grid is outer special The weight of matching characteristic point pair where sign point；For every one first grid of each image to be spliced, according in first grid The weight of matching characteristic point pair where the weight of matching characteristic point pair where characteristic point and the outer characteristic point of first grid, adjustment should The homography matrix of image to be spliced, using homography matrix adjusted as the homography matrix of first grid.

In a kind of implementation, after establishing the first grid, in every one first grid of each image to be spliced Each characteristic point, can be according to the weight of first grid, matching characteristic point pair where determining this feature point by following formula Weight:

Wherein, w_*For the weight of matching characteristic point pair where this feature point, M is of matching characteristic point in first grid Number, γ are the weight of first grid, and the value range of γ is 0~1.

Here, according to actual needs, γ can be by artificially adjusting control, to reduce lower confidence match characteristic point pair The influence that homography matrix is sought.Matching characteristic point is the characteristic point of matching characteristic point centering.

It, can be according to the power of first grid for each characteristic point outside every one first grid of each image to be spliced Value, the weight of matching characteristic point pair where determining this feature point by following formula:

Wherein, w'_*For the weight of matching characteristic point pair where this feature point, M is matching characteristic point in first grid Number, t_*For the coordinate of this feature point, t_iFor the coordinate of i-th of matching characteristic point in first grid.Matching characteristic point is Characteristic point with characteristic point centering.

For every one first grid of each image to be spliced, matching characteristic where characteristic point in first grid is being determined After the weight of matching characteristic point pair where the outer characteristic point of the weight and first grid of point pair, according to feature in first grid The weight of matching characteristic point pair where the weight of matching characteristic point pair where point and the outer characteristic point of first grid, adjustment should be wait spell The homography matrix of map interlinking picture, using homography matrix adjusted as the homography matrix of first grid.

As above, first grid of image to be spliced for one, according to formula (11) (12) to the matrix in formula (5) A is iterated:

Based on formula (13) and formula (6), the homography matrix of first grid can be determined are as follows:

SVD, the homography matrix of available first grid are carried out to the matrix W A in formula (14).

S108: according to the homography matrix of the homography matrix of each image to be spliced and every one first grid, splice basis and spell Map interlinking picture and at least an image to be spliced.

Homography matrix in one embodiment of the application, for each image to be spliced, according to every one first grid The overlapping region of image to be spliced is projected to the plane where basic stitching image, the homography matrix according to image to be spliced will The Non-overlapping Domain of the image to be spliced is projected to the plane where basic stitching image, is cut algorithm using figure and is searched image most Excellent splicing seams carry out flared end in optimal splicing seams two sides later, establish Gaussian convolution image to stitching image, utilize different convolution The convolved image of core carries out down-sampling, make the difference and establishes laplacian image, and establishes corresponding weight table, and root to each layer The fusion that image is carried out according to weight table, finally restores fused image, generates the blending image under original resolution.

In the embodiment of the present application, the feature distribution in every one first grid is more uniform, the list according to every one first grid It answers matrix to splice basic stitching image and an at least image to be spliced, avoids the non-uniform figure of feature distribution in overlapping region As the problem of using single homography matrix, according to the homography matrix of every one first grid, splice basic stitching image and to be spliced Image in overlapping region in the non-uniform situation of feature distribution, solves the problems, such as dislocation and the ghost image of image mosaic.

In practical application, if the matching characteristic point in the Non-overlapping Domain and overlapping region of image to be spliced is farther out, More serious distortion and deformation can occur for the position of Non-overlapping Domain, the reason is as follows that:

1) Non-overlapping Domain and matching characteristic point distance farther out so that weight is excessive and causes stitching image folding line and torsion It is bent；

2) characteristic point is unevenly distributed, and the characteristic point of general close shot is more, and the characteristic point of distant view is few, this causes Non-overlapping Domain Weight it is arteriopathy uneven, therefore far from matching characteristic point, especially edge can there is a phenomenon where folding line, Mou Xiete Black hole can also be generated by determining scene.

In one embodiment of the application, there is the phenomenon that distortion is with deformation, reference to reduce in Non-overlapping Domain Fig. 3, Fig. 3 are second of flow diagram of image split-joint method provided by the embodiments of the present application, this method comprises:

S301: basic stitching image and at least an image to be spliced are obtained.

S302: the characteristic point of basic stitching image and at least one image to be spliced is extracted.

S303: matching the characteristic point of every two images in an at least image to be spliced and basic stitching image, Determine multiple matching characteristic points pair.

S304: it according to the coordinate of determining multiple matching characteristic point centering feature points, calculates each image to be spliced and is directed to The homography matrix of basic stitching image.

S305: according to the homography matrix of each image to be spliced, the overlapping region of each image to be spliced is determined.

S306: according to the re-projection error of matching characteristic point pair, first is carried out to the overlapping region of each image to be spliced Grid is established.

S307: for every one first grid of each image to be spliced, according to the weight of first grid, adjustment should be wait spell The homography matrix of map interlinking picture obtains the homography matrix of first grid.

Step S301-S307 is identical as step S101-S107, and details are not described herein again.

S308: according to parameter preset, the second grid foundation is carried out to the Non-overlapping Domain of each image to be spliced.

Here, Non-overlapping Domain is the region in each image to be spliced in addition to overlapping region.Illustratively, in the present invention In embodiment, the size of every one second grid can not be waited, can also be equal in magnitude.

Image to be spliced for one, if the size of every one second grid is identical, the list according to every one second grid is answered Matrix splices basic stitching image and an at least image to be spliced, it is ensured that the Non-overlapping Domain of image mosaic it is smoothed It crosses, reduces in Non-overlapping Domain and the phenomenon that distortion is with deformation occur.

Above-mentioned parameter preset can be the number of grid of foundation, or the size of grid, the embodiment of the present application is to this Without limiting.

S309: for every one second grid of each image to be spliced, according to other adjacent second nets of second grid The weighting ratio of lattice adjusts the homography matrix of the image to be spliced, obtains the homography matrix of second grid.

In one embodiment of the application, for every one second grid of each image to be spliced, can according to this The weighting ratio of other adjacent the second grids of two grids, determines the weight of other adjacent the second grids of second grid.It is right In every one second grid of each image to be spliced, according to the weight of other adjacent the second grids of second grid, adjustment should The homography matrix of image to be spliced, using homography matrix adjusted as the homography matrix of second grid.

It, can be according to the second grid phase for every one second grid of each image to be spliced in a kind of implementation The weighting ratio of other adjacent the second grids determines the power of other adjacent the second grids of second grid by following formula Value:

Wherein, w_jFor the weight of j-th adjacent of other the second grid of second grid, Q_iIt is adjacent for second grid The weighting ratio of i-th of other the second grid, S are the total number of other second grids adjacent with second grid.

In the embodiment of the present application, the weighting ratio of the second grid can be controlled by artificial adjusting.For example, setting is vertical The weighting ratio of second grid in direction is greater than the weighting ratio of the second grid of horizontal direction, weighting ratio as shown in Figure 4 Distribution diagram, different directions have distinguished different weighting ratios in Fig. 4, wherein the weighting ratio of vertical direction is maximum, horizontal direction Weighting ratio it is minimum, this can effectively meet the need that human eye will be sensitive to the dislocation of horizontal direction to the dislocation of vertical direction It asks.

For every one second grid of each image to be spliced, other adjacent the second grids of second grid are being determined After weight, according to the weight of other adjacent the second grids of second grid, the homography matrix of the image to be spliced is adjusted, it will Homography matrix of the homography matrix adjusted as second grid:

In formula (16), H_*For the homography matrix of second grid, w_jFor j-th adjacent of second grids of second grid Weight, h_jFor the homography matrix of j-th adjacent of second grids of second grid.

For example, image to be spliced as shown in Figure 5, grid 1,2,3 is the grid of overlapping region, net in the image to be spliced Lattice 4,5,6,7,8,9 are the grid of Non-overlapping Domain；The homography matrix h of grid 1 currently has been determined according to formula (2)-(14)₁、 The homography matrix h of grid 2₂, grid 3 homography matrix h₃Homography matrix.

For grid 4, the grid of adjacent known homography matrix has grid 1,2, weighting ratio distribution as shown in connection with fig. 4 Figure, it is known that the weighting ratio of grid 1 is 1 for grid 4, the weighting ratio of grid 2 is 2, at this point it is possible to determine The weight w of grid 1₄₁Are as follows: 1/ (1+2)=1/3；It can determine the weight w of grid 2₄₂Are as follows: 2/ (1+2)=2/3；The list of grid 4 Answer matrix h₄Are as follows:

For grid 5, the grid of adjacent known homography matrix has grid 1,2 and grid 4, weight as shown in connection with fig. 4 Pro rate figure, it is known that the weighting ratio of grid 1 is 2 for grid 5, the weighting ratio of grid 2 is 1, grid 4 Weighting ratio be 4, at this point it is possible to determine grid 1 weight w₅₁Are as follows: 2/ (2+1+4)=2/7；It can determine the weight of grid 2 w₅₂Are as follows: 1/ (2+1+4)=1/7；It can determine the weight w of grid 4₅₃Are as follows: 4/ (2+1+4)=4/7；The homography matrix h of grid 5₅ Are as follows:

And so on, difference can be in the hope of the homography matrix of grid 6,7,8,9.

S310: according to the homography matrix of the homography matrix of every one first grid and every one second grid, splice basis splicing Image and at least an image to be spliced.

In one embodiment of the application, for each image to be spliced, it can be answered according to the list of every one first grid Matrix projects the overlapping region of image to be spliced to the plane where basic stitching image, and the list according to every one second grid is answered Matrix projects the Non-overlapping Domain of image to be spliced to the plane where basic stitching image, cuts algorithm using figure and searches image Optimal splicing seams, later optimal splicing seams two sides carry out flared end, Gaussian convolution image is established to stitching image, utilizes difference The convolved image of convolution kernel carries out down-sampling, make the difference and establishes laplacian image, and establishes corresponding weight table to each layer, And the fusion of image is carried out according to weight table, finally fused image is restored, generates the fusion under original resolution Image.

Using the embodiment of the present application, after determining that each image to be spliced is directed to the homography matrix of basic stitching image, root According to the re-projection error between matching characteristic point pair, the first grid foundation is carried out to the overlapping region of each image to be spliced, is thrown again Shadow error is an important parameter being characterized by, and can be established characteristic point similar in feature at one according to re-projection error In first grid, in turn, for every one first grid of each image to be spliced, according to the weight of first grid, adjustment should The homography matrix of image to be spliced obtains the homography matrix of first grid, avoids the non-uniform image of feature distribution and uses The problem of single homography matrix, splices basic stitching image and image to be spliced according to the homography matrix of every one first grid, In image in the non-uniform situation of feature distribution, dislocation and the ghost image of image mosaic are solved the problems, such as.

Corresponding with image split-joint method embodiment, the embodiment of the present application also provides a kind of image splicing devices.With reference to figure 6, Fig. 6 be a kind of structural schematic diagram of image splicing device provided by the embodiments of the present application, and described device includes:

Acquiring unit 601, for obtaining basic stitching image and at least an image to be spliced；

Extraction unit 602, for extracting the characteristic point of basic stitching image and at least one image to be spliced；

Matching unit 603, for the feature to every two images in an at least image to be spliced and basic stitching image Point is matched, and determines multiple matching characteristic points pair；

Computing unit 604 calculates each wait spell for the coordinate according to determining multiple matching characteristic point centering feature points Map interlinking picture is directed to the homography matrix of basic stitching image；

Determination unit 605 determines the overlapping of each image to be spliced for the homography matrix according to each image to be spliced Region；

Unit 606 is established, for the re-projection error according to matching characteristic point pair, to the overlay region of each image to be spliced Domain carries out the first grid foundation；

Adjustment unit 607, for every one first grid for each image to be spliced, according to the power of first grid Value, adjusts the homography matrix of the image to be spliced, obtains the homography matrix of first grid；

Concatenation unit 608, for according to the homography matrix of each image to be spliced and the homography matrix of every one first grid, Splice basic stitching image and an at least image to be spliced.

In one embodiment of the application, extraction unit 602 specifically can be used for:

The characteristic point in an at least image to be spliced and basic stitching image is extracted according to SIFT algorithm.

In one embodiment of the application, matching unit 603 specifically can be used for:

For each characteristic point of each image in an at least image to be spliced and basic stitching image, other are calculated Euclidean distance between the description operator of the characteristic point of image and the description operator of this feature point；Other images are at least one wait spell Image in map interlinking picture and basic stitching image in addition to the image；This feature point is corresponding with the smallest description operator of Euclidean distance Characteristic point constitute matching characteristic point pair.

In one embodiment of the application, above-mentioned image splicing device can also include:

In one embodiment of the application, determination unit 605 specifically can be used for for each image to be spliced, really The overlapping region of the fixed image to be spliced is based on Fig. 6 with reference to second of structural schematic diagram of image splicing device shown in Fig. 7, In the device, determination unit 605 may include:

First determines subelement 6051, for determining that four vertex of the image to be spliced are put down where basic stitching image Abscissa and ordinate on face；

Second determines subelement 6052, for determine four vertex of matching image basic stitching image institute in the plane Abscissa and ordinate；Matching image is with the feature point group of the image to be spliced at where the characteristic point of matching characteristic point pair Image；

Third determines subelement 6053, for determining the first abscissa, the second abscissa, the first ordinate and the second vertical seat Mark；First abscissa is the minimum abscissa on four vertex of minimum abscissa and matching image on four vertex of image to be spliced In maximum value；Second abscissa be four vertex of image to be spliced four vertex of maximum abscissa and matching image most Minimum value in big abscissa；First ordinate is the minimum ordinate on four vertex of image to be spliced and matching image four Maximum value in the minimum ordinate on vertex；Second ordinate is maximum ordinate and the matching on four vertex of image to be spliced Minimum value in the maximum ordinate on four vertex of image；

4th determines subelement 6054, for being sat by the first abscissa, the second abscissa, the first ordinate and second are vertical Mark, determines the overlapping region of the image to be spliced and matching image.

In one embodiment of the application, with reference to the third structural schematic diagram of image splicing device shown in Fig. 8, base In Fig. 6, the device, establishing unit 606 may include:

Subelement 6061 is obtained, for the re-projection error from the corresponding all matching characteristic points pair of each image to be spliced In, obtain maximum re-projection error and minimum re-projection error；

First establishes subelement 6062, is used for according to maximum re-projection error and minimum re-projection error, to each wait spell The overlapping region of map interlinking picture carries out the first grid foundation.

In one embodiment of the application, first establishes subelement, specifically can be used for:

In one embodiment of the application, with reference to the 4th kind of structural schematic diagram of image splicing device shown in Fig. 9, base In Fig. 6, adjustment unit 607 may include:

5th determines subelement 6071, for every one first grid for each image to be spliced, according to first net The weight of lattice determines the weight of matching characteristic point pair where characteristic point in first grid；

6th determines subelement 6072, for every one first grid for each image to be spliced, according to first net The weight of lattice, the weight of matching characteristic point pair where determining the outer characteristic point of first grid；

The first adjustment subelement 6073, for every one first grid for each image to be spliced, according to first net The weight of the weight of matching characteristic point pair where characteristic point and the outer characteristic point place matching characteristic point pair of first grid, is adjusted in lattice The homography matrix of the whole image to be spliced, using homography matrix adjusted as the homography matrix of first grid.

In one embodiment of the application, the 5th determines subelement 6071, specifically can be used for:

In one embodiment of the application, the 6th determines subelement 6072, specifically can be used for:

In one embodiment of the application, with reference to the 5th kind of structural schematic diagram of image splicing device shown in Fig. 10, Based on Fig. 6, concatenation unit 608 may include:

Second establishes subelement 6081, for being carried out to the Non-overlapping Domain of each image to be spliced according to parameter preset Second grid is established；Non-overlapping Domain is the region in each image to be spliced in addition to overlapping region；

Second adjustment subelement 6082, for every one second grid for each image to be spliced, according to second net The weighting ratio of other adjacent the second grids of lattice, adjusts the homography matrix of the image to be spliced, obtains the list of second grid Answer matrix；

Splice subelement 6083, for singly answering square according to the homography matrix of every one first grid and every one second grid Battle array splices basic stitching image and an at least image to be spliced.

In one embodiment of the application, second adjustment subelement 6082 specifically can be used for:

Corresponding with image split-joint method embodiment, the embodiment of the present application also provides a kind of electronic equipment.The electronic equipment Including processor and memory；Memory, for storing computer program；Processor is stored on memory for executing Computer program realizes above-mentioned image split-joint method.

Electronic equipment as shown in figure 11, including processor 1101 and memory 1103.In addition, electronic equipment can also communicate Interface 1102 and communication bus 1104, wherein processor 1101, communication interface 1102, memory 1103 pass through communication bus 1104 complete mutual communication；

Memory 1103, for storing computer program；

Processor 1101 when for executing the computer program stored on memory, realizes image split-joint method.Its In, image split-joint method includes:

Obtain basic stitching image and an at least image to be spliced；

Extract the characteristic point of basic stitching image and at least one image to be spliced；

The characteristic point of every two images in an at least image to be spliced and basic stitching image is matched, determination is more A matching characteristic point pair；

According to the coordinate of determining multiple matching characteristic point centering feature points, calculates each image to be spliced and spelled for basis The homography matrix of map interlinking picture；

According to the homography matrix of the homography matrix of each image to be spliced and every one first grid, splice basic stitching image An at least image to be spliced.

In one embodiment of the application, the characteristic point of basic stitching image and at least one image to be spliced is extracted Step may include:

In one embodiment of the application, to every two images in an at least image to be spliced and basic stitching image Characteristic point matched, determine the step of multiple matching characteristic points pair, may include:

In one embodiment of the application, every two figures in an at least image to be spliced and basic stitching image Before the characteristic point of picture is matched, above-mentioned image split-joint method can also include:

In one embodiment of the application, according to the homography matrix of each image to be spliced, each figure to be spliced is determined The step of overlapping region of picture, may include:

Determine four vertex of the image to be spliced basic stitching image abscissa and ordinate in the plane；

Determine four vertex of matching image basic stitching image abscissa and ordinate in the plane；Matching figure As being the feature point group with the image to be spliced into the image where the characteristic point of matching characteristic point pair；

Determine the first abscissa, the second abscissa, the first ordinate and the second ordinate；First abscissa is that this is to be spliced Maximum value in the minimum abscissa on four vertex of minimum abscissa and matching image on four vertex of image；Second abscissa is Minimum value in the maximum abscissa on four vertex of maximum abscissa and matching image on four vertex of image to be spliced；First Ordinate be four vertex of image to be spliced four vertex of minimum ordinate and matching image minimum ordinate in most Big value；Second ordinate, which is that the maximum on four vertex of maximum ordinate and matching image on four vertex of image to be spliced is vertical, to be sat Minimum value in mark；

By the first abscissa, the second abscissa, the first ordinate and the second ordinate, determine the image to be spliced with match The overlapping region of image.

In one embodiment of the application, according to the re-projection error of matching characteristic point pair, to each image to be spliced Overlapping region carry out the first grid the step of establishing, may include:

In one embodiment of the application, according to maximum re-projection error and minimum re-projection error, to each wait spell The overlapping region of map interlinking picture carries out the step of the first grid is established, and may include:

In one embodiment of the application, for every one first grid of each image to be spliced, according to first net The step of weight of lattice adjusts the homography matrix of the image to be spliced, obtains the homography matrix of first grid may include:

In one embodiment of the application, according to the weight of first grid, characteristic point institute in first grid is determined The weight of matching characteristic point pair the step of, may include:

In one embodiment of the application, according to the weight of first grid, the outer characteristic point institute of first grid is determined The weight of matching characteristic point pair the step of, may include:

In one embodiment of the application, according to the list of the homography matrix of each image to be spliced and every one first grid The step of answering matrix, splicing basic stitching image and an at least image to be spliced may include:

According to parameter preset, the second grid foundation is carried out to the Non-overlapping Domain of each image to be spliced；Non-overlapping Domain For the region in each image to be spliced in addition to overlapping region；

According to the homography matrix of the homography matrix of every one first grid and every one second grid, splice basic stitching image and An at least image to be spliced.

In one embodiment of the application, for every one second grid of each image to be spliced, according to second net The weighting ratio of other adjacent the second grids of lattice, adjusts the homography matrix of the image to be spliced, obtains the list of second grid The step of answering matrix may include:

In one embodiment of the application, according to the weighting ratio of other adjacent the second grids of second grid, really The step of weight of other adjacent the second grids of fixed second grid, may include:

Above-mentioned communication bus 1104 can be PCI (Peripheral Component Interconnect, external components Interconnection standards) bus or EISA (Extended Industry Standard Architecture, expanding the industrial standard structure) Bus etc..The communication bus 1104 can be divided into address bus, data/address bus, control bus etc..For convenient for indicating, in Figure 11 only It is indicated with a thick line, it is not intended that an only bus or a type of bus.

Communication interface 1102 is for the communication between above-mentioned electronic equipment and other equipment.

Memory 1103 may include RAM (Random Access Memory, random access memory), also may include NVM (Non-Volatile Memory, nonvolatile memory), for example, at least a magnetic disk storage.Optionally, memory 1103 can also be that at least one is located remotely from the storage device of aforementioned processor.

Processor 1104 can be general processor, including CPU (Central Processing Unit, central processing Device), NP (Network Processor, network processing unit) etc.；Can also be DSP (Digital Signal Processing, Digital signal processor), ASIC (Application Specific Integrated Circuit, specific integrated circuit), It is FPGA (Field-Programmable Gate Array, field programmable gate array) or other programmable logic device, discrete Door or transistor logic, discrete hardware components.

The embodiment of the present application also provides a kind of machine readable storage medium, calculating is stored in machine readable storage medium Machine program, realizes image split-joint method when computer program is executed by processor.Wherein image split-joint method includes:

Obtain basic stitching image and an at least image to be spliced；

It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including the element.

Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.It is spelled especially for image For connection device, electronic equipment, machine readable storage medium embodiment, implement since it is substantially similar to image split-joint method Example, so being described relatively simple, related place is referring to the part explanation of Fig. 1-image split-joint method embodiment shown in fig. 5 It can.

The foregoing is merely the preferred embodiments of the application, are not intended to limit the protection scope of the application.It is all Any modification, equivalent replacement, improvement and so within spirit herein and principle are all contained in the protection scope of the application It is interior.

Claims

1. a kind of image split-joint method, which is characterized in that the described method includes:

Obtain basic stitching image and an at least image to be spliced；

The characteristic point of every two images in an at least image to be spliced and the basic stitching image is matched, really Fixed multiple matching characteristic points pair；

According to the coordinate of determining multiple matching characteristic point centering feature points, calculates each image to be spliced and spelled for the basis The homography matrix of map interlinking picture；

According to the re-projection error of matching characteristic point pair, the first grid foundation is carried out to the overlapping region of each image to be spliced；

The image to be spliced is adjusted according to the weight of first grid for every one first grid of each image to be spliced Homography matrix obtains the homography matrix of first grid；

According to the homography matrix of the homography matrix of each image to be spliced and every one first grid, splice the basic stitching image With an at least image to be spliced.

2. the method according to claim 1, wherein described extract the basic stitching image and described at least one The step of opening the characteristic point of image to be spliced, comprising:

An at least image to be spliced and the basic stitching image are extracted according to Scale invariant features transform SIFT algorithm In characteristic point.

3. the method according to claim 1, wherein described at least an image to be spliced and the base The characteristic point of every two images is matched in plinth stitching image, determines the step of multiple matching characteristic points pair, comprising:

For each characteristic point of each image in an at least image to be spliced and the basic stitching image, calculate Euclidean distance between the description operator of the characteristic point of other images and the description operator of this feature point；Other described images are described Image in an at least image to be spliced and the basic stitching image in addition to the image；Most by this feature point and Euclidean distance The corresponding characteristic point of small description operator constitutes matching characteristic point pair.

4. according to the method described in claim 3, it is characterized in that, at least an image to be spliced and the basis Before the characteristic point of every two images is matched in stitching image, the method also includes:

For each characteristic point of extraction, 360 degree of histogram of gradients are established using the gradient information of this feature vertex neighborhood pixel, are obtained Obtain the description operator of this feature point spatially；Normalize the description operator of this feature point.

5. the method according to claim 1, wherein the homography matrix according to each image to be spliced, really The step of overlapping region of fixed each image to be spliced, comprising:

Determine four vertex of the image to be spliced the basic stitching image abscissa and ordinate in the plane；

Determine four vertex of matching image the basic stitching image abscissa and ordinate in the plane；Described It is the feature point group with the image to be spliced into the image where the characteristic point of matching characteristic point pair with image；

Determine the first abscissa, the second abscissa, the first ordinate and the second ordinate；First abscissa is that this is to be spliced Maximum value in the minimum abscissa on four vertex of minimum abscissa and the matching image on four vertex of image；Described second Abscissa is in the maximum abscissa on four vertex of maximum abscissa and the matching image on four vertex of image to be spliced Minimum value；First ordinate is that the minimum ordinate on four vertex of image to be spliced and the matching image four push up Maximum value in the minimum ordinate of point；Maximum ordinate and institute of second ordinate for four vertex of image to be spliced State the minimum value in the maximum ordinate on four vertex of matching image；

By first abscissa, second abscissa, first ordinate and second ordinate, determining should be wait spell The overlapping region of map interlinking picture and the matching image.

6. the method according to claim 1, wherein the re-projection error according to matching characteristic point pair, right The overlapping region of each image to be spliced carries out the step of the first grid is established, comprising:

From the re-projection error of the corresponding all matching characteristic points pair of each image to be spliced, obtain maximum re-projection error and Minimum re-projection error；

According to maximum re-projection error and minimum re-projection error, the first grid is carried out to the overlapping region of each image to be spliced It establishes.

7. according to the method described in claim 6, it is characterized in that, described miss according to maximum re-projection error and minimum re-projection Difference carries out the step of the first grid is established to the overlapping region of each image to be spliced, comprising:

For each image to be spliced, according to the following formula, the first grid foundation is carried out to the overlapping region of the image to be spliced:

Wherein, △ h_iIt is missed for the re-projection of matching characteristic point pair where characteristic point in i-th of first grids of the image to be spliced Difference, △ h_minFor the corresponding minimum re-projection error of the image to be spliced of acquisition, △ h_maxFor the image pair to be spliced of acquisition The maximum re-projection error answered, N are positive integer；The value range of i is 0~N-1.

8. the method according to claim 1, wherein every one first net for each image to be spliced Lattice adjust the homography matrix of the image to be spliced according to the weight of first grid, obtain the homography matrix of first grid Step, comprising:

For every one first grid of each image to be spliced, according to the weight of first grid, determine special in first grid The weight of matching characteristic point pair where sign point；

For every one first grid of each image to be spliced, according to the weight of first grid, determine that first grid is outer special The weight of matching characteristic point pair where sign point；

For every one first grid of each image to be spliced, according to matching characteristic point pair where characteristic point in first grid The weight of matching characteristic point pair, adjusts the homography matrix of the image to be spliced where weight and the outer characteristic point of first grid, will Homography matrix of the homography matrix adjusted as first grid.

9. according to the method described in claim 8, it is characterized in that, the weight according to first grid, determine this first In grid where characteristic point the step of the weight of matching characteristic point pair, comprising:

For each characteristic point in first grid, the power of matching characteristic point pair where determining this feature point by following formula Value:

Wherein, w_*For the weight of matching characteristic point pair where this feature point, γ is the weight of first grid, the value range of γ For the number of 0~1, the M characteristic point that be in first grid include in matching characteristic point pair.

10. according to the method described in claim 8, it is characterized in that, the weight according to first grid, determine this first Where the outer characteristic point of grid the step of the weight of matching characteristic point pair, comprising:

For each characteristic point outside first grid, the power of matching characteristic point pair where determining this feature point by following formula Value:

Wherein, w'_*For the weight of matching characteristic point pair where this feature point, t_*For the coordinate of this feature point, t_iFor first grid It include for interior i-th the coordinate in matching characteristic point centering feature point.

11. the method according to claim 1, wherein the homography matrix according to each image to be spliced and The step of homography matrix of every one first grid, the splicing basic stitching image and an at least image to be spliced, packet It includes:

According to parameter preset, the second grid foundation is carried out to the Non-overlapping Domain of each image to be spliced；The Non-overlapping Domain For the region in each image to be spliced in addition to overlapping region；

For every one second grid of each image to be spliced, according to the weight ratio of other adjacent the second grids of second grid Example, adjusts the homography matrix of the image to be spliced, obtains the homography matrix of second grid；

According to the homography matrix of the homography matrix of every one first grid and every one second grid, splice the basic stitching image and An at least image to be spliced.

12. according to the method for claim 11, which is characterized in that every one second net for each image to be spliced Lattice adjust the homography matrix of the image to be spliced according to the weighting ratio of other adjacent the second grids of second grid, obtain The step of homography matrix of second grid, comprising:

For every one second grid of each image to be spliced, according to the weight ratio of other adjacent the second grids of second grid Example, determines the weight of other adjacent the second grids of second grid；

For every one second grid of each image to be spliced, according to the weight of other adjacent the second grids of second grid, The homography matrix for adjusting the image to be spliced, using homography matrix adjusted as the homography matrix of second grid.

13. according to the method for claim 12, which is characterized in that described other second nets adjacent according to second grid The weighting ratio of lattice, the step of determining the weight of other adjacent the second grids of second grid, comprising:

Wherein, w_jFor the weight of j-th adjacent of other the second grid of second grid, Q_jFor adjacent j-th of second grid The weighting ratio of other the second grids, Q_iFor the weighting ratio of i-th adjacent of other the second grid of second grid, S be with The total number of other adjacent the second grids of second grid.

14. the described in any item methods of 1-13 according to claim 1, which is characterized in that every one second network it is equal in magnitude.

15. a kind of image splicing device, which is characterized in that described device includes:

Matching unit, for the feature to every two images in an at least image to be spliced and the basic stitching image Point is matched, and determines multiple matching characteristic points pair；

Computing unit calculates each image to be spliced for the coordinate according to determining multiple matching characteristic point centering feature points For the homography matrix of the basic stitching image；

Determination unit determines the overlapping region of each image to be spliced for the homography matrix according to each image to be spliced；

Unit is established, for the re-projection error according to matching characteristic point pair, the overlapping region of each image to be spliced is carried out First grid is established；

Adjustment unit, for every one first grid for each image to be spliced, according to the weight of first grid, adjustment should The homography matrix of image to be spliced obtains the homography matrix of first grid；

Concatenation unit, for splicing institute according to the homography matrix of each image to be spliced and the homography matrix of every one first grid State basic stitching image and an at least image to be spliced.

16. device according to claim 15, which is characterized in that the extraction unit is specifically used for:

17. device according to claim 15, which is characterized in that the matching unit is specifically used for:

18. device according to claim 17, which is characterized in that described device further include:

Obtaining unit establishes 360 degree using the gradient information of this feature vertex neighborhood pixel for each characteristic point for extraction Histogram of gradients obtains the description operator of this feature point spatially；

19. device according to claim 15, which is characterized in that the determination unit, specifically for for each wait spell Map interlinking picture determines the overlapping region of the image to be spliced, comprising:

First determines subelement, for determine four vertex of the image to be spliced the basic stitching image institute in the plane Abscissa and ordinate；

Second determines subelement, for determine four vertex of matching image the basic stitching image cross in the plane Coordinate and ordinate；The matching image is with the feature point group of the image to be spliced at where the characteristic point of matching characteristic point pair Image；

Third determines subelement, for determining the first abscissa, the second abscissa, the first ordinate and the second ordinate；It is described First abscissa is the horizontal seat of minimum on four vertex of minimum abscissa and the matching image on four vertex of image to be spliced Maximum value in mark；Second abscissa is the maximum abscissa and the matching image four on four vertex of image to be spliced Minimum value in the maximum abscissa on a vertex；First ordinate is the minimum ordinate on four vertex of image to be spliced With the maximum value in the minimum ordinate on four vertex of the matching image；Second ordinate is this image four to be spliced Minimum value in the maximum ordinate on four vertex of the maximum ordinate on vertex and the matching image；

4th determines subelement, for by first abscissa, second abscissa, first ordinate and described the Two ordinates determine the overlapping region of the image to be spliced and the matching image.

20. device according to claim 15, which is characterized in that described to establish unit, comprising:

Subelement is obtained, for obtaining from the re-projection error of the corresponding all matching characteristic points pair of each image to be spliced Maximum re-projection error and minimum re-projection error；

First establishes subelement, is used for according to maximum re-projection error and minimum re-projection error, to each image to be spliced Overlapping region carries out the first grid foundation.

21. device according to claim 20, which is characterized in that described first establishes subelement, is specifically used for:

22. device according to claim 15, which is characterized in that the adjustment unit, comprising:

5th determines subelement, for every one first grid for each image to be spliced, according to the weight of first grid, Determine the weight of matching characteristic point pair where characteristic point in first grid；

6th determines subelement, for every one first grid for each image to be spliced, according to the weight of first grid, The weight of matching characteristic point pair where determining the outer characteristic point of first grid；

The first adjustment subelement, for every one first grid for each image to be spliced, according to feature in first grid The weight of matching characteristic point pair where the weight of matching characteristic point pair where point and the outer characteristic point of first grid, adjustment should be wait spell The homography matrix of map interlinking picture, using homography matrix adjusted as the homography matrix of first grid.

23. device according to claim 22, which is characterized in that the described 5th determines subelement, is specifically used for:

For each characteristic point in every one first grid of each image to be spliced, this feature point institute is determined by following formula In the weight of matching characteristic point pair:

24. device according to claim 22, which is characterized in that the described 6th determines subelement, is specifically used for:

For each characteristic point outside every one first grid of each image to be spliced, this feature point institute is determined by following formula In the weight of matching characteristic point pair:

25. device according to claim 15, which is characterized in that the concatenation unit, comprising:

Second establishes subelement, for carrying out the second grid to the Non-overlapping Domain of each image to be spliced according to parameter preset It establishes；The Non-overlapping Domain is the region in each image to be spliced in addition to overlapping region；

Second adjustment subelement is adjacent according to second grid for every one second grid for each image to be spliced The weighting ratio of other the second grids adjusts the homography matrix of the image to be spliced, obtains the homography matrix of second grid；

Splice subelement, for splicing institute according to the homography matrix of every one first grid and the homography matrix of every one second grid State basic stitching image and an at least image to be spliced.

26. device according to claim 25, which is characterized in that the second adjustment subelement is specifically used for:

27. device according to claim 26, which is characterized in that the second adjustment subelement is specifically used for:

For every one second grid of each image to be spliced, according to the following formula, adjacent other of second grid are determined The weight of two grids:

28. according to the described in any item devices of claim 25-27, which is characterized in that every one second network it is equal in magnitude.

29. a kind of electronic equipment, which is characterized in that including processor and memory；

The memory, for storing computer program；

The processor realizes any institute of claim 1-14 for executing the computer program stored on the memory The method and step stated.