CN110223222A

CN110223222A - Image split-joint method, image splicing device and computer readable storage medium

Info

Publication number: CN110223222A
Application number: CN201810175736.0A
Authority: CN
Inventors: 王艺伟; 刘丽艳; 王炜
Original assignee: Ricoh Co Ltd
Current assignee: Ricoh Co Ltd
Priority date: 2018-03-02
Filing date: 2018-03-02
Publication date: 2019-09-10
Anticipated expiration: 2038-03-02
Also published as: CN110223222B

Abstract

The embodiment of the present invention provides image split-joint method, device and computer readable storage medium, described image includes at least the first image and the second image, wherein image split-joint method includes: that the first image to be spliced and the second image are carried out characteristic point detection and matching, multiple Feature Points Matchings pair are obtained, each described Feature Points Matching is to the fisrt feature point for including the first image and the second feature point of second image；First image is marked off at least two first splicing regions, according to Feature Points Matching to the first homography matrix for calculating separately each first splicing regions；First image is marked off into multiple first grids, and according at least one of the first homography matrix of at least two first splicing regions, calculates the first grid homography matrix of each first grid of the first image；The first grid of each of the first image is coordinately transformed according to its corresponding first grid homography matrix, and combines the second image to form image after splicing.

Description

Image split-joint method, image splicing device and computer readable storage medium

Technical field

This application involves field of image processing more particularly to a kind of image split-joint methods, image splicing device and computer Readable storage medium storing program for executing.

Background technique

Image mosaic technology is that the image that several have lap (can be different time, different perspectives or difference Sensor obtain) be combined into width large size seamless high-definition picture technology, this is always graphics and machine vision Where research interest.

In the prior art, merging algorithm for images solves the problems, such as image mosaic often through the mode of grid optimization, and And also it can retain the processing of (Shape Preserving) using such as shape to obtain relatively natural splicing effect.Specifically Ground, existing image mosaic technology can generally carry out characteristic point detection and matching first, then according to the result of Feature Points Matching Estimate a homography matrix, and the image according to this homography matrix for gridding completes splicing.However, in above-mentioned image In splicing, it often will limit the precision of final stitching image to the estimation mode of homography matrix, influence splicing effect.

Therefore, it is necessary to the image split-joint methods that one kind can further increase image mosaic precision.

Summary of the invention

In order to solve the above technical problems, according to an aspect of the invention, there is provided a kind of image split-joint method, the figure As including at least the first image and the second image, which comprises carry out the first image to be spliced and the second image special The detection of sign point and matching, obtain multiple Feature Points Matchings pair, and wherein each described Feature Points Matching is to including first figure The fisrt feature point of picture and the second feature point of second image；The first image is marked off at least two first splicings Region, according to the Feature Points Matching to the first homography matrix for calculating separately each first splicing regions；By described first Image marks off multiple first grids, and according in first homography matrix of at least two first splicing regions At least one, calculates the first grid homography matrix of each first grid of the first image；It will be in the first image Each first grid is coordinately transformed according to its corresponding first grid homography matrix, and in conjunction with second image with shape At image after splicing.

According to another aspect of the present invention, a kind of image splicing device is provided, described image includes at least the first figure Picture and the second image, described device include: matching unit, are configured to the first image to be spliced and the second image carrying out feature Point detection and matching, obtain multiple Feature Points Matchings pair, and wherein each described Feature Points Matching is to including the first image Fisrt feature point and second image second feature point；Matrix calculation unit is configured to divide the first image At least two first splicing regions out are singly answered according to the Feature Points Matching calculate separately each first splicing regions first Property matrix；Grid dividing unit is configured to marking off the first image into multiple first grids, and according to described at least two At least one of described first homography matrix of first splicing regions calculates the of each first grid of the first image One grid homography matrix；Coordinate transformation unit is configured to the first grid of each of the first image according to its correspondence The first grid homography matrix be coordinately transformed, and in conjunction with second image with formed splicing after image.

According to another aspect of the present invention, a kind of image splicing device is provided, described image includes at least the first image And second image, described device includes: processor and memory, it is stored with computer program instructions in the memory, In, when the computer program instructions are run by the processor, so that the processor executes following steps: will be to be spliced The first image and the second image carry out characteristic point detection and matching, multiple Feature Points Matchings pair are obtained, wherein described in each Feature Points Matching is to the fisrt feature point for including the first image and the second feature point of second image；By described first Image marks off at least two first splicing regions, according to the Feature Points Matching to calculating separately each first splicing regions First homography matrix；The first image is marked off into multiple first grids, and according at least two first splice region At least one of first homography matrix in domain, the first grid list for calculating each first grid of the first image are answered Property matrix；The first grid of each of the first image is subjected to coordinate change according to its corresponding first grid homography matrix It changes, and in conjunction with second image to form image after splicing.

According to another aspect of the present invention, a kind of computer readable storage medium is provided, computer journey is stored thereon with Sequence instruction, the computer program instructions realize following image mosaic step when being executed by processor, and wherein described image is at least Including the first image and the second image: the first image to be spliced and the second image being carried out characteristic point detection and matching, obtained Multiple Feature Points Matchings pair, wherein each described Feature Points Matching is to including the fisrt feature point of the first image and described The second feature point of second image；The first image is marked off at least two first splicing regions, according to the characteristic point Matching is to the first homography matrix for calculating separately each first splicing regions；The first image is marked off into multiple first nets Lattice, and according at least one of first homography matrix of at least two first splicing regions calculate described the First grid homography matrix of each first grid of one image；The first grid of each of the first image is right according to its The the first grid homography matrix answered is coordinately transformed, and in conjunction with second image to form image after splicing.

Above-mentioned image split-joint method, image splicing device and computer readable storage medium according to the present invention, Neng Gougen It is respectively formed the first different homography matrixes according to multiple first splicing regions for marking off the first image, and according to institute The first different homography matrixes is stated the first image of gridding is coordinately transformed and be spliced, to improve spelling obtained The precision of rear image is connect, splicing effect is improved.

Detailed description of the invention

Embodiments of the present invention is described in detail in conjunction with the accompanying drawings, above and other objects of the present invention, feature, Advantage will become apparent.

Fig. 1 shows the flow chart of image split-joint method according to an embodiment of the invention；

Fig. 2 (a) shows a schematic diagram of the first image according to an embodiment of the invention；Fig. 2 (b) shows The schematic diagram of two images；Fig. 2 (c) shows the signal that the first image and the second image are carried out to image after spliced splicing Figure；

Fig. 3 shows the schematic diagram of Feature Points Matching pair according to an embodiment of the invention；

Fig. 4 shows in the first image according to an embodiment of the invention the schematic diagram of fisrt feature point and along splicing edge The corresponding distributed number histogram in edge direction；

Fig. 5 shows the division schematic diagram of the first splicing regions according to an embodiment of the invention；

Fig. 6 shows the splicing that the first image and the second image according to an embodiment of the invention implement image split-joint method The schematic diagram of image afterwards；

Image carries out the schematic diagram after orientation consistency correction after Fig. 7 shows splicing according to an embodiment of the invention；

Fig. 8 shows the block diagram of image splicing device according to an embodiment of the invention；

Fig. 9 shows the block diagram of image splicing device according to an embodiment of the invention.

Specific embodiment

Image split-joint method, image splicing device and calculating according to an embodiment of the present invention described below with reference to accompanying drawings Machine readable storage medium storing program for executing.In the accompanying drawings, identical reference label indicates identical element from beginning to end.It is understood that retouching here The embodiment stated is merely illustrative, and is not necessarily to be construed as limiting the scope of the invention.

In the embodiment of the present invention, in order to overcome image mosaic caused by homography matrix estimation mode in the prior art smart Spend inadequate problem, the present invention considers that image to be spliced is carried out region division during image mosaic, according to dividing To different zones obtain different homography matrixes, and be coordinately transformed and splice accordingly, to improve image mosaic Precision improves splicing effect.

Image split-joint method according to an embodiment of the present invention is described below with reference to Fig. 1.Fig. 1 shows the image split-joint method 100 flow chart.Wherein, described image includes at least the first image and the second image.

As shown in Figure 1, in step s101, by the first image to be spliced and the second image carry out characteristic point detection and Match, obtain multiple Feature Points Matchings pair, wherein each described Feature Points Matching is to the fisrt feature including the first image The second feature point of point and second image.

In embodiments of the present invention, it is expected that being that a width range is bigger by the first image and second image mosaic Image after splicing.Wherein, it can have lap each other in the first image and the second image, also, lap distinguished Position in the first image and the second image is herein with no restrictions.For example, an example, Fig. 2 (a) are shown according to the present invention One schematic diagram of the first image；Fig. 2 (b) shows the schematic diagram of the second image；Fig. 2 (c) show the first image and Second image carries out the schematic diagram of image after spliced splicing, it is seen then that in the example shown in Fig. 2 (a)-Fig. 2 (c), the first figure The lap of picture and the second image is located at the right side of the first image and the left side of the second image, thus image after splicing Left-half basic source is in the first image, and right half part basic source image after the second image, splicing becomes a width model Enclose bigger image.In another of the invention example, it is also possible that lap be located at the first image downside and The upside of second image, in this case, the first image and the second image will realize splicing up and down, spliced image it is upper Half part basic source is in the first image, and lower half portion basic source is in the second image.In another example of the invention, the The lap of one image and the second image can be located at any position in the first image and the second image, and mutually Between there is certain rotation angle, thus spliced image will also be revolved according at least one of the first image and the second image The angle for turning certain is constituted.Foregoing description is merely illustrative, in practical applications, can also use any first image and second The overlap mode of image.

In this step, characteristic point detection and matching will be carried out to the first image to be spliced and the second image, to obtain Multiple Feature Points Matchings pair.Specifically, it is possible, firstly, to obtain the first image and second image respectively.In the present invention In one example, the first image and the second image can be on object (such as mobile robot, intelligent vehicle, unmanned plane etc.) respectively The image that the shooting unit of outfit obtains, wherein shooting unit can be monocular camera or video camera, naturally it is also possible to be binocular Or more mesh cameras or video camera, it is not limited here.Acquired the first image and the second image can be respectively different time, It is obtained in different location or certain angular field of view, as long as having certain lap each other.

It, can be based on preset characteristic point detection mode respectively to the after obtaining the first image and the second image One image and the second image carry out characteristic point detection.In embodiments of the present invention, preset characteristic point detection mode may include Such as Scale invariant features transform (Scale Invariant Feature Transform, SIFT) feature, acceleration are steadily and surely The various feature point detecting methods such as (Speeded Up Robust Features, SURF) feature, Harris angle point, or ORB (Oriented FAST and Rotated BRIEF) feature point detecting method.After detecting characteristic point, it is alternatively possible to The characteristic point of first image and the second image detection is described, such as can be using gray feature, Gradient Features, parallax letter Breath etc. is various to be used for the methods that feature describes to describe the characteristic point in the first image and the second image.

Finally, the characteristic point that the first image detects and the characteristic point that second image detection arrives can be carried out Matching, to obtain multiple Feature Points Matchings pair.It is alternatively possible to utilize the movement statistics (Grid-based based on grid Motion Statistics, GMS) Lai Jinhang Feature Points Matching.It, can be by matching correctness in Feature Points Matching Judgement, with eliminate it is certain mistake, can not matched characteristic point, only leave correct characteristic point and corresponding characteristic point Pairing, to improve matched stability.After matching finishes, each acquired Feature Points Matching is to including described the The fisrt feature point of one image and the second feature point of second image, wherein all fisrt feature points of the first image are equal From the characteristic point detected in the first image before, can be the first image detection to characteristic point in whole or in which A part；Similarly, all second feature points of the second image be also possible to the second image detection to characteristic point in whole Or in which a part.All second feature points of all fisrt feature points and the second image in first image are one-to-one Relationship respectively constitutes each Feature Points Matching pair.In one example of an embodiment of the present invention, acquired Feature Points Matching To can be four or more.Fig. 3 shows in one embodiment of the invention and carries out characteristic point detection, and benefit using ORB The schematic diagram of the Feature Points Matching pair obtained after carrying out Feature Points Matching with GMS, wherein left side is the first image, right side the Two images.Wherein, characteristic point detection is carried out using ORB in example shown in Fig. 3 and carries out Feature Points Matching using GMS, it can be right Image rotation and change of scale have stronger robustness, with the error of characteristic point after reduction matching.

In step s 102, the first image is marked off at least two first splicing regions, according to the characteristic point Matching is to the first homography matrix for calculating separately each first splicing regions.

In this step, it is alternatively possible to according at least partly fisrt feature point in the first image, by described One image marks off at least two first splicing regions.Wherein, the mode for dividing the first splicing regions to the first image can be with base In grouping and cluster to fisrt feature point part or all of in the first image.

In one example, fisrt feature point can be carried out according to the quantity of the fisrt feature point along splicing edge direction Grouping, and clustered fisrt feature point using clustering algorithm.Specifically, it can calculate first in the first image extremely Fisrt feature point quantity distribution histogram of the small part fisrt feature point along the splicing edge direction of the first image.Fig. 4 shows The schematic diagram of fisrt feature point in the first image according to an embodiment of the invention is gone out and along the corresponding of splicing edge direction Distributed number histogram, the first image in Fig. 4 corresponds to first image in left side in Fig. 3.In figure 4, it can be seen that the The fisrt feature point quantity that the top half of one image has is relatively more, and the fisrt feature point quantity that lower half portion has compared with It is few.Then, the quantity of first splicing regions can be determined according to the fisrt feature point quantity distribution histogram.Specifically Ground is such as fitted using Gauss curve fitting as shown in figure 4, can be fitted to the distributed number histogram of fisrt feature point, And the parameters such as derivative, second dervative, peak value or halfwidth of each point on curve after digital simulation, and it is true according to above-mentioned parameter The quantity of fixed the first marked off splicing regions namely the quantity of fisrt feature point grouping.For example, according to Gauss curve fitting song When line carries out the grouping of fisrt feature point, a k value can be preset, for example, 1, then further according to fisrt feature point in different zones The distance between adjacent peak judges the variation of k value on occupied quantitative proportion and/or curve.As shown in Figure 4, Gauss curve fitting There is multiple wave crests and multiple troughs in curve.For the Gauss curve fitting curve of diagram, it may be considered that between adjacent peak away from From occupied by the fisrt feature point in each region that this distance is greater than certain preset threshold, and marks off accordingly Quantitative proportion when also complying with preset condition, k value can add 1.It is not above in Fig. 4, between distance A and other peak values pre- If distance, and pre-determined distance is crossed apart from B ultrasound.On this basis, it can use the trough position between two wave crests at the both ends distance B Set the region for marking off the grouping of two characteristic points.Then, then judge in the two regions occupied by each region fisrt feature point Quantitative proportion whether meet preset condition.For example, when judging that quantitative proportion occupied by each region fisrt feature point is big When participating in the 10% of fisrt feature point total amount of statistics, k value can add 1, become 2.That is, the grouping of fisrt feature point Number can be 2, and the quantity of the first splicing regions marked off from the first image accordingly may be 2.When being determined After the quantity of one splicing regions, division circle of the first splicing regions can be further determined that according to the region that features described above point is grouped Limit.Then, the first image can be marked off according to the quantity and division limits of identified first splicing regions multiple First splicing regions.For example, can be referring initially to the region division mode on histogram above-mentioned, and utilize clustering algorithm (example Such as k-means algorithm, k=2) it by fisrt feature point clustering is 2 groups, and according to the fisrt feature click and sweep split-phase after cluster Two the first splicing regions up and down answered, as shown in Figure 5.Fig. 5 is shown according to an embodiment of the present invention, in example shown in Fig. 4 The division schematic diagram of first splicing regions.It may include corresponding one group first in the one or more of first splicing regions Wholly or largely (such as the one group of fisrt feature point that may include preset ratio (such as 90%)) of characteristic point.Wherein, optional Ground can also obtain the data center of every group of fisrt feature point during clustering to fisrt feature point.

The division mode of the grouping of the point of fisrt feature shown in Fig. 4 and Fig. 5 and cluster mode and the first splicing regions is only It can also be using other any characteristic point groupings and region division mode in the practical application of the embodiment of the present invention for example. For example, in another example, can also be drawn according to spatial distribution of the texture of the first image, color, light and shade and/or object etc. Divide multiple first splicing regions in the first image.It further optionally, can for dividing the fisrt feature point of the first splicing regions Think the whole of fisrt feature point in the first image, or a part of fisrt feature point correspondingly marks off First splicing regions also can take up the whole of the first image, or only occupy a portion.Optionally, used first Characteristic point and the first splicing regions marked off, which can be located in the first image, has overlapping part with the second image.In addition, In the practical application of the embodiment of the present invention, the grouping of fisrt feature point and the first splicing regions of other quantity can also be marked off, Herein with no restrictions.For example, fisrt feature point can also be marked off upper, middle and lower three in another example of the embodiment of the present invention Group, and the first splicing regions are marked off into three the first splicing regions of corresponding upper, middle and lower.Optionally, adjacent two first spellings The line of demarcation connect between region can not be parallel with the splicing edge direction of the first image.

It, can be every to calculating separately according to Feature Points Matching after the first image is marked off multiple first splicing regions First homography matrix of a first splicing regions.For example, can be for two the first splicing regions difference up and down shown in fig. 5 Corresponding two the first homography matrixes up and down are calculated, for example can be respectively H_topAnd H_bottom.It is optional in calculating process Ground can estimate that each first spells using such as random sampling consistency (Random Sample Consensus, RANSAC) Connect the homography matrix in region.This mode for being directed to the first picture portion domain and estimating multiple homography matrixes, can be avoided it It is preceding to be invariably prone to estimate homography matrix based on relatively dense data in the prior art, to easily ignore other effectively Sparse data the problem of, improve homography matrix calculating accuracy and validity, improve image mosaic precision.When So, the calculation of above-mentioned first homography matrix depends on the division mode and quantity of the first splicing regions, for example, when first When splicing regions are along upper, middle and lower three of the first image mosaic edge direction, the first homography matrix may include H_top、H_mid And H_bottom, herein with no restrictions.

In step s 103, the first image is marked off into multiple first grids, and according to described at least two first At least one of described first homography matrix of splicing regions calculates the first net of each first grid of the first image Lattice homography matrix.

In this step, the first image is marked off into multiple first grids first, it specifically, can be according to the edge of image Feature, characteristic point or other information carry out gridding to the first image using various image lattice division methods.It then, can be with According to each first homography matrix of each first splicing regions of aforementioned correspondence, to calculate of each first grid in the first image One grid homography matrix.It optionally, first can be with when calculating the first grid homography matrix of one of them the first grid This first grid is calculated respectively at a distance from each first splicing regions, for example, the mass center difference of this first grid can be calculated Data center with the group of fisrt feature point in each first splicing regions is at a distance from splicing edge direction.Then, Ke Yigen The first homography matrix for corresponding to each of this first grid first splicing regions is obtained according to calculated distance Weight (such as alpha Method for Weight Distribution can be used), and according to the weight of each first splicing regions and described first Homography matrix calculates the first grid homography matrix of this first grid.Optionally, above-mentioned first grid homography is being obtained After matrix, corresponding first grid homography square can also be calculated to the first grid each of is marked off in the first image Battle array.

For example, the first homography of upper and lower two the first splicing regions can be utilized respectively in the example of Fig. 4 and Fig. 5 Matrix H_topAnd H_bottomTo calculate each first grid homography matrix.Specifically, it can calculate first in vertical direction, In first grid mass center in upper and lower two splicing regions at a distance from the data center of the group of fisrt feature point, such as divide It Wei not d_topAnd d_bottom, the first homography matrix of each of this first grid is then corresponded to according to calculated Distance Judgment H_topAnd H_bottomWeight W_topAnd W_bottom, finally according to the first homography matrix H_topAnd H_bottomIts corresponding weight W_topAnd W_bottomTo calculate the first grid homography matrix H=W of this first grid_top×H_top+W_bottom×H_bottom.It is calculating First homography matrix H_topAnd H_bottomWeight W_topAnd W_bottomDuring, in one example, when this first grid Mass center is higher than the corresponding data center C of the first splicing regions above_topWhen, W_topIt can be 1, W_bottomIt can be 0；When this The mass center of one grid is lower than the following corresponding data center C of the first splicing regions_bottomWhen, W_topIt can be 0, W_bottomIt can be with It is 1；And in other cases, W_bottom=d_top/(d_top+d_bottom), W_top=1-W_bottom.In another example, when this The mass center of one grid is higher than the corresponding data center (C of the first splicing regions above_top+ θ) when (θ is positive value), W_topCan be 1, W_bottomIt can be 0；When the mass center of this first grid is lower than the following corresponding data center (C of the first splicing regions_bottom- When θ), W_topIt can be 0, W_bottomIt can be 1；And in other cases, W_bottom=d_top/(d_top+d_bottom), W_top=1- W_bottom。

It is above-mentioned merely illustrative to the calculation of the first grid homography matrix, in the practical application of the embodiment of the present invention In, it may be considered that concrete application situation determines the first grid list using the calculating and weight expression way of any homography matrix Answering property matrix, herein with no restrictions.

In step S104, by the first grid of each of the first image according to its corresponding first grid homography Matrix is coordinately transformed, and in conjunction with second image to form image after splicing.

In this step, it can use the corresponding first grid homography matrix of each first grid to carry out the first grid Coordinate transform, and spliced with the original image of the second image or corresponding deformation, so that the first image and the second image Lap overlap, formed splicing after image.It specifically, can be by one or more angle points of each first grid point It is not used for the first grid homography matrix of coordinate transform, multiplied by it to obtain its coordinate after splicing in image；Then, may be used With on the basis of coordinate of the angle point being calculated after splicing in image, by the benefit respectively of other pixels in the first grid It is calculated with interpolation algorithm and obtains its corresponding coordinate after splicing in image；Finally, being schemed after splicing according to each pixel Coordinate correspondence relationship as in, image after filling splicing, to be spliced with the original image of the second image or corresponding deformation.

In embodiments of the present invention, the first grid marked off in the first image can take up the whole of the first image, It can take up a part therein.It, can also be to its in the first image when the first grid occupies a part of the first image He is partially in the way of other matrixing, such as similarity transformation, transition transformation etc., carries out grid dividing and coordinate transform, And it is packed into image after splicing.

In one example of an embodiment of the present invention, the original image of the second image can be packed into image after splicing, To be spliced with the first image after coordinate transform.In the example present, it is also assumed that each pixel in the second image Or each grid is multiplied with a unit matrix respectively, and is packed into image after splicing.

In another example of the embodiment of the present invention, the second image can also be carried out similar with above-mentioned first image Homography matrix calculate and coordinate transform, and by the second image completion after coordinate transform enter splicing after image, with obtain more Accurate image mosaic effect.Wherein, the image split-joint method of the embodiment of the present invention can also include: to draw second image At least two second splicing regions are separated, it is single to calculate separately each second splicing regions second according to the Feature Points Matching Answering property matrix；Second image is marked off into multiple second grids, and according to the institute of at least two second splicing regions At least one of second homography matrix is stated, the second grid homography square of each second grid of the second image is calculated Battle array.It is alternatively possible to the relationship of the first splicing regions and the Feature Points Matching pair divided according to aforementioned first image, by institute It states the second image and marks off at least two second splicing regions.Specifically, the first splicing that the first image marks off is being obtained After region, according to included in each first splicing regions every group of fisrt feature point, in conjunction in step s101 The relationship of the Feature Points Matching pair arrived, to obtain minute of the second feature point in corresponding second image of every group of fisrt feature point Group, and determine according to the characteristic point in the second image after grouping the division limits of the second splicing regions.It then, can be according to institute Second image is marked off multiple second splicing regions by the grouping of determining second feature point and division limits.Second splicing In the one or more in region, wholly or largely (such as may include pre- for corresponding one group of second feature point may include If one group of second feature point of ratio (such as 90%)).In one example, the quantity of the second splicing regions can be in the second image Identical as the quantity of the first splicing regions, the point of second feature included in the second splicing regions can be with corresponding first splicing Fisrt feature point in region corresponds substantially.

The division mode of above-mentioned second splicing regions is merely illustrative, can be in the practical application of the embodiment of the present invention Using other any characteristic point groupings and region division mode.It can also be independent to the division of the second splicing regions of the second image It carries out, is not influenced by the division result of the first splicing regions, specific division mode can be with aforementioned first splicing regions Division mode is similar, and details are not described herein.For example, in another example, can also according to the texture of the second image, color, Light and shade and/or the spatial distribution of object etc. divide the second splicing regions in the second image.Further optionally, for dividing The second feature point of second splicing regions can be the whole of second feature point in the second image, or second feature point A part, correspondingly, marking off the second splicing regions come also can take up the whole of the second image, or only occupy wherein A part.Optionally, the second splicing regions marked off, which can be located in the second image, has overlapping part with the first image. In addition, can also mark off the second splicing regions of other quantity in the practical application of the embodiment of the present invention, not limit herein System.Optionally, the line of demarcation between two adjacent the second splicing regions can not be flat with the splicing edge direction of the second image Row.

It, can be every to calculating separately according to Feature Points Matching after the second image is marked off multiple second splicing regions Second homography matrix of a second splicing regions.For example, can be calculated separately when there are upper and lower two the second splicing regions Corresponding two the second homography matrixes up and down, for example can be respectively H '_topAnd H '_bottom.In calculating process, optionally, Each second splicing can be estimated using such as random sampling consistency (Random Sample Consensus, RANSAC) The homography matrix in region.This mode for being directed to the second picture portion domain and estimating multiple homography matrixes, before can be avoided It is invariably prone to estimate homography matrix based on relatively dense data in the prior art, so that it is effective easily to ignore other The problem of sparse data, improves the accuracy and validity of homography matrix calculating, improves image mosaic precision.Certainly, The calculation of above-mentioned second homography matrix depends on the division mode and quantity of the second splicing regions, for example, when the second spelling When connecing region as along upper, middle and lower three of the second image mosaic edge direction, the second homography matrix may include H '_top、H’_mid And H '_bottom, herein with no restrictions.

In obtaining the second image after the corresponding homography matrix of the second splicing regions, the second image can be marked off Multiple second grids specifically can be according to the edge feature of image, characteristic point or other information, using various image lattices Division methods carry out gridding to the second image.Then, it can singly be answered according to each the second of each second splicing regions of aforementioned correspondence One or more of property matrix, to calculate the second grid homography matrix of each second grid in the second image.Optionally, When calculate one of them the second grid the second grid homography matrix when, can calculate first this second grid respectively with it is each The distance of second splicing regions, for example, the mass center of this second grid can be calculated respectively in each second splicing regions second Distance of the data center of the group of characteristic point along splicing edge direction.Then, it can be obtained and be corresponded to according to calculated distance In the second homography matrix of each of this second grid second splicing regions weight (such as can using alpha weigh Reassignment method), and this second grid is calculated according to the weight of each second splicing regions and second homography matrix The second grid homography matrix, specific weight calculation mode can be with the calculating of aforementioned first grid homography matrix weight Mode is similar, and details are not described herein.It optionally, can also be to second after obtaining above-mentioned second grid homography matrix The second grid each of is marked off in image calculates corresponding second grid homography matrix.It is above-mentioned to the second grid homography The calculation of matrix is merely illustrative, in the practical application of the embodiment of the present invention, it may be considered that concrete application situation, which uses, appoints The calculating of what homography matrix and weight expression way determine the second grid homography matrix, herein with no restrictions.

Correspondingly, described after carrying out the second grid dividing and the calculating of the second grid homography matrix to the second image In conjunction with second image with formed splicing after image can also include: by each of second image the second grid according to Its corresponding second grid homography matrix is coordinately transformed, and is schemed after forming splicing in conjunction with the first image after coordinate transform Picture.Specifically, one or more angle points of each second grid can be used for multiplied by it to the second grid of coordinate transform respectively Homography matrix, to obtain its coordinate after splicing in image；Then, can in the angle point being calculated after splicing image In coordinate on the basis of, other pixels in the second grid are utilized respectively interpolation algorithm and calculates and obtains it after splicing Corresponding coordinate in image；Finally, the coordinate correspondence relationship according to each pixel after splicing in image, schemes after filling splicing Picture, to be spliced with the first image after coordinate transform.

In embodiments of the present invention, the second grid marked off in the second image can take up the whole of the second image, It can take up a part therein.It, can also be to its in the second image when the second grid occupies a part of the second image He is partially in the way of other matrixing, such as similarity transformation, transition transformation etc., carries out grid dividing and coordinate transform, And it is packed into image after splicing.

Fig. 6 is shown according to an embodiment of the present invention, implements image mosaic to the first image shown in Fig. 3 and the second image The schematic diagram of image after the splicing of method.As can be seen that splicing shown in fig. 6 after image splicing adjacent edges transition more Naturally, splicing effect is more accurate.

However, can be seen that the spelling using the embodiment of the present invention with its enlarged diagram according to the content in Fig. 6 dotted line frame The image that the method for connecing is spliced may cause certain direction distortion, so that should near normal in image Object produce the rotation and offset of angle.Therefore, in an example of the present invention, image split-joint method can also include: The direction distortion of image after splicing is corrected.For example, the direction that can use image after orientation consistency correction splicing is abnormal Become.Specifically, straight-line detection can be carried out respectively in image after the first image, the second image and splicing first, obtain straight line Testing result.It is alternatively possible to be filtered to straight-line detection result and remove noise.It then, can be according to before obtaining Grid between image after corresponding relationship and/or preceding first image of splicing, the second image and splicing between Feature Points Matching pair Corresponding relationship etc. obtains the corresponding relationship between the obtained straight line of detection, and calculates certain straight line and phase after detection in image Angle ω in the first image and/or the second image answered between straight line.Finally, according to this angle ω in image after splicing Straight line carries out correction for direction, such as rotates, so that the straight line after splicing in image is straight before direction is distorted close to generating as far as possible Angle direction of the line in the first image and/or the second image.Fig. 7 is shown according to an embodiment of the present invention, by the void in Fig. 6 Wire frame inner region carries out the schematic diagram of image after the splicing after orientation consistency correction, it is seen then that image after splicing shown in Fig. 7 Straight line in solid box produces direction transformation, is closer to the direction in the second image on the right side of original Fig. 3.

In another example of the embodiment of the present invention, optionally, can also splicing edge to image after splicing into Row image co-registration, to eliminate the discontinuity of brightness of image or illumination as far as possible.For example, can be to the splicing edge of image after splicing Place carries out gradual change processing, to make brightness of image, illumination for splicing both sides of edges etc. consistent as far as possible as far as possible.

Although the image split-joint method of the above embodiment of the present invention only describes the implementation spliced for two images Mode, still, the above method of the embodiment of the present invention are applied equally to the splicing for three and three images above, Specific embodiment is similar with aforementioned image splicing, and details are not described herein.In addition, being directed to three and three or more Image mosaic during, image after being coordinately transformed to these images and disposably be spliced simultaneously can also be with Gradation processing is carried out for wherein adjacent two or more images and finally obtains image after splicing, it is not limited here.

Above-mentioned image split-joint method according to an embodiment of the present invention, can be according to mark off the first image multiple first Splicing regions are respectively formed the first different homography matrixes, and according to the first different homography matrix come to net The first image formatted is coordinately transformed and splices, and to improve the precision of image after splicing obtained, improves splicing effect.

In the following, describing image splicing device according to an embodiment of the present invention referring to Fig. 8.Fig. 8 is shown according to the present invention The block diagram of the image splicing device 800 of embodiment.Wherein, described image includes at least the first image and the second image.Such as Fig. 8 institute Show, image splicing device 800 includes matching unit 810, matrix calculation unit 820, grid dividing unit 830 and coordinate transform list Member 840.Other than these units, device 800 can also include other component, however, since these components and the present invention are real The content for applying example is unrelated, therefore omits its diagram and description herein.Further, since image mosaic according to an embodiment of the present invention The detail for the operations described below that device 800 executes is identical as the details described above with reference to Fig. 1 and Fig. 3-Fig. 7, therefore The repeated description to same detail is omitted herein in order to avoid repeating.

The matching unit 810 of image splicing device 800 in Fig. 8 is configured to the first image and the second image to be spliced Characteristic point detection and matching are carried out, multiple Feature Points Matchings pair are obtained, wherein each described Feature Points Matching is to including described The fisrt feature point of first image and the second feature point of second image.

In embodiments of the present invention, it is expected that being that a width range is bigger by the first image and second image mosaic Image after splicing.Wherein, it can have lap each other in the first image and the second image, also, lap distinguished Position in the first image and the second image is herein with no restrictions.For example, the lap of the first image and the second image Can be located at the right side of the first image and the left side of the second image, thus after splicing image left-half basic source in First image, and right half part basic source image after the second image, splicing becomes the bigger image of a width range.In this hair In another bright example, it is also possible that lap is located at the downside of the first image and the upside of the second image, at this In the case of kind, the first image and the second image will realize splicing up and down, and the top half basic source of spliced image is in the One image, and lower half portion basic source is in the second image.In another of the invention example, the first image and the second image Lap can be located at any position in the first image and the second image, and have certain rotation angle each other Degree, thus spliced image will also rotate a certain angle composition according at least one of the first image and the second image. Foregoing description is merely illustrative, in practical applications, can also use the overlap mode of any first image and the second image.

First image to be spliced and the second image are carried out characteristic point detection and matching by matching unit 810, more to obtain A Feature Points Matching pair.Specifically, it is possible, firstly, to obtain the first image and second image respectively.In the present invention one In a example, the first image and the second image can be matched on object (such as mobile robot, intelligent vehicle, unmanned plane etc.) respectively The image that standby shooting unit obtains, wherein shooting unit can be monocular camera or video camera, naturally it is also possible to be binocular or More mesh cameras or video camera, it is not limited here.Acquired the first image and the second image can be different time, no respectively With what is obtained in position or certain angular field of view, as long as having certain lap each other.

After obtaining the first image and the second image, matching unit 810 can be detected based on preset characteristic point Mode carries out characteristic point detection to the first image and the second image respectively.In embodiments of the present invention, preset characteristic point detection Mode may include such as Scale invariant features transform (Scale Invariant Feature Transform, SIFT) feature, Accelerate the various feature point detecting methods such as steady (Speeded Up Robust Features, SURF) feature, Harris angle point, It may be ORB (Oriented FAST and Rotated BRIEF) feature point detecting method.It, can after detecting characteristic point Selection of land can be described the characteristic point of the first image and the second image detection, such as can be special using gray feature, gradient Sign, parallax information etc. are various to be used for the methods that feature describes to describe the characteristic point in the first image and the second image.

Finally, matching unit 810 can arrive characteristic point that the first image detects and second image detection Characteristic point matched, to obtain multiple Feature Points Matchings pair.Optionally, matching unit 810 can use based on grid Movement statistics (Grid-based Motion Statistics, GMS) Lai Jinhang Feature Points Matching.It, can in Feature Points Matching With by matching correctness judgement, with eliminate it is certain mistake, can not matched characteristic point, only leave correct spy Sign point and corresponding Feature Points Matching pair, to improve matched stability.After matching finishes, acquired in matching unit 810 Each Feature Points Matching to include the first image fisrt feature point and second image second feature point, Wherein, the characteristic point that all fisrt feature points of the first image detect in the first image before being all from, can be first Image detection to characteristic point in whole or in which a part；Similarly, all second feature points of the second image can also be with The second image detection to characteristic point in whole or in which a part.All fisrt feature points and second in first image All second feature points of image are one-to-one relationships, respectively constitute each Feature Points Matching pair.Implement in the present invention In one example of example, acquired Feature Points Matching is to can be four or more.Fig. 3 shows a reality of the invention It applies in example and carries out characteristic point detection using ORB, and utilize the signal of the GMS Feature Points Matching pair obtain after Feature Points Matching Figure, wherein left side is the first image, and right side is the second image.Wherein, characteristic point detection is carried out using ORB in example shown in Fig. 3 And Feature Points Matching is carried out using GMS, there can be stronger robustness to image rotation and change of scale, after reducing matching The error of characteristic point.

The first image is marked off at least two first splicing regions by matrix calculation unit 820, according to the feature Point matching is to the first homography matrix for calculating separately each first splicing regions.

Optionally, matrix calculation unit 820 can be according at least partly fisrt feature point in the first image, by institute It states the first image and marks off at least two first splicing regions.Wherein, the mode for dividing the first splicing regions to the first image can Based on the grouping and cluster to fisrt feature point part or all of in the first image.

In one example, matrix calculation unit 820 can be according to the quantity of the fisrt feature point along splicing edge direction Fisrt feature point is grouped, and is clustered fisrt feature point using clustering algorithm.Specifically, institute can be calculated first At least partly fisrt feature point in the first image is stated along the fisrt feature point quantity of the splicing edge direction of the first image Distribution histogram.Fig. 4 shows in the first image according to an embodiment of the invention the schematic diagram of fisrt feature point and along spelling The corresponding distributed number histogram of edge direction is connect, the first image in Fig. 4 corresponds to first image in left side in Fig. 3.? As can be seen that the fisrt feature point quantity that has of top half of the first image is relatively more in Fig. 4, and what lower half portion had Fisrt feature point negligible amounts.Then, first splicing can be determined according to the fisrt feature point quantity distribution histogram The quantity in region.Specifically, as shown in figure 4, can be fitted to the distributed number histogram of fisrt feature point, such as using high This fitting is fitted, and parameters such as derivative, second dervative, peak value or halfwidth of each point on curve after digital simulation, And the quantity of the first marked off splicing regions namely the quantity of fisrt feature point grouping are determined according to above-mentioned parameter.For example, When carrying out the grouping of fisrt feature point according to Gauss curve fitting curve, a k value can be preset, for example, 1, then further according to difference The distance between adjacent peak judges the variation of k value on quantitative proportion occupied by fisrt feature point and/or curve in region.Such as Fig. 4 is it is found that have multiple wave crests and multiple troughs in Gauss curve fitting curve.For the Gauss curve fitting curve of diagram, it may be considered that The distance between adjacent peak, when in each region that this distance is greater than certain preset threshold, and marks off accordingly When quantitative proportion occupied by fisrt feature point also complies with preset condition, k value can add 1.In Fig. 4, distance A and other peak values Between be not above pre-determined distance, and cross pre-determined distance apart from B ultrasound.On this basis, two of the both ends distance B be can use Wave trough position between wave crest marks off the region of two characteristic points grouping.Then, then judge each region in the two regions Whether quantitative proportion occupied by fisrt feature point meets preset condition.For example, when judging shared by each region fisrt feature point According to quantitative proportion be all larger than participate in statistics fisrt feature point total amount 10% when, k value can add 1, become 2.That is, The packet count of fisrt feature point can be 2, and the quantity of the first splicing regions marked off from the first image accordingly can also Think 2.After the quantity of the first splicing regions has been determined, first can be further determined that according to the region that features described above point is grouped The division limits of splicing regions.It then, can be according to the quantity of identified first splicing regions and division limits by described One image marks off multiple first splicing regions.For example, can referring initially to the region division mode on histogram above-mentioned, and Using clustering algorithm (such as k-means algorithm, k=2) by fisrt feature point clustering be 2 groups, and according to cluster after Fisrt feature click and sweep is divided to corresponding two the first splicing regions up and down, as shown in Figure 5.Fig. 5 shows an implementation according to the present invention , the division schematic diagram of the first splicing regions in example shown in Fig. 4.It can wrap in the one or more of first splicing regions Containing corresponding one group of fisrt feature point wholly or largely (such as may include one group first of preset ratio (such as 90%) it is special Sign point).Wherein, optionally, during being clustered to fisrt feature point, in data that every group of fisrt feature point can also be obtained The heart.

The division mode of the grouping of the point of fisrt feature shown in Fig. 4 and Fig. 5 and cluster mode and the first splicing regions is only For example, in the practical application of the embodiment of the present invention, matrix calculation unit 820 can also be using other any characteristic point groupings With region division mode.For example, in another example, matrix calculation unit 820 can also texture, color according to the first image Color, light and shade and/or spatial distribution of object etc. divide multiple first splicing regions in the first image.Further optionally, it uses In the whole that the fisrt feature point for dividing the first splicing regions can be fisrt feature point in the first image, or first is special A part of point is levied, correspondingly, marking off the first splicing regions come also can take up the whole of the first image, or only account for According to a portion.Optionally, used fisrt feature point and the first splicing regions marked off can be located at the first image In with the second image have overlapping part.In addition, other quantity can also be marked off in the practical application of the embodiment of the present invention Fisrt feature point grouping and the first splicing regions, herein with no restrictions.For example, in another example of the embodiment of the present invention, Fisrt feature point can also be marked off to three groups of upper, middle and lower, and the first splicing regions are marked off into corresponding upper, middle and lower three first Splicing regions.Optionally, the line of demarcation between two adjacent the first splicing regions can not be with the splicing edge of the first image Direction is parallel.

After the first image is marked off multiple first splicing regions, matrix calculation unit 820 can be according to characteristic point Matching is to the first homography matrix for calculating separately each first splicing regions.For example, can be for two up and down shown in fig. 5 First splicing regions calculate separately corresponding two the first homography matrixes up and down, for example can be respectively H_topAnd H_bottom.? In calculating process, it is alternatively possible to using such as random sampling consistency (Random Sample Consensus, RANSAC) To estimate the homography matrix of each first splicing regions.Estimate multiple homography matrixes in this first picture portion domain that is directed to Mode is invariably prone to estimate homography matrix based on relatively dense data before can be avoided in the prior art, from And the problem of easily ignoring other effective sparse datas, the accuracy and validity of homography matrix calculating are improved, is improved Image mosaic precision.Certainly, the calculation of above-mentioned first homography matrix depend on the first splicing regions division mode and Quantity, for example, when the first splicing regions are along upper, middle and lower three of the first image mosaic edge direction, the first homography matrix It may include H_top、H_midAnd H_bottom, herein with no restrictions.

The first image can be marked off multiple first grids by grid dividing unit 830, and according to described at least two At least one of described first homography matrix of a first splicing regions calculates each first grid of the first image First grid homography matrix.

First image can be marked off multiple first grids first by grid dividing unit 830, specifically, can be according to figure Edge feature, characteristic point or the other information of picture carry out gridding to the first image using various image lattice division methods.With Afterwards, grid dividing unit 830 can be according to each first homography matrix of each first splicing regions of aforementioned correspondence, to calculate First grid homography matrix of each first grid in first image.Optionally, when calculating the of one of them the first grid When one grid homography matrix, this first grid can be calculated first respectively at a distance from each first splicing regions, for example, can To calculate the mass center of this first grid respectively with the data center of the group of fisrt feature point in each first splicing regions along splicing The distance of edge direction.Then, it can be obtained according to calculated distance and correspond to each of this first grid described first The weight (such as alpha Method for Weight Distribution can be used) of first homography matrix of splicing regions, and spelled according to each first The weight and first homography matrix that connect region calculate the first grid homography matrix of this first grid.It is optional It ground can also be equal to the first grid each of is marked off in the first image after obtaining above-mentioned first grid homography matrix Calculate corresponding first grid homography matrix.

For example, grid dividing unit 830 can be utilized respectively two first spellings up and down in the example of Fig. 4 and Fig. 5 Meet the first homography matrix H in region_topAnd H_bottomTo calculate each first grid homography matrix.It specifically, first can be with It calculates in vertical direction, the number of the group of fisrt feature point in the mass center of one of them the first grid and upper and lower two splicing regions According to the distance at center, such as respectively d_topAnd d_bottom, this first grid is then corresponded to according to calculated Distance Judgment Each first homography matrix H_topAnd H_bottomWeight W_topAnd W_bottom, finally according to the first homography matrix H_topAnd H_bottom Its corresponding weight W_topAnd W_bottomTo calculate the first grid homography matrix H=W of this first grid_top×H_top+ W_bottom×H_bottom.Calculating the first homography matrix H_topAnd H_bottomWeight W_topAnd W_bottomDuring, show at one In example, when the mass center of this first grid is higher than the corresponding data center C of the first splicing regions above_topWhen, W_topIt can be 1, W_bottomIt can be 0；When the mass center of this first grid is lower than the following corresponding data center C of the first splicing regions_bottomWhen, W_topIt can be 0, W_bottomIt can be 1；And in other cases, W_bottom=d_top/(d_top+d_bottom), W_top=1-W_bottom.? In another example, when the mass center of this first grid is higher than the corresponding data center (C of the first splicing regions above_top+ θ) when (θ is positive value), W_topIt can be 1, W_bottomIt can be 0；When the mass center of this first grid is lower than the first following splicing regions pair Data center (the C answered_bottom- θ) when, W_topIt can be 0, W_bottomIt can be 1；And in other cases, W_bottom=d_top/ (d_top+d_bottom), W_top=1-W_bottom。

It is above-mentioned merely illustrative to the calculation of the first grid homography matrix, in the practical application of the embodiment of the present invention In, grid dividing unit 830 is it is contemplated that concrete application situation uses the calculating and weight expression way of any homography matrix Determine the first grid homography matrix, herein with no restrictions.

Coordinate transformation unit 840 can be by the first grid of each of the first image according to its corresponding first grid Homography matrix is coordinately transformed, and in conjunction with second image to form image after splicing.

Coordinate transformation unit 840 can use the corresponding first grid homography matrix of each first grid to the first grid It is coordinately transformed, and is spliced with the original image of the second image or corresponding deformation, so that the first image and second The lap of image overlaps, and forms image after splicing.It specifically, can be by one or more angles of each first grid Point is used for the first grid homography matrix of coordinate transform multiplied by it respectively, to obtain its coordinate after splicing in image；With It afterwards, can be on the basis of coordinate of the angle point being calculated after splicing in image, by other pixels in the first grid Interpolation algorithm is utilized respectively to calculate and obtain its corresponding coordinate after splicing in image；Finally, being spelled according to each pixel Connect the coordinate correspondence relationship in rear image, image after filling splicing, with the original image of the second image or it is corresponding deform into Row splicing.

In one example of an embodiment of the present invention, coordinate transformation unit 840 can fill out the original image of the second image It is filled with image after splicing, to be spliced with the first image after coordinate transform.In the example present, it is also assumed that the second figure Each pixel or each grid as in are multiplied with a unit matrix respectively, and are packed into image after splicing.

In another example of the embodiment of the present invention, coordinate transformation unit 840 can also to the second image carry out with it is upper The first image similar homography matrix calculating and coordinate transform are stated, and the second image completion after coordinate transform is entered into splicing Image afterwards, to obtain more accurate image mosaic effect.On the basis of this, aforementioned matrix calculation unit 820 can also be by institute It states the second image and marks off at least two second splicing regions, spliced according to the Feature Points Matching to calculating separately each second Second homography matrix in region；And second image can also be marked off multiple second by aforementioned grid dividing unit 830 Grid, and according at least one of second homography matrix of at least two second splicing regions, described in calculating Second grid homography matrix of each second grid of the second image.

Optionally, matrix calculation unit 820 can be according to the first splicing regions and the spy that aforementioned first image divides The relationship of sign point matching pair, marks off at least two second splicing regions for second image.Specifically, first is being obtained After the first splicing regions that image marks off, every group of fisrt feature according to included in each first splicing regions Point, in conjunction with the relationship for the Feature Points Matching pair that matching unit 810 obtains, to obtain corresponding second figure of every group of fisrt feature point The grouping of second feature point as in, and determine according to the characteristic point in the second image after grouping the division of the second splicing regions Boundary.Then, the second image can be marked off multiple second according to the grouping and division limits of identified second feature point Splicing regions.In the one or more of second splicing regions, may include corresponding one group of second feature point whole or Most of (such as the one group of second feature point that may include preset ratio (such as 90%)).In one example, in the second image The quantity of second splicing regions can be identical as the quantity of the first splicing regions, second feature included in the second splicing regions Point can correspond substantially with the fisrt feature point in corresponding first splicing regions.

The division mode of above-mentioned second splicing regions is merely illustrative, in the practical application of the embodiment of the present invention, matrix meter Calculating unit 820 can also be using other any characteristic point groupings and region division mode.To the second splicing regions of the second image Division can also independently carry out, do not influenced by the division result of the first splicing regions, specific division mode can with it is preceding The division mode for stating the first splicing regions is similar, and details are not described herein.For example, in another example, it can also be according to second Spatial distribution of the texture of image, color, light and shade and/or object etc. divides the second splicing regions in the second image.In addition, Optionally, the second feature point for dividing the second splicing regions can be the whole of second feature point in the second image, can also Think a part of second feature point, correspondingly, marking off the second splicing regions come also can take up the complete of the second image Portion, or only occupy a portion.Optionally, the second splicing regions marked off can be located in the second image and first Image has the part of overlapping.In addition, can also mark off other quantity second is spelled in the practical application of the embodiment of the present invention Region is connect, herein with no restrictions.Optionally, the line of demarcation between two adjacent the second splicing regions can not be with the second image Splicing edge direction it is parallel.

After the second image is marked off multiple second splicing regions, matrix calculation unit 820 can be according to characteristic point Matching is to the second homography matrix for calculating separately each second splicing regions.For example, when there are upper and lower two the second splice regions When domain, corresponding two the second homography matrixes up and down can be calculated separately, for example can be respectively H '_topAnd H '_bottom.It is counting During calculation, it is alternatively possible to be come using such as random sampling consistency (Random Sample Consensus, RANSAC) Estimate the homography matrix of each second splicing regions.This side for being directed to the second picture portion domain and estimating multiple homography matrixes Formula is invariably prone to estimate homography matrix based on relatively dense data before can be avoided in the prior art, thus The problem of easily ignoring other effective sparse datas improves the accuracy and validity of homography matrix calculating, improves figure As splicing precision.Certainly, the calculation of above-mentioned second homography matrix depends on the division mode sum number of the second splicing regions Amount, for example, the second homography matrix can when the second splicing regions are along upper, middle and lower three of the second image mosaic edge direction To include H '_top、H’_midAnd H '_bottom, herein with no restrictions.

In obtaining the second image after the corresponding homography matrix of the second splicing regions, grid dividing unit 830 can be with Second image is marked off into multiple second grids, specifically, can according to the edge feature of image, characteristic point or other information, Gridding is carried out to the second image using various image lattice division methods.It then, can be according to each second splicing of aforementioned correspondence One or more of each second homography matrix in region, to calculate the second grid list of each second grid in the second image Answering property matrix.Optionally, when calculating the second grid homography matrix of one of them the second grid, can calculate first this Two grids are respectively at a distance from each second splicing regions, for example, the mass center of this second grid can be calculated respectively with each Distance of the data center of the group of second feature point along splicing edge direction in two splicing regions.It then, can be according to calculating The distance that arrives obtain correspond to each of this second grid the second homography matrix of second splicing regions weight (such as Alpha Method for Weight Distribution can be used), and the of this second grid is calculated according to the weight and second homography matrix Two grid homography matrixes, specific weight calculation mode can be with the calculations of aforementioned first grid homography matrix weight Similar, details are not described herein.It optionally, can also be to the second image after obtaining above-mentioned second grid homography matrix In each of mark off the second grid and calculate corresponding second grid homography matrix.It is above-mentioned to the second grid homography matrix Calculation it is merely illustrative, in the practical application of the embodiment of the present invention, it may be considered that concrete application situation use any list The calculating of answering property matrix and weight expression way determine the second grid homography matrix, herein with no restrictions.

Correspondingly, after carrying out the second grid dividing and the calculating of the second grid homography matrix to the second image, coordinate Converter unit 840 can be by the second grid of each of second image according to its corresponding second grid homography matrix It is coordinately transformed, forms image after splicing in conjunction with the first image after coordinate transform.Specifically, coordinate transformation unit 840 can One or more angle points of each second grid to be used for multiplied by it to the second grid homography matrix of coordinate transform respectively, with Obtain its coordinate after splicing in image；It then, can be in the base of coordinate of the angle point being calculated after splicing in image On plinth, by other pixels in the second grid be utilized respectively interpolation algorithm calculate and obtain its after splicing it is corresponding in image Coordinate；Finally, the coordinate correspondence relationship according to each pixel after splicing in image, image after filling splicing, with coordinate Transformed first image is spliced.

However, can be seen that the spelling using the embodiment of the present invention with its enlarged diagram according to the content in Fig. 6 dotted line frame The image that the method for connecing is spliced may cause certain direction distortion, so that should near normal in image Object produce the rotation and offset of angle.Therefore, in an example of the present invention, coordinate transformation unit 840 can also be right The direction distortion of image is corrected after splicing.For example, can use the direction distortion of image after orientation consistency correction splicing. Specifically, straight-line detection can be carried out respectively in image after the first image, the second image and splicing first, obtain straight-line detection As a result.It is alternatively possible to be filtered to straight-line detection result and remove noise.It then, can be according to the feature obtained before Point corresponding relationship of the matching between and/or preceding first image of splicing, the second image and after splicing between image grid it is corresponding Relationship etc. obtains the corresponding relationship between the obtained straight line of detection, and calculate certain straight line after detection in image with it is corresponding Angle ω in first image and/or the second image between straight line.Finally, according to this angle ω to the straight line in image after splicing Correction for direction is carried out, such as is rotated, so that the straight line after splicing in image exists close to the straight line before the distortion of generation direction as far as possible Angle direction in first image and/or the second image.Fig. 7 is shown according to an embodiment of the present invention, by the dotted line frame in Fig. 6 Inner region carries out the schematic diagram of image after the splicing after orientation consistency correction, it is seen then that the solid line of image after splicing shown in Fig. 7 Straight line in frame produces direction transformation, is closer to the direction in the second image on the right side of original Fig. 3.

In another example of the embodiment of the present invention, optionally, coordinate transformation unit 840 can also be to image after splicing Splicing edge carry out image co-registration, to eliminate the discontinuity of brightness of image or illumination as far as possible.For example, after can be to splicing The splicing edge of image carries out gradual change processing, to make brightness of image, illumination for splicing both sides of edges etc. consistent as far as possible as far as possible.

Although the image splicing device of the above embodiment of the present invention only describes the implementation spliced for two images Mode, still, the above method of the embodiment of the present invention are applied equally to the splicing for three and three images above, Specific embodiment is similar with aforementioned image splicing, and details are not described herein.In addition, being directed to three and three or more Image mosaic during, image after being coordinately transformed to these images and disposably be spliced simultaneously can also be with Gradation processing is carried out for wherein adjacent two or more images and finally obtains image after splicing, it is not limited here.

Above-mentioned image splicing device according to an embodiment of the present invention, can be according to mark off the first image multiple first Splicing regions are respectively formed the first different homography matrixes, and according to the first different homography matrix come to net The first image formatted is coordinately transformed and splices, and to improve the precision of image after splicing obtained, improves splicing effect.

In the following, describing image splicing device according to an embodiment of the present invention referring to Fig. 9.Fig. 9 is shown according to the present invention The block diagram of the image splicing device 900 of embodiment.Wherein, described image includes at least the first image and the second image.Such as Fig. 9 institute Show, which can be computer or server.

As shown in figure 9, image splicing device 900 includes that one or more processors 910 and memory 920 remove certainly Except this, image splicing device 900 is also possible that stereoscopic camera and output device with multiple panorama cameras (do not show Out) etc., these components can be interconnected by the bindiny mechanism of bus system and/or other forms.It should be noted that shown in Fig. 9 The component and structure of image splicing device 900 be it is illustrative, and not restrictive, as needed, image splicing device 900 Also other assemblies and structure be can have.

Processor 910 can be central processing unit (CPU) or have data-handling capacity and/or instruction execution capability Other forms processing unit, and can use the computer program instructions stored in memory 920 to execute expectation Function, may include: that the first image to be spliced and the second image are subjected to characteristic point detection and matching, obtain multiple features Point matching pair, wherein each described Feature Points Matching is to fisrt feature point and second image including the first image Second feature point；The first image is marked off at least two first splicing regions, according to the Feature Points Matching to point The first homography matrix of each first splicing regions is not calculated；The first image is marked off into multiple first grids, and root According at least one of first homography matrix of at least two first splicing regions, it is every to calculate the first image First grid homography matrix of a first grid；By the first grid of each of the first image according to its corresponding first Grid homography matrix is coordinately transformed, and in conjunction with second image to form image after splicing.

Memory 920 may include one or more computer program products, and the computer program product may include Various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.The volatibility is deposited Reservoir for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non-volatile Memory for example may include read-only memory (ROM), hard disk, flash memory etc..It can be on the computer readable storage medium One or more computer program instructions are stored, processor 910 can run described program instruction, to realize sheet described above The function of the image splicing device of the embodiment of invention and/or other desired functions, and/or basis can be executed The image split-joint method of the embodiment of the present invention.Can also be stored in the computer readable storage medium various application programs and Various data.

In the following, describing computer readable storage medium according to an embodiment of the present invention, it is stored thereon with computer program and refers to It enables, wherein the computer program instructions realize following image mosaic step when being executed by processor, wherein described image is at least Including the first image and the second image: the first image to be spliced and the second image being carried out characteristic point detection and matching, obtained Multiple Feature Points Matchings pair, wherein each described Feature Points Matching is to including the fisrt feature point of the first image and described The second feature point of second image；The first image is marked off at least two first splicing regions, according to the characteristic point Matching is to the first homography matrix for calculating separately each first splicing regions；The first image is marked off into multiple first nets Lattice, and according at least one of first homography matrix of at least two first splicing regions calculate described the First grid homography matrix of each first grid of one image；The first grid of each of the first image is right according to its The the first grid homography matrix answered is coordinately transformed, and in conjunction with second image to form image after splicing.

Therefore, the present invention is explained in detail by using above-described embodiment；However, those skilled in the art should understand this hair The bright embodiment for being not limited to resonable explanation.The present invention can without departing substantially from the scope of the present invention being defined by the claims To be implemented as correction, modification mode.Therefore, the description of specification is intended merely to explain example, and does not apply to the present invention Add any restrictions meaning.

Claims

1. a kind of image split-joint method, described image includes at least the first image and the second image, which comprises

First image to be spliced and the second image are subjected to characteristic point detection and matching, obtain multiple Feature Points Matchings pair, In each described Feature Points Matching to include the first image fisrt feature point and second image second feature Point；

The first image is marked off at least two first splicing regions, it is each to calculating separately according to the Feature Points Matching First homography matrix of the first splicing regions；

The first image is marked off into multiple first grids, and according to described the first of at least two first splicing regions At least one of homography matrix calculates the first grid homography matrix of each first grid of the first image；

The first grid of each of the first image is coordinately transformed according to its corresponding first grid homography matrix, And in conjunction with second image to form image after splicing.

2. the method for claim 1, wherein

The method also includes: second image is marked off at least two second splicing regions, according to the characteristic point Pairing calculates separately the second homography matrix of each second splicing regions；Second image is marked off into multiple second nets Lattice, and according at least one of second homography matrix of at least two second splicing regions calculate described the Second grid homography matrix of each second grid of two images；

Second image described in the combination is to form image after splicing further include: by the second grid of each of second image It is coordinately transformed according to its corresponding second grid homography matrix, after forming splicing in conjunction with the first image after coordinate transform Image.

3. the method for claim 1, wherein described mark off at least two first splicing regions for the first image Include:

According at least partly fisrt feature point in the first image, the first image is marked off at least two first spellings Connect region.

4. method as claimed in claim 3, wherein at least partly fisrt feature point according in the first image, The first image, which is marked off at least two first splicing regions, includes:

Calculate the first image at least partly fisrt feature point along the first image splicing edge direction first Characteristic point distributed number histogram；

The quantity of first splicing regions is determined according to the fisrt feature point quantity distribution histogram；

The first image is marked off into the first splicing regions according to identified quantity.

5. method according to claim 2, wherein described that second image is marked off at least two second splicing regions Include:

The relationship of the first splicing regions and the Feature Points Matching pair that are divided according to the first image, by second image Mark off at least two second splicing regions.

6. the method for claim 1, wherein described every according to first homography matrix calculating the first image First grid homography matrix of a first grid includes:

One of them first grid is calculated respectively at a distance from each first splicing regions；

The first homography matrix for corresponding to each of this first grid first splicing regions is obtained according to the distance Weight；

The first grid homography matrix of this first grid is calculated according to the weight and first homography matrix.

7. method according to claim 2, wherein described every according to second homography matrix calculating, second image Second grid homography matrix of a second grid includes:

One of them second grid is calculated respectively at a distance from each second splicing regions；

The second homography matrix for corresponding to each of this second grid second splicing regions is obtained according to the distance Weight；

The second grid homography matrix of this second grid is calculated according to the weight and second homography matrix.

8. a kind of image splicing device, described image includes at least the first image and the second image, described device include:

Matching unit is configured to the first image to be spliced and the second image carrying out characteristic point detection and matching, obtain multiple Feature Points Matching pair, wherein each described Feature Points Matching is to the fisrt feature point and described second including the first image The second feature point of image；

Matrix calculation unit is configured to marking off the first image at least two first splicing regions, according to the feature Point matching is to the first homography matrix for calculating separately each first splicing regions；

Grid dividing unit is configured to marking off the first image into multiple first grids, and according to described at least two At least one of described first homography matrix of one splicing regions calculates the first of each first grid of the first image Grid homography matrix；

Coordinate transformation unit is configured to answer the first grid of each of the first image according to its corresponding first grid list Property matrix be coordinately transformed, and in conjunction with second image with formed splicing after image.

9. a kind of image splicing device, described image includes at least the first image and the second image, described device include:

Processor；

And memory, it is stored with computer program instructions in the memory,

Wherein, when the computer program instructions are run by the processor, so that the processor executes following steps:

10. a kind of computer readable storage medium, is stored thereon with computer program instructions, the computer program instructions are located Reason device realizes following image mosaic step when executing, and wherein described image includes at least the first image and the second image:

First image to be spliced and the second image are subjected to characteristic point detection and matching, obtained more

A Feature Points Matching pair, wherein each described Feature Points Matching is to the fisrt feature point and institute for including the first image State the second feature point of the second image；