CN112907452A - Optimal suture line searching method for image stitching - Google Patents

Abstract

The invention provides an optimal suture line searching method based on a mixed norm. The method calculates the cost of adjacent pixels by using L₁And L₂The norm, in combination, can well balance image smoothing with edge detail information. Meanwhile, the cost function adopts an exponential form, so that the color difference weight can be amplified, the distinction between the edge of the remarkable object and the non-edge area in the overlapped area is strengthened, and the searched suture line is prevented from passing through the remarkable object in the overlapped area. Firstly, respectively extracting feature points of each image by using an SIFT algorithm, and matching the feature points; then calculating a homography matrix between adjacent images, and finding a superposition area between the images through the homography matrix; and carrying out optimal suture line searching based on the mixed norm in the overlapping area, and completing image splicing on the basis of obtaining the optimal suture line.

Description

Optimal suture line searching method for image stitching

Technical Field

The invention belongs to the technical field of image processing, and particularly relates to an optimal suture line searching method for image splicing.

Background

The image stitching is to stitch two or more images containing overlapping areas in the same scene into one image with high resolution and wide viewing angle. The image splicing technology aims to increase the image visual angle and is beneficial to more intuitively and comprehensively grasping information. More contents can be presented than the image of the general view. The technology is widely applied to the aspects of medical imaging, remote sensing technology, virtual reality, video editing and the like.

The classic image stitching process mainly comprises two key steps of image registration and image fusion. At present, the mainstream image registration method is a method based on local features, and image fusion is to realize smooth transition of an image overlapping region and reduce the influence of illumination on an image. However, in a scene with a moving object or objects with different depths, the conventional linear weighting or multiband fusion method easily causes the problem of the stitching quality of the stitched images, such as object dislocation or artifacts in the coincident region. The optimal suture line searching method can solve the above problems well.

At present, researchers at home and abroad have made many studies on the optimal suture line search for Image stitching, and have achieved good results, such as a perception-based seam cutting method (Nan L, Liao T, Chao w.perspective-based seam stitching for Image stitching [ J ]. Signal Image and Video Processing,2018,12(3):1-8.), a stitching method based on a more optimal canonical energy function (Qiu X, Li q.

Disclosure of Invention

The invention provides an optimal suture line searching method based on a mixed norm. The method calculates the cost of adjacent pixels by using L₁And L₂The norm, in combination, can well balance image smoothing with edge detail information. Meanwhile, the cost function adopts an exponential form, so that the color difference weight can be amplified, the distinction between the edge of the remarkable object and the non-edge area in the overlapped area is strengthened, and the searched suture line is prevented from passing through the remarkable object in the overlapped area. Firstly, respectively extracting feature points of each image by using an SIFT algorithm, and matching the feature points; then calculating a homography matrix between adjacent images, and finding a superposition area between the images through the homography matrix; and carrying out optimal suture line searching based on the mixed norm in the overlapping area, and completing image splicing on the basis of obtaining the optimal suture line.

The technical scheme adopted by the invention comprises the following steps:

(1) reading two images to be spliced, and respectively recording the two imagesI₁And I₂；

(2) Image I by SIFT algorithm₁And I₂Extracting characteristic points;

(3) matching the detected characteristic points by using a K-D tree to obtain characteristic point pairs;

(4) performing feature point pair elimination (mismatching) on the image with the marked feature points by adopting an RANSAC algorithm and solving a homography matrix H;

(5) for image I according to homography matrix H₁And I₂Transforming, and recording the two transformed images as J₁And J₂；

(6) Obtaining a coincidence region omega and searching an optimal suture line;

(6a) taking image J₁And J₂Obtaining an omega image of a superposition area;

(6b) calculating image J₁Boundary B with overlap region omega₁And image J₂Boundary B with overlap region omega₂And the boundary B₁And boundary B₂The two intersections are respectively marked as S and T;

(6c) traversing all pixel points in the overlapping area, and respectively calculating the cost between each pixel point p and the pixel points q in the four adjacent areas, wherein the cost function is calculated as follows:

(1) if pixel point p or q is located at boundary B₁∪B₂Then, the cost between the two points is assigned to infinity;

(2) if the pixel points p and q are positioned in the same image J₁Or J₂If so, the cost between the two points is assigned to be 0;

(3) if the pixels p and q are located in different images, the cost V (p, q) between the two points

Calculated according to equation (1):

V(p,q)＝V(p)+V(q) (1)

V(x)＝[ω·||x||₂+(1-ω)·||x||₁]⁴ (2)

wherein, omega is a weight coefficient, omega belongs to [0,1 ]]，||x||₂＝||J₁(x)-J₂(x)||₂Is L₂Norm of，||x||₁＝||J₁(x)-J₂(x)||₁Is L₁And (4) norm.

(6d) Starting from the starting point S, searching the link with the minimum cost according to the cost among the pixels calculated in the step (6c) until the ending point T stops, and then the curve formed by the series of points is the optimal suture line.

Compared with the prior art, the invention has the following advantages:

(1) is combined with L₁Norm and L₂Norm dominance, L₁The norm has the advantage of preserving image edge information, L₂The norm has the characteristic of noise suppression on a flat region of an image, so that the cost function based on the mixed norm can not only smooth the image, but also well maintain the edge detail information of the image.

(2) The cost function is in an exponential form, so that the color difference weight can be amplified, the distinction between the edge and the non-edge area of the remarkable object in the overlapping area is strengthened, and the searched suture line is prevented from passing through the remarkable object in the overlapping area.

Therefore, the invention can search an optimal suture line under larger parallax, and can effectively reduce or avoid the splicing quality problems of object target dislocation, artifacts and the like.

Drawings

FIG. 1 is a flow chart of the present invention for implementing optimal suture line search for image stitching.

Fig. 2(a) is a first experimental image used in this example.

Fig. 2(b) is a second experimental image used in this example.

Fig. 3(a) is a transformed image of the first experimental image.

Fig. 3(b) is a transformed image of the second experimental image.

Fig. 4 is an image of the overlapping area of two transformed images.

Fig. 5 is an example of searching for an optimal suture line.

FIG. 6 is a splice diagram of the present invention for performing an optimal suture search of FIGS. 2(a) (b).

Fig. 7(a) is a mosaic of a perception-based seam cutting method.

Fig. 7(b) is a partially enlarged view of fig. 7 (a).

Fig. 8(a) is a splicing diagram of a splicing method based on a more optimal canonical energy function.

Fig. 8(b) is a partially enlarged view of fig. 8 (a).

Detailed Description

The invention and the technical effects are explained in detail by the drawings and the concrete examples as follows:

the present embodiment provides an optimal suture line searching method for image stitching, a specific flowchart is shown in fig. 1, and the specific implementation steps are as follows:

step 1, reading two images to be spliced, and respectively marking as I₁And I₂The images to be processed in this example are shown in fig. 2(a) and 2 (b). The two images are shot from two angles to the same large scene, and the two images are guaranteed to have a common area.

Step 2, for image I₁And I₂And respectively extracting the feature points by using a SIFT algorithm. The SIFT algorithm is selected because SIFT features have good stability and invariance, can adapt to rotation, scale scaling and brightness change, and can be free from the interference of view angle change, affine transformation and noise to a certain extent.

And 3, matching the detected feature points by using a K-D tree algorithm to obtain feature point pairs.

(3a) Selecting an image I₁The feature point in (1) is calculated, and the point and the image I are calculated₂Selecting a nearest neighbor Euclidean distance feature point and a next nearest neighbor Euclidean distance feature point from all feature points, and respectively recording the distances as d₁And d₂；

(3b) Calculating d₁And d₂And comparing the ratio with a given threshold of 0.4;

if the ratio is less than 0.4, the feature points are considered to be correctly matched, and the feature points of the two images are matched; otherwise, it is considered as an error match.

And 4, performing characteristic point pair elimination (mismatching) on the image with the marked characteristic points by adopting a RANSAC algorithm and solving a homography matrix H.

(4a) Taking the matching point pair obtained in the step (3) as a sample set, randomly extracting 4 matching point pairs in the sample set, ensuring that any 3 points in the same image are not collinear, and calculating the current homography matrix H according to the matching point pair_*；

(4b) According to the current homography matrix H_*And removing mismatching matching point pairs by a remapping error method to obtain correct matching point pairs;

(4c) and calculating the optimal homography matrix H by using a least square method according to all the correct matching point pairs.

And 5, transforming the two spliced images according to the homography matrix H, and respectively recording the two transformed images as J₁And J₂As shown in fig. 3(a) and 3 (b).

And 6, obtaining an overlap region omega and searching for an optimal suture line.

(6a) Taking image J₁And J₂Obtaining an overlap region omega image as shown in fig. 4;

(6b) calculating image J₁Boundary B with overlap region omega₁And image J₂Boundary B with overlap region omega₂And the boundary B₁And boundary B₂The two intersections are respectively marked as S and T;

(6c) traversing all pixel points in the overlapping area, and respectively calculating the cost between each pixel point p and the pixel points q in the four adjacent areas, wherein the cost function is calculated as follows:

(1) if pixel point p or q is located at boundary B₁∪B₂Then, the cost between the two points is assigned to infinity;

(2) if the pixel points p and q are positioned in the same image J₁Or J₂If so, the cost between the two points is assigned to be 0;

(3) if the pixel points p and q are located in different images, calculating the cost V (p, q) between the two points according to the formula (3):

V(p,q)＝V(p)+V(q) (3)

V(x)＝[ω·||x||₂+(1-ω)·||x||₁]⁴ (4)

wherein, the weight coefficient omega in the embodiment of the invention is 0.2, | | x | | purple sweet₂＝||J₁(x)-J₂(x)||₂Is L₂Norm, | x | luminance₁＝||J₁(x)-J₂(x)||₁Is L₁And (4) norm.

(6d) Starting from the starting point S, searching the link with the minimum cost according to the cost among the pixels calculated in the step (6c) until the ending point T stops, and then the curve formed by the series of points is the optimal suture line.

FIG. 5 is an example of searching for the best stitching line, each gray circle represents a pixel, and the numbers on the lines between the gray circles are the cost between pixels calculated according to step (6 c). And starting from the point S, comparing the cost between the point S and the four-adjacent-domain pixel, and selecting the link with the minimum cost value. The thick line segments in fig. 5 represent the least costly links, and the best suture is made up of all thick line segments.

And 7, fusing the pixels of the two transformed images along the two sides of the optimal suture line by using a Poisson fusion algorithm to form a spliced image. The stitched image is shown in fig. 6.

Claims (1)

1. An optimal suture line searching method for image stitching is characterized by comprising the following steps:

(1) reading two images to be spliced, and respectively marking as I₁And I₂；

(2) Image I by SIFT algorithm₁And I₂Extracting characteristic points;

(3) matching the detected characteristic points by using a K-D tree to obtain characteristic point pairs;

(4) performing feature point pair elimination (mismatching) on the image with the marked feature points by adopting an RANSAC algorithm and solving a homography matrix H;

(5) for image I according to homography matrix H₁And I₂Transforming, and recording the two transformed images as J₁And J₂；

(6) Obtaining a coincidence region omega and searching an optimal suture line;

(6a) taking image J₁And J₂Obtaining an omega image of a superposition area;

(6b) calculating image J₁Boundary B with overlap region omega₁And image J₂Boundary B with overlap region omega₂And the boundary B₁And boundary B₂The two intersections are respectively marked as S and T;

(6c) traversing all pixel points in the overlapping area, and respectively calculating the cost between each pixel point p and the pixel points q in the four adjacent areas, wherein the cost function is calculated as follows:

(1) if pixel point p or q is located at boundary B₁∪B₂Then, the cost between the two points is assigned to infinity;

(2) if the pixel points p and q are positioned in the same image J₁Or J₂If so, the cost between the two points is assigned to be 0;

(3) if the pixel points p and q are located in different images, the cost V (p, q) between the two points is calculated according to the formula (1):

V(p,q)＝V(p)+V(q) (1)

V(x)＝[ω·||x||₂+(1-ω)·||x||₁]⁴ (2)

wherein, omega is a weight coefficient, omega belongs to [0,1 ]]，||x||₂＝||J₁(x)-J₂(x)||₂Is L₂Norm, | x | luminance₁＝||J₁(x)-J₂(x)||₁Is L₁And (4) norm.

(6d) Starting from the starting point S, searching the link with the minimum cost according to the cost among the pixels calculated in the step (6c) until the ending point T stops, and then the curve formed by the series of points is the optimal suture line.

Images