CN118015237B - Multi-view image stitching method and system based on global similarity optimal seam - Google Patents
Multi-view image stitching method and system based on global similarity optimal seam Download PDFInfo
- Publication number
- CN118015237B CN118015237B CN202410417204.9A CN202410417204A CN118015237B CN 118015237 B CN118015237 B CN 118015237B CN 202410417204 A CN202410417204 A CN 202410417204A CN 118015237 B CN118015237 B CN 118015237B
- Authority
- CN
- China
- Prior art keywords
- image
- images
- global
- vehicle
- difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 9
- 230000009466 transformation Effects 0.000 claims description 25
- 239000011159 matrix material Substances 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 13
- 238000012544 monitoring process Methods 0.000 claims description 12
- 238000003708 edge detection Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 7
- 230000001131 transforming effect Effects 0.000 claims description 6
- 230000002146 bilateral effect Effects 0.000 claims description 4
- 238000010845 search algorithm Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 3
- 238000000844 transformation Methods 0.000 claims description 3
- 238000005286 illumination Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005206 flow analysis Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to a multi-view image stitching method and system based on global similarity optimal stitching, which are characterized in that two images of the same vehicle are firstly obtained and preprocessed, the preprocessed images are registered and aligned based on global similarity characteristics, and finally, the registered and aligned images are stitched and fused by finding optimal pixels from the images by using an optimal stitching algorithm with minimum global energy, so that a natural seamless high-quality complete image of the vehicle is obtained, image distortion caused by registration is reduced, the problems of inconsistent illumination, scale relation and the like are effectively processed, and the stitched images can be saved as much as possible.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a multi-view image stitching method and system based on a global similarity optimal seam.
Background
With the rapid development of traffic industry, the automobile has more and more possession, the traffic road condition is complex, and an intelligent traffic management system is comprehensively built for better monitoring and management. In monitoring systems and traffic management, it is critical to track and monitor the entire course of a vehicle. When the vehicle passes through the monitoring area, the complete vehicle information can be obtained by splicing the pictures of the two cameras, so that continuous observation of the vehicle motion is realized, and traffic flow analysis and optimization are further carried out.
Although the current urban road almost fully realizes camera coverage, complete image information cannot be acquired due to limited field of view of the cameras. In order to obtain complete information of the same vehicle, when the vehicle enters the picture of one camera from the picture of the other camera, an image stitching algorithm can be adopted to stitch the pictures of the two cameras, so that complete vehicle image information is obtained.
The image stitching technology has important supporting functions in the fields of traffic management, infrastructure maintenance and the like, and is one of the important means for solving the urban traffic problem. In order to solve the problems of limited single image visual angle and low manual inspection efficiency of the traffic monitoring system, the image stitching technology has obvious application value in traffic monitoring. But applying image stitching techniques in traffic monitoring and management still faces some challenges. For example, image registration has the problems of low timeliness and unstable spatial transformation models, and may lead to poor stitching effects. Meanwhile, for the situations of large parallax and fast moving vehicles and the like, dislocation and ghosting can occur in the splicing process, and then the splicing effect is affected. Therefore, a series of processing operations are required in the stitching process to mitigate this phenomenon and alleviate pixel distortion. In order to cope with the series of problems, researchers perform a lot of work, wherein the SIFT (SCALE INVARIANT Feature Transform) registration algorithm has the characteristic of scale invariance, has rotation, illumination and scale invariance, and greatly improves the robustness of feature points in feature extraction, but because different cameras are arranged at different heights, distances and angles, photographed pictures are not on the same projection plane, if images obtained by registration are directly spliced in a seamless manner, the problem of dislocation can be inevitably caused, and the visual consistency of actual pictures is damaged. To minimize the visible artifacts caused by such misalignment problems, the suture must be found for splicing. The fusion method of the optimal suture line can effectively relieve the problems of dislocation, ghosting and the like by minimizing the energy function and searching the optimal suture line.
In summary, in the application context of acquiring the complete image information of the vehicle, it is a concern to splice the vehicle images in the camera. In order to improve the accuracy and robustness of image stitching, researchers are struggling to explore and develop methods such as image preprocessing technology, image registration technology, image fusion algorithm and the like so as to provide more effective technical means for image stitching in a traffic management system.
Disclosure of Invention
In order to solve the problem of acquiring complete vehicle information by splicing pictures of two cameras, the invention provides a multi-view image splicing method and a system based on a global similarity optimal seam.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
The invention provides a multi-view image stitching method based on a global similarity optimal seam, which comprises the following steps:
s1, image acquisition: when a vehicle enters a camera monitoring area, a first camera shoots a vehicle photo, and when the vehicle enters a second camera, the same vehicle photo is shot immediately, so that two images of the same vehicle are obtained;
s2, image preprocessing: preprocessing the image obtained in the step S1 to obtain a preprocessed image;
S3, registering and aligning based on global similarity characteristics: carrying out global contour feature extraction on the preprocessed images, then carrying out space transformation, putting the two images into the same coordinate system, carrying out matching registration by utilizing global contour information, determining an overlapping region according to similarity information, and enabling overlapping parts of the two images to be aligned in space to obtain a registration alignment image;
s4, image stitching based on the optimal joint: and registering the aligned images, finding out the optimal pixels from the images by using an optimal joint algorithm with minimum global energy to perform stitching fusion, and obtaining a natural seamless high-quality complete image of the vehicle.
As a further technical scheme of the invention, the image preprocessing process in the step S2 is as follows: firstly, carrying out equalization processing on two vehicle images obtained in the step S1 by using a histogram, then removing noise from the equalized images by using bilateral filtering, and then carrying out deblurring processing on the images by using inverse filtering to obtain two preprocessed imagesAnd/>。
As a further technical scheme of the invention, the specific process of the step S3 is as follows:
s31, carrying out global contour feature extraction on the preprocessed image obtained in the S2,
Defining the ith reference point on the original contour of the image asThe reference points belong to an image contour space containing N points, and are connected according to the natural sequence of the vehicle contour to obtain a global contour feature sequence of the vehicle image;
then in a polar coordinate system to The rest N-1 points/>, which are the origin, are obtainedRelative polar coordinates of (2)Wherein i, j=1, 2,3, N; i+.j,/>Representing the reference point/>To the point/>Polar distance between,/>Representing the reference point/>To the point/>Polar angle between; transforming the relative coordinate data according to the natural sequence of the image contour points in the shape to form a sequence/>The method comprises the following steps: /(I)=/>; Wherein/>For the relative polar coordinates of the ith reference point and the jth point, the order of j is defined by 1,2,3, N transforms to i+1, …, N,1, …, i-1;
then the sequence is Dividing to obtain a plurality of mutually independent subsequences, such as subsequences [1, a ], [ a+1,2a ], and the like, wherein the formula is as follows: /(I)Where b is a subsequence sequence number, a is a positive integer,/>Is a constant,/>Obtaining the global contour feature/>, for the relative polar coordinates of the ith reference point and the jth point；
S32, using the global homography and global contour information to imageAnd image/>Registering and aligning, constructing space transformation between two images, and inputting the two images/>, processed by S2And/>And corresponding SURF feature matching points, image/>Feature matching points/>，/>Feature matching points/>Feature matching points/>，/>Image/>, respectivelyFeature matching point coordinates and image/>Feature matching point coordinates of (a); the linear transformation of homogeneous coordinates between two images is expressed as: /(I)Where x' is x,/>, in homogeneous coordinatesHomogeneous coordinates of/>，/>Homogeneous coordinates of/>,/>Is contour information,/>Defined as homography matrix,/>Row by/>,,/>Composition, i.e
，
Two images/>, which are row and column values of homography 3x3 matrixAnd/>The mapping between is expressed as:
，
，
Transforming two input images by using homography matrix H, placing them on the same reference plane to obtain registered aligned images ；
S33, the whole image is processedGlobal similarity transformation is performed, namely: /(I)Where S represents a global similarity transformation, μ and ρ are weighting coefficients,/>Is the i-th local homography,/>For updated local transformations, the final registration alignment image/>, is obtained after transformation。
As a further technical scheme of the invention, the specific process of the step S4 is as follows:
S41, extracting registration alignment images obtained in S3 Respectively marked as omega and omega ', and constructing a similarity difference matrix E reflecting the overlapped areas omega and omega';
S42, sorting all the difference values of the E, and rapidly calculating a minimum threshold value E by using a binary search algorithm; under the condition of the minimum threshold e, the eight-communication area where the starting point and the end point are located is represented as R, the eight-communication area is expanded along eight adjacent directions from the starting point, the pixel difference value in the R is updated to be the sum of the minimum difference from the starting point, and the updated pixel is regarded as a new expansion point until the pixel is expanded to the end point; and (3) returning to the starting point from the end point along the pixel path with the minimum difference sum value to obtain an optimal joint, and realizing seamless splicing and fusion of the images through the optimal joint to obtain a natural high-quality complete image of the vehicle.
As a further technical scheme of the invention, the specific process of constructing the similarity difference matrix E in the step S41 is as follows:
Color differences are calculated using color differences in LAB color space ，，/>，/>，Wherein/>Is the color difference,/>、/>、/>RGB channel value of Ω,/>、/>RGB channel values are Ω';
then the high frequency part of the overlapping area is used for constructing structural difference, and the parameters of omega and omega' are as follows Gaussian filtering of (5) to give/>And/>And calculates/>, by gaussian differential edge detectionAnd/>Structural differences between, namely:，/>-/> Wherein/> Is a structural difference,/>And/>The difference parameter, θ, is a positive integer constant,/>Is Gaussian differential edge detection formula,/>For/>Gaussian differential edge detection,/>For/>Is a Gaussian differential edge detection of (1);
Then linear information of the overlapping regions Ω and Ω 'is acquired using a Line Segment Detector (LSD), and then the linear information of Ω and Ω' is subtracted to obtain a linear difference, which is expressed as ；
Finally, adding the three differences to obtain a difference matrix。
In a second aspect, the present invention provides a multi-view image stitching system based on a global similarity best seam, comprising:
the image acquisition module is used for acquiring vehicle images shot by the two cameras when the vehicle enters the camera monitoring area;
The image preprocessing module is used for preprocessing the vehicle images shot by the two cameras;
The registration alignment module is used for carrying out feature matching and space transformation alignment on the preprocessed image to obtain a registration alignment image;
and the image stitching module based on the optimal seam is used for searching the optimal seam of the registered alignment images, and stitching and fusing are carried out by using the minimum global energy to obtain a natural seamless high-quality complete image of the vehicle.
In a third aspect, the invention provides an electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the method of the first aspect.
In a fourth aspect, the present invention provides a computer readable storage medium storing computer instructions which, when executed by a processor, perform the method of the first aspect.
Compared with the prior art, the invention has the beneficial effects that:
The invention provides a multi-view image stitching method based on an optimal global similarity seam, which aims to solve the problem that a vehicle enters one camera picture from another camera picture and stitch two camera pictures, so that a complete vehicle image is obtained, and the innovation of the method is mainly characterized in three aspects: the method has the advantages that the global vehicle contour feature extraction and matching are carried out on the images, the registration is carried out based on the global similarity feature, the image distortion caused by registration is reduced, the picture stitching fusion is carried out based on the global minimum energy optimal joint algorithm, and the method has the following advantages:
(1) By extracting the vehicle contour features from the images and matching the extracted contour information, the problems of inconsistent illumination, scale relation and the like can be effectively solved, and the obvious vehicle contour features with uniform colors can be obtained.
(2) And determining an overlapping area of the images based on a global similarity feature registration alignment method, registering and aligning the images, and reducing distortion caused by image alignment by adding similarity constraint so that the images after subsequent splicing are more natural.
(3) The image stitching fusion is carried out based on the global minimum energy optimal stitching algorithm, the searching range of the stitching can be limited by defining accurate difference cost through the global minimum energy, the effectiveness of the stitching is indirectly improved, the optimal stitching is found, the original image information is saved as much as possible, and the more natural and seamless high-quality complete image of the vehicle is obtained.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the present disclosure and do not constitute a limitation on the invention.
Fig. 1 is a schematic flow chart of a multi-view image stitching method based on a global similarity optimal seam.
Fig. 2 is a block diagram of a multi-view image stitching system based on a global similarity optimal seam according to the present invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms also are intended to include the plural forms, and furthermore, it is to be understood that the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, processes, methods, systems, products or devices that comprise a series of steps or units, are not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or inherent to such processes, methods, products or devices.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
Example 1:
As shown in fig. 1, the embodiment provides a multi-view image stitching method based on a global similarity optimal seam, which includes the following steps:
s1, image acquisition:
when a vehicle enters a camera monitoring area, a first camera shoots a vehicle photo, and when the vehicle enters a second camera, the same vehicle photo is shot immediately, so that two images of the same vehicle are obtained.
S2, image preprocessing:
Firstly, using a histogram to equalize two vehicle images obtained in the step S1, processing the problem of uneven brightness of the images, enabling bilateral filtering to achieve the effect of noise removal of the original images, well keeping edge information, then using bilateral filtering to remove noise of the equalized images, keeping the whole details of the original images while removing the noise, using inverse filtering to deblur the images after the image is de-noised, and effectively removing noise and blurring while preprocessing color and detail information of the restored images to obtain two preprocessed images And/>And the later image registration alignment is easy.
S3, registering and aligning based on global similarity characteristics:
s31, carrying out global contour feature extraction on the preprocessed image obtained in the S2,
Defining the ith reference point on the original contour of the image asThe reference points belong to an image contour space containing N points, and the points are connected according to the natural sequence of the vehicle contour so as to acquire a global contour feature sequence of the vehicle image;
then in a polar coordinate system to The rest N-1 points/>, which are the origin, are obtainedRelative polar coordinates of (2)Wherein i, j=1, 2,3, N; i+.j,/>Representing the reference point/>To the point/>Polar distance between,/>Representing the reference point/>To the point/>Polar angle between; transforming the relative coordinate data according to the natural sequence of the image contour points in the shape to form a sequence/>The method comprises the following steps: /(I)=/>; Wherein/>For the relative polar coordinates of the ith reference point and the jth point, the order of j is defined by 1,2,3, N transforms to i+1, …, N,1, …, i-1; by/>Smoothing is carried out to solve the contradiction in the process, so that the robustness is improved;
Then in the case of a trade-off between accuracy and sensitivity, Comprising N-1 sequences, which are split to obtain several subsequences that are independent of each other, namely: /(I)Where b is a subsequence sequence number, a is a positive integer,/>Is a constant,/>Obtaining the global contour feature/>, for the relative polar coordinates of the ith reference point and the jth point。
S32, using the global homography and global contour information to imageAnd image/>Registering and aligning, constructing space transformation between two images, and inputting the two images/>, processed by S2And/>And corresponding SURF feature matching points, image/>Feature matching points/>，/>Feature matching points/>Feature matching points/>，/>Image/>, respectivelyFeature matching point coordinates and image/>Feature matching point coordinates of (a); the linear transformation of homogeneous coordinates between two images is expressed as: /(I)Where x' is x,/>, in homogeneous coordinatesHomogeneous coordinates of/>，/>Homogeneous coordinates of/>,/>Is contour information,/>Defined as homography matrix,/>Row by/>,,/>Composition, i.e
，
Two images/>, which are row and column values of homography 3x3 matrixAnd/>The mapping between is expressed as:
，
，
Transforming two input images by using homography matrix H, placing them on the same reference plane to obtain registered aligned images ；
S33, in order to adjust the distortion of the registration image, pairUsing global similarity transformation to mitigate perspective distortion in aligned images, we will/>, the whole imageGlobal similarity transformation is performed, namely: /(I)Where S represents a global similarity transformation, μ and ρ are weighting coefficients,/>Is the i-th local homography,/>For updated local transformations, the final registration alignment image/>, is obtained after transformation。
S4, image stitching based on the optimal joint:
S41, extracting registration alignment images obtained in S3 Is marked as omega and omega 'respectively, and a similarity difference matrix E reflecting the overlapped areas omega and omega' is constructed, specifically:
To better accommodate human eye perception of color, color differences in LAB color space are used to calculate color differences ，/>，/>，，/>Wherein/>Is the color difference,/>、/>、/>RGB channel value of Ω,/>、/>RGB channel values are Ω';
constructing structural differences using high frequency portions of overlapping regions to reduce structural differences of low frequency portions using parameters of Ω and Ω Gaussian filtering of (5) to give/>And/>And gaussian differential edge detection to calculate/>And/>Structural differences between, namely: /(I)，/>-/>Wherein/>Is a structural difference,/>And/>The difference parameter, θ, is a positive integer constant,/>Is Gaussian differential edge detection formula,/>For/>Gaussian differential edge detection,/>For/>Is a Gaussian differential edge detection of (1);
highlighting differences between structural objects by detecting line segment information of contour features, acquiring linear information of overlapping areas Ω and Ω 'using a Line Segment Detector (LSD), and then subtracting the linear information of Ω and Ω' to obtain a linear difference, expressed as ；
Adding the three differences to obtain a difference matrixE is a difference matrix, which is a two-dimensional matrix of values.
The start and end points of the seam are typically located at the juncture of the two registered alignment images S42. If two pixels can be connected in an uninterrupted line on a two-dimensional matrix, they must be located in the same eight-communication area, specifically:
Setting a threshold value E to ensure that the pixel difference value on the joint is smaller than E, minimizing E on the premise that the starting point and the end point are located in the same eight-communication area, sequencing all the difference values of E, and rapidly calculating the minimum threshold value E by utilizing a binary search algorithm;
under the condition of a minimum threshold e, the eight-communication area where the starting point and the ending point are located is represented as R, the search range of the joint is limited in R, and the search area R is defined by constraint of a numerical value e, wherein each pixel has a specific difference value;
Searching an optimal seam by a stitching search algorithm based on the minimum global difference, starting from a starting point, expanding along eight adjacent directions, updating a pixel difference value in R to be the sum of the minimum difference from the starting point, and regarding the updated pixel as a new expansion point until the pixel is expanded to an end point; and (3) returning to the starting point from the end point along the pixel path with the minimum difference sum value to obtain an optimal joint, and realizing seamless splicing and fusion of the images through the optimal joint to obtain a natural high-quality complete image of the vehicle.
According to the embodiment, based on global similarity contour feature extraction, an image pair Ji Peizhun is carried out by utilizing a contour feature-based registration algorithm and finding an image overlapping region, and due to reasons such as algorithm error accumulation, color difference caused by illumination conditions, color difference caused by geometric and luminosity deviation errors, discontinuous spliced images and the like, visible seams and color brightness difference of the whole image are easily generated near the boundary between two images, image fusion processing is carried out by searching for optimal seam splicing, the differences are corrected, seams are eliminated, the influence of dislocation and ghost caused by a registration alignment process is eliminated, a high-quality spliced image is obtained, the seam and brightness difference are eliminated on the premise that original image vehicle information is not lost through the optimal seam splicing algorithm, and a seamless natural high-quality complete vehicle image is obtained.
In the embodiment, simulation experiments are performed on the method and the existing algorithm, the experiments adopt PSNR and SSIM indexes for measurement, the used data set contains about 50 groups of photos of the same vehicle, the PSNR and SSIM of each method are compared with average values, the result is shown in table 1, and compared with the prior art, the method in the embodiment has good performance in image splicing, eliminates ghost in an overlapping area, reduces image distortion and splices images more naturally.
Table 1: comparison of the accuracy of this embodiment with other algorithms
。
Example 2:
As shown in fig. 2, the present embodiment provides an image stitching system based on a global similarity optimal stitching method, including:
the image acquisition module is used for acquiring vehicle images shot by the two cameras when the vehicle enters the camera monitoring area;
The image preprocessing module is used for preprocessing the vehicle images shot by the two cameras;
The registration alignment module is used for carrying out feature matching and space transformation alignment on the preprocessed image to obtain a registration alignment image;
and the image stitching module based on the optimal seam is used for searching the optimal seam of the registered alignment images, and stitching and fusing are carried out by using the minimum global energy to obtain a natural seamless high-quality complete image of the vehicle.
Here, it should be noted that the above-mentioned modules correspond to steps S1 to S4 in embodiment 1, and the above-mentioned modules are the same as examples and application scenarios implemented by the corresponding steps, but are not limited to those disclosed in embodiment 1. It should be noted that the modules described above may be implemented as part of a system in a computer system, such as a set of computer-executable instructions.
In further embodiments, there is also provided:
An electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the method described in embodiment 1. For brevity, the description is omitted here.
It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include read only memory and random access memory and provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store information of the device type.
A computer readable storage medium storing computer instructions which, when executed by a processor, perform the method described in embodiment 1.
The method in embodiment 1 may be directly embodied as a hardware processor executing or executed with a combination of hardware and software modules in the processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method. To avoid repetition, a detailed description is not provided herein.
Those of ordinary skill in the art will appreciate that the elements of the various examples described in connection with the present embodiments, i.e., the algorithm steps, can be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it should be understood that it is not intended to limit the scope of the invention, but rather, it is intended to cover all modifications or variations within the scope of the invention as defined by the claims of the present invention.
Claims (6)
1. The multi-view image stitching method based on the global similarity optimal seam is characterized by comprising the following steps of:
s1, image acquisition: when a vehicle enters a camera monitoring area, a first camera shoots a vehicle photo, and when the vehicle enters a second camera, the same vehicle photo is shot immediately, so that two images of the same vehicle are obtained;
S2, image preprocessing: preprocessing the image acquired in the step S1 to obtain two preprocessed images And/>；
S3, registering and aligning based on global similarity characteristics: carrying out global contour feature extraction on the preprocessed images, then carrying out space transformation, putting the two images into the same coordinate system, carrying out matching registration by utilizing global contour information, determining an overlapping region according to similarity information, and enabling overlapping parts of the two images to be aligned in space to obtain a registration alignment image; the specific process is as follows:
S31, carrying out global contour feature extraction on the registration alignment image obtained in the S2,
Defining the ith reference point on the original contour of the image asThe reference points belong to an image contour space containing N points, and are connected according to the natural sequence of the vehicle contour to obtain a global contour feature sequence of the vehicle image;
then in a polar coordinate system to The relative polar coordinates of the rest N-1 points are obtained as the origin, wherein the formula isWherein i, j=1, 2, 3..n; i.noteq.j, transforming the relative coordinate data according to the natural order of the image contour points in the shape to form a sequence/>The method comprises the following steps: /(I)=/>; Wherein/>The relative polar coordinates of the ith reference point and the jth point;
then the sequence is Dividing to obtain a plurality of mutually independent subsequences, such as subsequences [1, a ], [ a+1,2a ], and the like, wherein the formula is as follows: /(I)Where b is a subsequence sequence number, a is a positive integer,/>Is a constant,/>The relative polar coordinates of the ith reference point and the jth point; obtaining the global outline feature/>；
S32, using the global homography and global contour information to imageAnd image/>Registering and aligning, constructing space transformation between two images, and inputting the two images/>, processed by S2And/>And corresponding SURF feature matching points, image/>Feature matching points/>，/>Feature matching points/>Feature matching points/>，/>Respectively, imagesFeature matching point coordinates and image/>Feature matching point coordinates of (a); the linear transformation of homogeneous coordinates between two images is expressed as:/>Where x' is x,/>, in homogeneous coordinatesHomogeneous coordinates of/>，/>Homogeneous coordinates of/>,/>Is contour information,/>Defined as homography matrix,/>Row by/>,,/>Composition, i.e
，
Two images/>, which are row and column values of homography 3x3 matrixAnd/>The mapping between is expressed as:
，
，
Transforming two input images by using homography matrix H, placing them on the same reference plane to obtain registered aligned images ；
S33, the whole image is processedGlobal similarity transformation is performed, namely: /(I)Where S represents a global similarity transformation, μ and ρ are weighting coefficients,/>Is the i-th local homography,/>For updated local transformations, the final registration alignment image/>, is obtained after transformation；
S4, image stitching based on the optimal joint: registering the aligned images, finding out the best pixels from the images by using the best joint algorithm with the minimum global energy to carry out splicing and fusion, so as to obtain a natural seamless high-quality complete image of the vehicle; the specific process is as follows:
S41, extracting registration alignment images obtained in S3 Respectively marked as omega and omega ', and constructing a similarity difference matrix E reflecting the overlapped areas omega and omega';
S42, sorting all the difference values of the E, and rapidly calculating a minimum threshold value E by using a binary search algorithm; under the condition of the minimum threshold e, the eight-communication area where the starting point and the end point are located is represented as R, the eight-communication area is expanded along eight adjacent directions from the starting point, the pixel difference value in the R is updated to be the sum of the minimum difference from the starting point, and the updated pixel is regarded as a new expansion point until the pixel is expanded to the end point; and (3) returning to the starting point from the end point along the pixel path with the minimum difference sum value to obtain an optimal joint, and realizing seamless splicing and fusion of the images through the optimal joint to obtain a natural high-quality complete image of the vehicle.
2. The multi-view image stitching method based on the global similarity best seam according to claim 1, wherein the image preprocessing in step S2 is as follows: firstly, carrying out equalization processing on two vehicle images obtained in the step S1 by using a histogram, then removing noise from the equalized images by using bilateral filtering, and then carrying out deblurring processing on the images by using inverse filtering to obtain two preprocessed imagesAnd/>。
3. The multi-view image stitching method based on the global similarity optimal seam according to claim 2, wherein the specific process of constructing the similarity difference matrix E in step S41 is as follows:
Color differences are calculated using color differences in LAB color space ,,/>，/>，Wherein/>Is the color difference,/>、/>、/>RGB channel value of Ω,/>、/>RGB channel values are Ω';
then the high frequency part of the overlapping area is used for constructing structural difference, and the parameters of omega and omega' are as follows Is Gaussian filtered to obtainAnd/>And calculates/>, by gaussian differential edge detectionAnd/>Structural differences between, namely:，/>-/> Wherein/> Is a structural difference,/>And/>A difference parameter, θ, is a positive integer constant;
Then linear information of the overlapping regions Ω and Ω 'is acquired using a Line Segment Detector (LSD), and then the linear information of Ω and Ω' is subtracted to obtain a linear difference, which is expressed as ；
Finally, adding the three differences to obtain a difference matrix。
4. A multi-view image stitching system based on global similarity best seams, characterized in that it is capable of performing the method according to any one of claims 1-3, comprising:
the image acquisition module is used for acquiring vehicle images shot by the two cameras when the vehicle enters the camera monitoring area;
The image preprocessing module is used for preprocessing the vehicle images shot by the two cameras;
The registration alignment module is used for carrying out feature matching and space transformation alignment on the preprocessed image to obtain a registration alignment image;
and the image stitching module based on the optimal seam is used for searching the optimal seam of the registered alignment images, and stitching and fusing are carried out by using the minimum global energy to obtain a natural seamless high-quality complete image of the vehicle.
5. An electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the method of any one of claims 1-3.
6. A computer readable storage medium storing computer instructions which, when executed by a processor, perform the method of any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410417204.9A CN118015237B (en) | 2024-04-09 | 2024-04-09 | Multi-view image stitching method and system based on global similarity optimal seam |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410417204.9A CN118015237B (en) | 2024-04-09 | 2024-04-09 | Multi-view image stitching method and system based on global similarity optimal seam |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118015237A CN118015237A (en) | 2024-05-10 |
| CN118015237B true CN118015237B (en) | 2024-06-21 |
Family
ID=90944694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410417204.9A Active CN118015237B (en) | 2024-04-09 | 2024-04-09 | Multi-view image stitching method and system based on global similarity optimal seam |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118015237B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112991176A (en) * | 2021-03-19 | 2021-06-18 | 南京工程学院 | Panoramic image splicing method based on optimal suture line |
| WO2021120407A1 (en) * | 2019-12-17 | 2021-06-24 | 大连理工大学 | Parallax image stitching and visualization method based on multiple pairs of binocular cameras |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6377306B1 (en) * | 1998-09-23 | 2002-04-23 | Honeywell International Inc. | Method and apparatus for providing a seamless tiled display |
| CN103903237B (en) * | 2014-03-21 | 2017-03-29 | 上海大学 | Sonar image sequence assembly method is swept before one kind |
| CN104463786B (en) * | 2014-12-03 | 2017-06-16 | 中国科学院自动化研究所 | A kind of mobile robot image split-joint method and device |
| CN108416732A (en) * | 2018-02-02 | 2018-08-17 | 重庆邮电大学 | A kind of Panorama Mosaic method based on image registration and multi-resolution Fusion |
| CN110390640B (en) * | 2019-07-29 | 2023-04-28 | 齐鲁工业大学 | Template-based Poisson fusion image splicing method, system, equipment and medium |
| CN111047510B (en) * | 2019-12-17 | 2023-02-14 | 大连理工大学 | Large-field-angle image real-time splicing method based on calibration |
| CN112950468A (en) * | 2021-02-01 | 2021-06-11 | 咪咕文化科技有限公司 | Image splicing method, electronic device and readable storage medium |
| US11587250B2 (en) * | 2021-06-21 | 2023-02-21 | University Of Electronic Science And Technology Of China | Method for quantitatively identifying the defects of large-size composite material based on infrared image sequence |
| CN114565833A (en) * | 2021-12-27 | 2022-05-31 | 理大产学研基地(深圳)有限公司 | Full convolution network building extraction method based on contour guidance and structure attention |
| CN116579920A (en) * | 2023-04-12 | 2023-08-11 | 哈尔滨工程大学 | Image stitching method and system based on heterogeneous multimode panoramic stereoscopic imaging system |
| CN116664643A (en) * | 2023-06-28 | 2023-08-29 | 哈尔滨市科佳通用机电股份有限公司 | Railway train image registration method and equipment based on SuperPoint algorithm |
-
2024
- 2024-04-09 CN CN202410417204.9A patent/CN118015237B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021120407A1 (en) * | 2019-12-17 | 2021-06-24 | 大连理工大学 | Parallax image stitching and visualization method based on multiple pairs of binocular cameras |
| CN112991176A (en) * | 2021-03-19 | 2021-06-18 | 南京工程学院 | Panoramic image splicing method based on optimal suture line |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118015237A (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110390640B (en) | Template-based Poisson fusion image splicing method, system, equipment and medium | |
| CN110223330B (en) | Registration method and system for visible light and infrared images | |
| EP1901228B1 (en) | Apparatus, method and program for image matching | |
| US6798923B1 (en) | Apparatus and method for providing panoramic images | |
| CN106447601B (en) | Unmanned aerial vehicle remote sensing image splicing method based on projection-similarity transformation | |
| CN104392416A (en) | Video stitching method for sports scene | |
| CN103902953B (en) | A kind of screen detecting system and method | |
| CN109544635B (en) | Camera automatic calibration method based on enumeration heuristic | |
| CN111553939A (en) | Image registration algorithm of multi-view camera | |
| CN113793266A (en) | Multi-view machine vision image splicing method, system and storage medium | |
| Gao et al. | Accurate and efficient ground-to-aerial model alignment | |
| CN116152068A (en) | Splicing method for solar panel images | |
| CN112365518A (en) | Image splicing method based on optimal suture line self-selection area gradual-in and gradual-out algorithm | |
| Attard et al. | Image mosaicing of tunnel wall images using high level features | |
| Wu et al. | Adaptive image registration via hierarchical voronoi subdivision | |
| CN113506216B (en) | Rapid suture line optimizing method for panoramic image stitching | |
| CN114897676A (en) | Unmanned aerial vehicle remote sensing multispectral image splicing method, device and medium | |
| CN114897684A (en) | Vehicle image splicing method and device, computer equipment and storage medium | |
| CN107895344A (en) | Video-splicing device and method | |
| Saalfeld | Computational methods for stitching, alignment, and artifact correction of serial section data | |
| CN106651756B (en) | Image registration method based on SIFT and verification mechanism | |
| CN118015237B (en) | Multi-view image stitching method and system based on global similarity optimal seam | |
| CN117635421A (en) | Image stitching and fusion method and device | |
| CN117611525A (en) | Visual detection method and system for abrasion of pantograph slide plate | |
| CN114463170B (en) | Large scene image stitching method for AGV application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |