WO2015165222A1 - Method and device for acquiring panoramic image - Google Patents
Method and device for acquiring panoramic image Download PDFInfo
- Publication number
- WO2015165222A1 WO2015165222A1 PCT/CN2014/088726 CN2014088726W WO2015165222A1 WO 2015165222 A1 WO2015165222 A1 WO 2015165222A1 CN 2014088726 W CN2014088726 W CN 2014088726W WO 2015165222 A1 WO2015165222 A1 WO 2015165222A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- images
- sub
- overlapping
- foreground
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 4
- 238000009499 grossing Methods 0.000 claims description 68
- 230000011218 segmentation Effects 0.000 claims description 50
- 239000011159 matrix material Substances 0.000 claims description 23
- 230000009466 transformation Effects 0.000 claims description 19
- 230000009977 dual effect Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 14
- 102000016917 Complement C1 Human genes 0.000 claims description 6
- 108010028774 Complement C1 Proteins 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 20
- 238000005516 engineering process Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013519 translation Methods 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
Definitions
- Embodiments of the present invention relate to image processing technologies, and in particular, to a method and an apparatus for acquiring a panoramic image.
- ultra-high definition video (image) acquisition devices are very expensive, so it is a natural alternative to form a camera array with relatively inexpensive lower resolution cameras.
- the existing panoramic stitching technique superimposes (or approximately coincides) the optical centers of a plurality of cameras, thereby obtaining a panoramic image of a large field of view.
- the invention provides a method and a device for acquiring a panoramic image, so that a high-resolution panoramic image with a large field of view can be obtained only by obtaining the internal and external parameters of the camera without calibration.
- an embodiment of the present invention provides a method for acquiring a panoramic image, including: acquiring at least two first images; respectively extracting a background image and a foreground image of the first image; and determining at least two of the first images.
- first overlapping image and a first non-overlapping image formed by the background image and a second overlapping image and a second non-overlapping image formed by determining at least two foreground images of the first image; performing the first overlapping image Dividing to obtain a plurality of first divided sub-images, and dividing the second overlapping image to obtain a plurality of second divided sub-images; for each of the first divided sub-images, at least two background images of the first image Select corresponding a first splicing sub-image, selecting, for each of the second divided sub-images, a corresponding second splicing sub-image in at least two foreground images of the first image; corresponding to the plurality of the first divided sub-images
- the first spliced sub-image is spliced to obtain a target background image
- the second spliced sub-image corresponding to the plurality of the second divided sub-images is spliced to obtain a target foreground image; and the target background image, the target
- the foreground template of the first image is determined according to the depth value of the first image; the foreground template of the first image is 0, 1 Forming a matrix, at the same pixel point, extracting pixel points of the first image corresponding to the foreground template 1 to form a foreground image of the first image; and extracting one of the foreground templates at the same pixel point Corresponding pixel points of the first image constitute a background image of the first image.
- the determining the first overlap of the at least two background images formed by the first image And the image and the first non-overlapping image, and determining the second overlapping image and the second non-overlapping image of the foreground image formed by the at least two of the first images specifically comprising: converting each of the background images into a background virtual image, each of the foreground images being converted into a foreground virtual image by a homography transformation; calculating the first overlapping image and the first non-overlapping image of each of the background images according to the background virtual image, according to The foreground virtual image calculates the second overlapping image and the second non-overlapping image of each of the foreground images.
- the superimposing the image to obtain the plurality of first divided sub-images comprises: calculating a first difference value of the first overlapping image on each color channel, and performing Laplacian filtering transformation and smoothing on the first difference Processing, obtaining a first Morse function value; determining the first divided sub-image on the first overlapping image according to the first Morse function value; and dividing the second overlapping image to obtain a plurality of second
- the dividing the sub-image includes: calculating a second difference image of the second overlapping image on each color channel, performing Laplacian filtering transformation and smoothing processing on the second difference image to obtain a second mole Function value;
- the second Morse function value determines the second divided sub-image on the second overlay image.
- the determining, by the first Morse function value, the first on the first overlapping image The method of determining the first partial minimum point, the first local maximum point, and the first overlapping point on the first overlapping image according to the first Morse function value of the pixel point on the first overlapping image a saddle point; determining a first segmentation sub-image according to each of the first local minimum point, a first local maximum point, and two of the first saddle points; the determining according to the second Morse function value
- the second divided sub-image on the second overlapping image specifically includes: determining a second local minimum on the second overlapping image according to a second Morse function value of a pixel point on the second overlapping image a value point, a second local maximum point and a second saddle point; a second segmentation sub-image is determined according to each of the second local minimum points, a second local maximum point, and two of the second saddle points.
- the method further includes: determining a first maximum point of each of the first divided sub-images, and determining, by using a region algorithm, a first duality corresponding to the first overlapping image according to the first maximum point Dividing the sub-image; after determining the second divided sub-image on the second overlapping image according to the second Morse function value, further comprising: determining a second maximum of each of the second divided sub-images a value point, using a region algorithm, determining a second dual segmentation sub-image corresponding to the second overlapping image according to the second maximum point.
- the background of each of the first divided sub-images in at least two of the first images Selecting a corresponding first spliced sub-image in the image, the method comprising: calculating a smoothing overhead according to an average value of the maximum color difference of the connected first dual-divided sub-images, according to the image center of the first overlapping image to the first Calculating a data overhead for the distance of the dual-divided sub-image; selecting the corresponding first splice sub-image for each of the first dual-divided sub-images according to the smoothing overhead and the data overhead;
- the second divided sub-image selects the corresponding second splice sub-image in the foreground image of the at least two of the first images, and specifically includes: calculating light according to an average value of maximum color differences of the connected second dual-divided sub-images a sliding overhead, calculating a data overhead according to a distance from an image center of
- the performing the first dual-divided sub-image according to the smoothing overhead and the data overhead Selecting the corresponding first spliced sub-image includes: constructing a first map according to the first dual-divided sub-image; determining a weight of each side of the first graph according to the smoothing overhead and the data overhead; And the first dual-divided sub-image is classified according to the weight of each edge of the first figure by using a graph cutting method, and the first stitching corresponding to each of the first dual-divided sub-images is selected according to the classification result.
- the selecting the corresponding second splice sub-image for each of the second dual-divided sub-images according to the smoothing overhead and the data overhead comprising: constructing according to the second dual-divided sub-image structure a second graph; determining a weight of each edge of the second graph according to the smoothing overhead and the data overhead; and using the graph cutting method to the second dual segment according to a weight of each edge of the second graph Figure Classified according to the classification result of each of said second dual segmented sub-images corresponding to the selected second sub-image mosaic.
- an embodiment of the present invention provides a device for acquiring a panoramic image, including: an acquiring module, configured to acquire at least two first images; and an extracting module, configured to extract a background image and a foreground image of the first image; a determining module, configured to determine a first overlapping image and a first non-overlapping image of the background image of the at least two first image first images, and to determine a foreground image of the at least two first image first images a second overlapping image and a second non-overlapping image; a segmentation module, configured to segment the first overlapping image to obtain a plurality of first divided sub-images, and divide the second overlapping image to obtain a plurality of second divided sub-images a selection module, configured to select, for each of the first divided sub-images, a corresponding first splice sub-image in a background image of at least two of the first images, and at least two for each of the second divided sub-images Corresponding second splicing sub-images
- the foreground template of the first image is determined according to the depth value of the first image; the first foreground template is composed of 0, 1. a matrix, at a same pixel, extracting pixel points of the first image corresponding to the foreground template 1 to form a foreground image of the first image; and extracting a corresponding one of the foreground templates at the same pixel point
- the pixel of the first image constitutes a background image of the first image.
- the determining module is specifically configured to: Converting the background image into a first background virtual image, converting each of the first foreground images into a first foreground virtual image by homography conversion; calculating the respective of the first background images according to the first background virtual image a first overlapping image and the first non-overlapping image, and calculating the second overlapping image and the second non-overlapping image of each of the first foreground images according to the first foreground virtual image.
- the segmentation module is specifically used to Calculating a first difference value of the first overlapping image on each color channel, performing a Laplace filter transformation and smoothing processing on the first difference value to obtain a first Morse function value; a Morse function value determining the first divided sub-image on the first overlapping image; calculating a second difference image of the second overlapping image on each color channel, sequentially for the second difference image Performing a Laplacian transform transform and smoothing to obtain a second Morse function value; and determining the second divided sub-image on the second overlapping image according to the second Morse function value.
- the segmentation module is further configured to: determine, according to a first mole of pixels on the first overlapping image The sigma function value determines a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image; according to each of the first local minimum point, a first local maximum point, and Determining, by the two first saddle points, a first divided sub-image; determining a second local minimum point on the second overlapping image according to a second Morse function value of the pixel on the second overlapping image, Two local maximum points and a second saddle point; according to each The second partial minimum point, a second local maximum point, and the two second saddle points determine a second segmentation sub-image.
- the segmentation module is further configured to: determine a first maximum value of each of the first segmentation sub-images Point, using a region algorithm, determining a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point; determining a second maximum point of each of the second divided sub-images, using a region algorithm And determining, according to the second maximum point, a second dual-divided sub-image corresponding to the second overlapping image.
- the selecting module is specifically configured to: determine, according to a maximum color difference of the connected first dual-divided sub-images Calculating a smoothing overhead, calculating a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image; and each of the first duals according to the smoothing overhead and the data overhead Dividing the sub-image to select the corresponding first splice sub-image; calculating a smoothing overhead according to the average of the maximum color differences of the connected second dual-divided sub-images, according to the image center of the second overlapping image to the Calculating a data overhead of the distance of the two dual-divided sub-images; selecting the corresponding second splice sub-image for each of the second dual-divided sub-images according to the smoothing overhead and the data overhead.
- the selecting module is further configured to: construct a first image according to the first dual-divided sub-image,
- the first map includes: a plurality of the first dual-divided sub-images; determining a weight of each side of the first graph according to the smoothing overhead and the data overhead; and each edge according to the first graph
- the weighting is performed by using a graph cutting method to classify the first dual-divided sub-image, and selecting the corresponding first splice sub-image for each of the first dual-divided sub-images according to the classification result;
- selecting, by the data overhead, the second splice sub-image corresponding to each of the second dual-divided sub-images comprising: constructing a second map according to the second dual-divided sub-image, where the second map includes: a plurality of the second dual-divided sub-images; determining a weight of each side of the second graph according to
- the method and device for acquiring a panoramic image divides a first overlapping image of a background image of a first image captured by at least two cameras and a second overlapping image of a foreground image, for each first segmentation
- the sub-image selects a corresponding first splice sub-image in the background image of the at least two first images, and selects a corresponding second splice sub-image in the foreground image of the at least two first images for each second divided sub-image,
- the first spliced sub-image is spliced to obtain a target background image
- the second spliced sub-image is spliced to obtain a target foreground image
- the target background image, the target foreground image, the first non-overlapping image and the second non-overlapping image are synthesized, thereby Get a high resolution target panoramic image with a larger field of view.
- FIG. 1 is a flowchart of a method for acquiring a panoramic image according to an embodiment of the present invention
- FIG. 2 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention.
- 3A is a schematic diagram of a first divided sub-image according to an embodiment of the present invention.
- FIG. 3B is a schematic diagram of a first segmentation sub-image according to another embodiment of the present invention.
- FIG. 4A is a schematic diagram of a target panoramic image according to an embodiment of the present invention.
- FIG. 4B is a schematic diagram of a target panoramic image according to another embodiment of the present invention.
- FIG. 5 is a flowchart of a method for acquiring a panoramic image according to still another embodiment of the present invention.
- FIG. 6A is a schematic diagram of cutting according to still another embodiment of the present invention.
- 6B is a schematic diagram showing a classification result of a graph cut according to still another embodiment of the present invention.
- 6C is a schematic diagram of a target panoramic image according to another embodiment of the present invention.
- FIG. 7 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention.
- FIG. 8 is a schematic diagram of a camera array according to still another embodiment of the present invention.
- FIG. 9 is a schematic diagram of a background image mosaic according to another embodiment of the present invention.
- FIG. 10 is a schematic structural diagram of a device for acquiring a panoramic image according to an embodiment of the present invention.
- FIG. 1 is a flowchart of a method for acquiring a panoramic image according to an embodiment of the present invention.
- the method is applicable to the field of image processing, and the execution body of the method may be a device for acquiring a panoramic image, where the device may be a computer or the like.
- the specific steps of the device and the method for acquiring the panoramic image include:
- S101 Acquire at least two first images.
- the computer obtains at least two first images, wherein each of the first images is a captured image at different angles to the same object, and the field of view of the first image is different due to different shooting angles, and the first image is obtained.
- the calibration data of the camera of the image wherein the calibration data includes an internal parameter matrix of the camera, an external parameter matrix, the internal reference matrix includes parameters such as resolution and focal length of the camera, and the external parameter matrix includes positional parameters such as translation and rotation of the camera.
- S102 Extract a background image and a foreground image of the first image, respectively.
- the pixel points constitute a foreground image of the first image.
- the pixel points of the first image corresponding to the complement 1 of the foreground template may be extracted to form a background image of the first image.
- the depth value of the first image is divided to define a threshold range, and the depth value is within the threshold range and the numbers “1” and “0” are respectively outside the threshold range, and the first image may be according to the depth value.
- Converted into a matrix consisting of "0" and "1” the formed matrix is a foreground template, and at the same pixel point, the pixel of the first image corresponding to the foreground template 1 is extracted to form a foreground image of the first image. Further, at the same pixel point, the pixel points of the first image corresponding to the complement 1 of the foreground template may be extracted to form a background image of the first image.
- the method of acquiring the foreground image and the background image of the first image may also be other foreground and background image detecting methods.
- S103 Determine a first overlapping image and a first non-overlapping image formed by the background images of the at least two first images, and determine a second overlapping image and a second non-overlapping image formed by the foreground images of the at least two first images.
- each background image is converted into a background virtual image by a homography transformation
- each foreground image is converted into a foreground virtual image by a homography transformation
- a first overlapping image and a first non-image of each background image are calculated according to the background virtual image.
- the images are superimposed, and then the second superimposed image and the second non-overlapping image of the respective foreground images are calculated from the foreground virtual images.
- the calculation method of the homography transformation may be determined according to an internal parameter matrix, an outer parameter matrix, and a depth value of each camera, and the depth value may be a depth value of a background image of a certain first image, or may be a customized depth value.
- the background image of the first image captured by the camera is The correspondence between the corresponding background virtual images is converted to Where d denotes [001], K i denotes the 3x3 internal reference matrix of the i-th camera, where K v can take the average of all camera internal parameters, and R v and t v can be obtained by interpolating all camera external parameters.
- the method of homography transformation is not limited to the above method.
- S104 Segmenting the first superimposed image to obtain a plurality of first divided sub-images, and dividing the second superimposed image to obtain a plurality of second divided sub-images.
- a first difference value of the first overlapping image on each color channel is calculated, and a Laplace filter transform and a smoothing process are sequentially performed on the first difference value to obtain a first Morse function value, and then according to the first mole.
- the function value determines the first divided sub-image on the first overlapping image, and the second overlapping image is also divided to obtain a plurality of second divided sub-images, specifically comprising: calculating a second difference of the second overlapping image on each color channel
- the value image is subjected to Laplacian transform transform and smoothing processing on the second difference image to obtain a second Morse function value, and then the second divided sub-image on the second overlapping image is determined according to the second Morse function value.
- Determining, according to the first Morse function value, the first divided sub-image on the first overlapping image comprises: determining a first local minimum on the first overlapping image according to a first Morse function value of the pixel point on the first overlapping image a value point, a first local maximum point, and a first saddle point, and determining a first segmentation sub-image according to each of the first local minimum point, a first local maximum point, and two first saddle points, that is, each The first divided sub-images each include the above four points.
- determining the second divided sub-image on the second overlapping image according to the second Morse function value specifically: determining, according to the second Morse function value of the pixel point on the second overlapping image, the second on the second overlapping image Local minimum point, second local maximum point and second saddle point, then root A second segmentation sub-image is determined according to each of the second local minimum points, a second local maximum point, and two second saddle points.
- the method further includes: determining a first maximum point of each first divided sub-image, using a region algorithm, determining a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point, and determining a second according to the second Morse function value
- the method further includes: determining a second maximum point of each second divided sub-image, and determining, by using a region algorithm, a second corresponding to the second overlapping image according to the second maximum point Dual split sub-images.
- S105 Select, for each first segmentation sub-image, a corresponding first tiled sub-image in the background images of the at least two first images, and select, in each of the foreground images of the at least two first images, for each second segmentation sub-image.
- the second splice image select, for each first segmentation sub-image, a corresponding first tiled sub-image in the background images of the at least two first images, and select, in each of the foreground images of the at least two first images, for each second segmentation sub-image. The second splice image.
- first calculating a smoothing overhead according to an average value of maximum color differences of the connected first dual-divided sub-images, calculating a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image, and then according to the smoothing
- the overhead and the data overhead are selected for each of the first dual-divided sub-images, and the corresponding first splice sub-images are selected for each of the second divided sub-images in the foreground image of the at least two first images.
- the second splicing sub-image includes: calculating a smoothing overhead according to an average value of maximum color differences of the connected second dual-divided sub-images, and calculating a data overhead according to a distance from the image center of the second overlapping image to the second dual-divided sub-image, Corresponding second splice sub-images are selected for each second dual-divided sub-image according to smoothing overhead and data overhead.
- the corresponding first splice sub-image is selected for each first dual-divided sub-image according to the smoothing overhead and the data overhead, including: first, constructing the first map according to the first dual-divided sub-image, the first graph includes: multiple The first dual-divided sub-image, and secondly, the weight of each edge of the first graph is determined according to the smoothing overhead and the data overhead. Finally, the first dual-divided sub-image is performed by using a graph cutting method according to the weight of each edge of the first graph.
- Sorting selecting a corresponding first splice sub-image for each first dual-divided sub-image, and similarly selecting a corresponding second splice sub-image for each second dual-divided sub-image according to smoothing overhead and data overhead
- the method includes: constructing a second map according to the second dual segmentation sub-image, and the second image includes: a plurality of second pairs Evenly dividing the sub-image, determining the weight of each edge of the second graph according to the smoothing overhead and the data overhead, and classifying the second dual-divided sub-image according to the weight of each edge of the second graph, according to the classification result
- Each second dual-divided sub-image selects a corresponding second splice sub-image.
- S106 splicing the first spliced sub-images corresponding to the plurality of first divided sub-images to obtain a target background image, and splicing the second spliced sub-images corresponding to the plurality of second divided sub-images to obtain a target foreground image.
- the first spliced sub-image is selected, all the first spliced sub-images are spliced to obtain a target background image, and the second spliced sub-image corresponding to the plurality of second divided sub-images is spliced to obtain a target foreground. image.
- S107 Synthesize the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a target panoramic image.
- the target image is formed by the synthesis algorithm on the target background image and the target foreground image, and then the target image and the first non-overlapping image and the second non-overlapping image are combined to obtain a target panoramic image.
- the present invention provides a method for acquiring a panoramic image, wherein at least a first overlapping image of a background image of at least two first images and a second overlapping image of a foreground image are segmented, at least for each of the first divided sub-images Selecting a corresponding first splice sub-image in the background image of the two first images, and selecting a corresponding second splice sub-image in the foreground image of the at least two first images for each second divided sub-image, the first splice
- the image is spliced to obtain a target background image
- the second spliced sub-image is spliced to obtain a target foreground image
- the target background image, the target foreground image, the first non-overlapping image and the second non-overlapping image are combined to obtain a larger view.
- High resolution target panoramic image of the field range At least a first overlapping image of a background image of at least two first images and a second overlapping image of a fore
- FIG. 2 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention.
- the method may be applied to the field of image processing technology, and the execution subject may be a smart device such as a computer, where the previous embodiment is based on the previous embodiment.
- the method is mainly the refinement of step S104, and specifically includes the following steps:
- S1041 Calculate a first difference value of the first overlapping image on each color channel, perform a Laplace filter transformation and a smoothing process on the first difference, and obtain a first Morse function value.
- calculating a first difference of the first overlapping image on the red, yellow, and blue color channels wherein the difference formula is I 1 and I 2 respectively represent the color functions corresponding to the two first images, (x, y) represents the position coordinates of the pixel points, i represents the i-th color channel, and the first overlapping image is measured by the difference formula described above.
- the misalignment condition is then subjected to Laplacian filter transformation and smoothing processing on the first difference to obtain a first Morse function value.
- S1042 Determine a first divided sub-image on the first overlapping image according to the first Morse function value.
- FIG. 3A is a schematic diagram of a first segmentation sub-image according to an embodiment of the present invention
- FIG. A first divided sub-image diagram provided by an embodiment, as shown in FIG. 3A and FIG. 3B, each of the four points defines a first divided sub-image, wherein the four points are: a first local minimum point, and a first Local maximum point and two first saddle points.
- the smoothing parameters selected in the smoothing process are different, so the size of the obtained first divided sub-images is not the same.
- the smoothing parameter used in FIG. 3A is 10, and the smoothing parameters used in FIG. 3B are used.
- FIG. 4A is a schematic diagram of a target panoramic image according to an embodiment of the present invention.
- the first dual-divided sub-image is obtained by using the region algorithm, so there is a first segmentation.
- the boundary of the sub-image is not aligned with the first overlapping image, so there may be a splicing slot.
- FIG. 4B is a schematic diagram of a target panoramic image according to another embodiment of the present invention, since the first overlapping image is determined according to the first Morse function value.
- the method further includes: determining a first maximum point of each first divided sub-image, and determining, by using a region algorithm, a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point, The method can solve the problem that the boundary of the first divided sub-image cannot be aligned with the first overlapping image, and thus the obtained target panoramic image does not have a stitching gap.
- a second difference image of the second overlapping image on each color channel may be calculated, and the second difference image is sequentially subjected to Laplacian transform transform and smoothing processing to obtain a second Morse function value.
- calculating a second difference of the second overlapping image on the red, yellow, and green color channels where the difference formula is I 1 and I 2 respectively represent the color functions corresponding to the two cameras providing the second image, (x, y) represents the coordinate position of the pixel point, and i represents the i-th color channel, which is measured by the difference formula described above.
- the two overlapping images are misaligned, and then the second difference is sequentially subjected to Laplacian transform and smoothing to obtain a second Morse function value. Then, the second divided sub-image on the second overlapping image is determined according to the second Morse function value.
- the embodiment provides a method for acquiring a panoramic image, wherein the first and second overlapping images on the first overlapping image are determined by calculating a first Morse function value and a second Morse function value.
- the second divided sub-image is further divided into a first overlapping image and a second overlapping image by using a region algorithm, so that the first overlapping image and the second overlapping image are aligned and segmented, and then the stitching method is further obtained.
- FIG. 5 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention.
- the method may be applied to the field of image processing technology, and the execution subject may be a smart device such as a computer, where the first embodiment is based on the first embodiment.
- the method is mainly for the refinement of step S105, and specifically includes the following steps:
- S1051 Calculate a smoothing overhead according to an average value of maximum color differences of the connected first dual-divided sub-images, and calculate a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image.
- a color difference is obtained on the red, yellow and blue color channels, and the color difference indicates that the first background virtual image captured by the two cameras is in each pixel.
- the color difference on the point, the maximum color difference is obtained, and finally the maximum color difference on the three color channels is averaged, the smoothing overhead is determined according to the average value, and, according to the image center of the first overlapping image to the first dual segmentation sub-image The distance calculates the data overhead.
- S1052 Select a corresponding first splice sub-image for each first dual-divided sub-image according to the smoothing overhead and the data overhead.
- the first map according to the first dual-divided sub-image determining the weight of each edge of the first graph according to the smoothing overhead and the data overhead, and using the graph cutting method to the first according to the weight of each edge of the first graph
- the dual-divided sub-images are classified, and corresponding first splice sub-images are selected for each first dual-divided sub-image according to the classification result. For example, if there are two cameras TL and TR, the cutting line of the first dual-divided sub-image is determined by the graph cutting method. Since the first graph includes the first dual-cut slice, the terminal node is further included, and the terminal node is used for FIG.
- FIG. 6A is a schematic diagram of a graph cut according to still another embodiment of the present invention.
- the node s and the node t respectively represent two cameras. If a first dual split sub-image and the node s are on the same side of the cutting line, The corresponding area image in the first background image captured by the camera corresponding to the first dual-divided sub-image selection node s is the first splicing sub-image, and FIG. 6B is a schematic diagram of the classification result of the dicing cut according to another embodiment of the present invention.
- the sub-image 601 is composed of a first divided sub-image formed by a first image captured by the camera TL
- the sub-image 602 is composed of a first divided sub-image formed by the first image captured by the camera TR, according to the segmentation
- the first splice sub-image may be selected.
- the first splice sub-image corresponding to one of the first divided sub-images in the sub-image 601 is taken from the first image captured by the TL.
- the first stitching sub-image is stitched to form a target background image
- the second stitching sub-image can be selected in the same manner to finally form a target background image.
- FIG. 6C is a target panoramic image according to another embodiment of the present invention.
- the schematic diagram, as shown in FIG. 6C, is to obtain a target panoramic image by synthesizing the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image.
- the smoothing overhead may also be calculated according to the average of the maximum color differences of the connected second dual-divided sub-images, and the data overhead is calculated according to the distance from the image center of the second overlapping image to the second dual-divided sub-image.
- a color difference is obtained on the red, yellow and green color channels, and the color difference indicates that the first foreground virtual image captured by the two cameras is in each pixel.
- the color difference at the point, the maximum color difference is obtained, and finally the maximum color difference on the three color channels is averaged, the smoothing overhead is determined according to the average value, and, according to the image center of the second overlapping image to the second dual segmentation sub-image The distance calculates the data overhead. Then, according to the smoothing overhead and the data overhead, the corresponding first is selected for each second dual-divided sub-image. Two stitched sub-images.
- constructing a second map according to the second dual-divided sub-image determining a weight of each edge of the second graph according to the smoothing overhead and the data overhead, and using the graph cutting method to the second according to the weight of each edge of the second graph
- the dual-divided sub-images are classified, and corresponding second splice sub-images are selected for each second dual-divided sub-image according to the classification result.
- the embodiment provides a method for acquiring a panoramic image, which mainly includes: selecting a corresponding first splice sub-image and each second dual split for each first dual-divided sub-image by calculating a smooth overhead and a data overhead.
- the sub-image selects the corresponding second splice sub-image, and achieves effective splicing on the basis of the above-described graph cutting method, thereby obtaining a high-resolution target panoramic image with a larger field of view range.
- FIG. 7 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention.
- the method may be applied to the technical field of image processing, and the execution subject may be a smart device such as a computer.
- the embodiment exemplifies a method for acquiring a panoramic image.
- FIG. 8 is a schematic diagram of a camera array according to another embodiment of the present invention. It is assumed that there are four cameras TL, TR, BL, and BR, and the camera array of 2*2 is the same. The object is photographed, and the shooting angles thereof are different.
- the depth value may be a depth value of the background image of a certain first image, or may be a customized depth value.
- the method for acquiring the panoramic image specifically includes the following steps:
- S701 Acquire four first images and corresponding four foreground templates.
- the computer obtains four first images, wherein the four first images are respectively composed of four cameras TL, TR, BL, and BR, and the camera array of 2*2 is taken by the same object, and the shooting angle thereof is not The same, so the field of view of the first image is different, and the calibration data of the camera for capturing the first image is obtained, wherein the calibration data includes an internal parameter matrix of the camera, an external parameter matrix, and the internal reference matrix includes parameters such as resolution and focal length of the camera.
- the outer parameter matrix includes positional parameters such as translation, rotation, and the like of the camera.
- the first image may be converted into one according to the depth value a matrix consisting of "0" and "1"
- S702 Extract a foreground image and a background image of each first image by using a foreground template.
- the depth value of the first image is divided to define a threshold range, where the depth value is within the threshold range and outside the threshold range respectively correspond to binary “1” and “0”, then
- the first image may be converted into a matrix consisting of “0” and “1” according to the depth value, and the formed matrix is a foreground template, and at the same pixel point, the pixel of the first image corresponding to the foreground template 1 is extracted, and the pixel is formed.
- the image and background image methods can also be other foreground, background image detection methods.
- S703 Determine a depth value of the background image, and a homography transformation of the background image to the virtual background image.
- the method for determining the depth value may be a depth value of the background image of a certain first image as a depth value of the background image, or may be a customized depth value.
- the homography transformation is determined by: assuming that the normal vector n of the plane in which the background virtual image is located is parallel to the z-axis of the coordinate system of the i-th camera, and the distance between the camera and the camera is z bg , then the camera The homography between the captured background image of the first image and the corresponding first background virtual image is converted to Where d denotes [001], K i denotes the 3x3 internal reference matrix of the i-th camera, where K v can take the average of all camera internal parameters, and R v and t v can be obtained by interpolating all camera external parameters.
- the method of homography transformation is not limited to the above method.
- S704 Perform segmentation based on the first overlapping image of the first image captured by the cameras TL and TR respectively, and perform splicing of the background image based on the segmentation result, and select the first spliced sub-image to form a target background image.
- a first difference value of the first overlapping image on each color channel is calculated, and a Laplace filter transform and a smoothing process are sequentially performed on the first difference value to obtain a first Morse function value, and then according to the first mole.
- the sigma function value determines a first segmentation sub-image on the first superimposed image, wherein determining the first segmentation sub-image on the first superimposed image according to the first Morse function value specifically includes: according to the pixel point on the first superimposed image a Morse function value determining a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image, according to each of the first local minimum point, a first local maximum point, and two The first saddle point determines a first segmentation sub-image, that is to say each of the first segmentation sub-images includes the above four points.
- the method further includes: determining First maximum of each first segmented sub-image Point, using a region algorithm, determining a first dual segmentation sub-image corresponding to the first overlapping image according to the first maximum point.
- selecting a corresponding first splice sub-image in the background images of the at least two first images for each of the first divided sub-images, forming a target background image including: according to a maximum color difference of the connected first dual-divided sub-images Calculating a smoothing overhead by an average value, calculating a data overhead according to a distance from the image center of the first overlapping image to the first dual-divided sub-image, and then selecting, for each of the first dual-divided sub-images, according to the smoothing overhead and the data overhead Corresponding to the first splice sub-image,
- the corresponding first splice sub-image is selected for each first dual-divided sub-image according to the smoothing overhead and the data overhead, including: first, constructing the first map according to the first dual-divided sub-image, the first graph includes: multiple The first dual-divided sub-image, and secondly, the weight of each edge of the first graph is determined according to the smoothing overhead and the data overhead. Finally, the first dual-divided sub-image is performed by using a graph cutting method according to the weight of each edge of the first graph. Classification, selecting a corresponding first splice sub-image for each first dual-divided sub-image according to the classification result. Finally, the first splice sub-images described above are spliced to form a target background image.
- S705 Perform segmentation based on the first overlapping image of the first image captured by the cameras BL and BR respectively, and perform splicing of the background image based on the segmentation result, and select the first spliced sub-image to form a target background image.
- This step is similar to S704 and will not be described here.
- S706 segment the superimposed image of the target background image formed by the first image captured by the TL and the TR and the target background image formed by the first image captured by the BL and the BR, and form a final target background image based on the segmentation result.
- FIG. 9 is a schematic diagram of background image mosaic according to another embodiment of the present invention.
- the first line of images is the first image captured by cameras TL, TR, BL, and BR, and the second line is left.
- the image is the target background image formed by the first image captured by TL and TR
- the image on the right is the target background image formed by the first image captured by BL and BR
- the image on the third line is taken on TL and TR.
- the target background image formed by the first image and the overlapping image of the target background image formed by the first image captured by the BL and the BR are segmented, and a final target background image is formed based on the segmentation result.
- S707 Determine a depth value of the foreground image, and a homography transformation of the foreground image to the virtual foreground image.
- S708 Perform segmentation based on the second overlapping image of the first image captured by the cameras TL and TR respectively, and perform splicing of the foreground image based on the segmentation result to form a target foreground image.
- S709 Determine a target panoramic image according to the first splice sub image and the second splice sub image.
- the target non-overlapping image determined by the target background image, the target foreground image, and the background image of the first image, and the second non-overlapping image determined by the foreground image of the first image are synthesized, and finally the target panoramic image is obtained.
- the target background image obtained by the background image stitching process shown in FIG. 9 is similar to the foreground stitching process of the background image stitching process shown in FIG. 9, the obtained target foreground image, and simultaneously determined for the background image of the first image.
- the first non-overlapping image, the second non-overlapping image determined by the foreground image of the first image are synthesized, and finally the target panoramic image is obtained.
- the present invention provides a method for acquiring a panoramic image, which includes dividing a first overlapping image to form a first divided sub-image, and equally dividing the second overlapping image to form a second divided sub-image for each first
- the segmented sub-image is selected corresponding to the first spliced sub-image, and the corresponding second spliced sub-image is selected for each second sub-sub-image
- the first spliced sub-image is spliced to obtain the target background image
- the second spliced sub-image is spliced
- the target foreground image is finally combined with the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a high-resolution target panoramic image of a larger field of view.
- FIG. 10 is a schematic structural diagram of a device for acquiring a panoramic image according to an embodiment of the present invention.
- the device may be a smart device such as a computer.
- the device for acquiring a panoramic image includes: an acquiring module 111, configured to acquire at least two first An image extraction module 112, configured to extract a background image and a foreground image of the first image; a determining module 113, configured to determine a first overlapping image and a first non-overlapping image formed by the background images of the at least two first images, and determine a second overlapping image and a second non-overlapping image formed by the foreground image of the at least two first images; wherein the extracting module 112 is configured to: determine a foreground template of the first image according to the depth value of the first image; the first foreground template is a matrix consisting of 0,1, at the same pixel point, extracting pixel points of the first image corresponding to the foreground template 1 to form a foreground image of the first image; at the
- the determining module 113 is specifically configured to: convert each first background image into a first background virtual image by a homography conversion, convert each first foreground image into a first foreground virtual image by a homography conversion; The image calculates a first overlap image and a first non-overlapping image of each of the first background images, and calculates a second overlap image and a second non-overlapping image of each of the first foreground images according to the first foreground virtual image.
- the segmentation module 114 is configured to segment the first overlapping image to obtain a plurality of first divided sub-images, and divide the second overlapping image to obtain a plurality of second divided sub-images; and the selecting module 115 is configured to The sub-image selects a corresponding first splice sub-image in the background images of the at least two first images, and selects a corresponding second splice image in the foreground image of the at least two first images for each second divided sub-image;
- the module 116 is configured to splicing a first spliced sub-image corresponding to the plurality of first divided sub-images to obtain a target background image, and splicing the second spliced sub-image corresponding to the plurality of second divided sub-images to obtain a target foreground image;
- the module 117 is configured to synthesize the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a target panoramic image.
- the device for acquiring the panoramic image of the embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 1 , and the implementation principle and technical effects thereof are similar, and details are not described herein again.
- the segmentation module 114 is specifically configured to: calculate a first difference value of the first overlapping image on each color channel, and perform a Laplace filter transformation on the first difference sequentially. Smoothing to obtain a first Morse function value; determining a first segmentation sub-image on the first overlay image according to the first Morse function value; and further for calculating a second difference of the second overlay image on each color channel.
- the value image is subjected to Laplacian transform transform and smoothing processing on the second difference image to obtain a second Morse function value; and the second divided sub-image on the second superimposed image is determined according to the second Morse function value.
- the segmentation module 114 is further configured to: determine, according to a first Morse function value of the pixel point on the first overlapping image, a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image Determining a first segmentation sub-image according to each of the first local minimum point, a first local maximum point, and the two first saddle points; and further for using a second Morse function of the pixel points on the second overlapping image The value determines a second local minimum point on the second overlapping image, a second local maximum point and a second saddle point; and a second local maximum point and two second saddle points are determined according to each of the second local minimum points A second segmented sub-image.
- the segmentation module 114 is further configured to: determine a first maximum point of each first segmentation sub-image, and determine, according to the first maximum point, using a region algorithm Determining a first dual-divided sub-image corresponding to the first overlapping image; determining a second maximum point of each second divided sub-image, determining a second dual-divided sub-correlation corresponding to the second overlapping image according to the second maximum point using a region algorithm image.
- the device for acquiring the panoramic image in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 2, and the implementation principle and technical effects are similar, and details are not described herein again.
- the selecting module 115 is specifically configured to: calculate a smoothing overhead according to an average value of maximum color differences of the connected first dual-divided sub-images, according to the image of the first overlapping image Calculating a data overhead from a distance from the center to the first dual-divided sub-image; selecting a corresponding first splice sub-image for each first dual-divided sub-image according to smoothing overhead and data overhead; according to a maximum color of the connected second dual-divided sub-image The average value of the difference is used to calculate a smoothing overhead, and the data overhead is calculated according to the distance from the image center of the second overlapping image to the second dual-divided sub-image; and the second stitching is selected for each second dual-divided sub-image according to the smoothing overhead and the data overhead.
- the selecting module 115 is further configured to: construct a first map according to the first dual-divided sub-image, where the first image includes: a plurality of first dual-divided sub-images; and determining, according to the smoothing overhead and the data overhead, the right of each side of the first graph
- the first dual-divided sub-image is classified according to the weight of each edge of the first graph by using a graph cutting method, and the corresponding first splice sub-image is selected for each first dual-divided sub-image according to the classification result; according to the smoothing overhead
- the data overhead is: selecting a second second sub-segment image for each second dual-divided sub-image, comprising: constructing a second image according to the second dual-divided sub-image, the second image comprising: a plurality of second dual-divided sub-images; Smoothing overhead and data overhead determine the weight of each edge of the second graph; according to the weight of each edge of the second graph, the second dual--
- the apparatus for acquiring the panoramic image of the embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 5, and the implementation principle and technical effects are similar, and details are not described herein again.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
Provided are a method and device for acquiring a panoramic image, the method comprising: acquiring at least two first images; extracting the background images and foreground images of the first images; determining a first overlapping image and a first non-overlapping image formed by the background images of the at least two first images, and determining a second overlapping image and a second non-overlapping image formed by the foreground images of the at least two first images; dividing the first overlapping image to obtain a plurality of first divided sub-images, and dividing the second overlapping image to obtain a plurality of second divided sub-images; selecting first splicing sub-images for each first divided sub-image, and selecting second splicing sub-images for each second divided sub-image; splicing the first splicing sub-images to obtain a target background image, and splicing the second splicing sub-images to obtain a target foreground image; and synthesizing the target background image, the target foreground image, the first non-overlapping image and the second non-overlapping image to obtain a target panoramic image having high resolution.
Description
本发明实施例涉及图像处理技术,尤其涉及全景图像的获取方法和装置。Embodiments of the present invention relate to image processing technologies, and in particular, to a method and an apparatus for acquiring a panoramic image.
随着科技的发展,人们对于分辨率的追求达到一个新的高度,尤其是显示设备的屏幕越来愈大,这也客观上要求采集设备有更高的分辨率。With the development of technology, people's pursuit of resolution has reached a new height, especially the display device's screen is getting bigger and bigger, which also objectively requires higher resolution of the acquisition device.
目前超高清的视频(图像)采集设备的价格十分昂贵,于是通过相对廉价的较低分辨率的相机组成相机阵列就成了一种自然的替代方案。现有的全景拼接技术,将多个相机的光心重合(或者近似重合),从而得到大的视场的全景图像。At present, ultra-high definition video (image) acquisition devices are very expensive, so it is a natural alternative to form a camera array with relatively inexpensive lower resolution cameras. The existing panoramic stitching technique superimposes (or approximately coincides) the optical centers of a plurality of cameras, thereby obtaining a panoramic image of a large field of view.
然而,这些技术需要精密的光学设备以实现多个相机的光心重合,实现过程比较复杂、所需成本较高。However, these technologies require sophisticated optical devices to achieve optical center coincidence of multiple cameras, which is complicated and costly.
发明内容Summary of the invention
本发明提供一种全景图像的获取方法和装置,从而实现无需相机之间共光心,只需要通过标定获得相机的内外参数便可得到大的视场的高分辨率全景图像。The invention provides a method and a device for acquiring a panoramic image, so that a high-resolution panoramic image with a large field of view can be obtained only by obtaining the internal and external parameters of the camera without calibration.
第一方面,本发明实施例提供一种全景图像的获取方法,包括:获取至少两张第一图像;分别提取所述第一图像的背景图像和前景图像;确定至少两张所述第一图像的背景图像形成的第一重叠图像和第一非重叠图像,以及确定至少两张所述第一图像的前景图像形成的第二重叠图像和第二非重叠图像;对所述第一重叠图像进行分割得到多个第一分割子图像,对所述第二重叠图像进行分割得到多个第二分割子图像;为每个所述第一分割子图像在至少两张所述第一图像的背景图像中选取对应
第一拼接子图像,为每个所述第二分割子图像在至少两张所述第一图像的前景图像中选取对应的第二拼接子图像;对多个所述第一分割子图像对应的第一拼接子图像进行拼接得到目标背景图像,对多个所述第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像;对所述目标背景图像、所述目标前景图像、所述第一非重叠图像和所述第二非重叠图像进行合成,得到目标全景图像。In a first aspect, an embodiment of the present invention provides a method for acquiring a panoramic image, including: acquiring at least two first images; respectively extracting a background image and a foreground image of the first image; and determining at least two of the first images. a first overlapping image and a first non-overlapping image formed by the background image, and a second overlapping image and a second non-overlapping image formed by determining at least two foreground images of the first image; performing the first overlapping image Dividing to obtain a plurality of first divided sub-images, and dividing the second overlapping image to obtain a plurality of second divided sub-images; for each of the first divided sub-images, at least two background images of the first image Select corresponding
a first splicing sub-image, selecting, for each of the second divided sub-images, a corresponding second splicing sub-image in at least two foreground images of the first image; corresponding to the plurality of the first divided sub-images The first spliced sub-image is spliced to obtain a target background image, and the second spliced sub-image corresponding to the plurality of the second divided sub-images is spliced to obtain a target foreground image; and the target background image, the target foreground image, and the target image The first non-overlapping image and the second non-overlapping image are combined to obtain a target panoramic image.
结合第一方面,在第一方面的第一种可能的实现方式中,根据所述第一图像的深度值确定所述第一图像的前景模板;所述第一图像的前景模板为0,1组成的矩阵,在同一像素点处,提取前景模板1所对应的所述第一图像的像素点,构成所述第一图像的前景图像;在同一像素点处,提取前景模板的补中1所对应的所述第一图像的像素点,构成所述第一图像的背景图像。With reference to the first aspect, in a first possible implementation manner of the first aspect, the foreground template of the first image is determined according to the depth value of the first image; the foreground template of the first image is 0, 1 Forming a matrix, at the same pixel point, extracting pixel points of the first image corresponding to the foreground template 1 to form a foreground image of the first image; and extracting one of the foreground templates at the same pixel point Corresponding pixel points of the first image constitute a background image of the first image.
结合第一方面或第一方面的第一种可能的实现方式,在第一方面的第二种可能的实现方式中,所述确定至少两张所述第一图像形成的背景图像的第一重叠图像和第一非重叠图像,以及确定至少两张所述第一图像形成的前景图像的第二重叠图像和第二非重叠图像,具体包括:通过单应变换将每个所述背景图像转换为背景虚拟图像,通过单应变换将每个所述前景图像转换为前景虚拟图像;根据所述背景虚拟图像计算各个所述背景图像的所述第一重叠图像和所述第一非重叠图像,根据所述前景虚拟图像计算各个所述前景图像的所述第二重叠图像和所述第二非重叠图像。In conjunction with the first aspect or the first possible implementation of the first aspect, in a second possible implementation of the first aspect, the determining the first overlap of the at least two background images formed by the first image And the image and the first non-overlapping image, and determining the second overlapping image and the second non-overlapping image of the foreground image formed by the at least two of the first images, specifically comprising: converting each of the background images into a background virtual image, each of the foreground images being converted into a foreground virtual image by a homography transformation; calculating the first overlapping image and the first non-overlapping image of each of the background images according to the background virtual image, according to The foreground virtual image calculates the second overlapping image and the second non-overlapping image of each of the foreground images.
结合第一方面或第一方面的第一种可能的实现方式或第一方面的第二种可能的实现方式,在第一方面的第三种可能的实现方式中,所述对所述第一重叠图像进行分割得到多个第一分割子图像,包括:计算所述第一重叠图像在各个颜色通道上的第一差值,对所述第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值;根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像;对所述第二重叠图像进行分割得到多个第二分割子图像,具体包括:计算所述第二重叠图像在各个颜色通道上的第二差值图像,对所述第二差值图像依次进行拉普拉斯滤波变换和平滑处理,得到第二摩尔斯函数值;根据
所述第二摩尔斯函数值确定所述第二重叠图像上的所述第二分割子图像。With reference to the first aspect, or the first possible implementation manner of the first aspect, or the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, The superimposing the image to obtain the plurality of first divided sub-images comprises: calculating a first difference value of the first overlapping image on each color channel, and performing Laplacian filtering transformation and smoothing on the first difference Processing, obtaining a first Morse function value; determining the first divided sub-image on the first overlapping image according to the first Morse function value; and dividing the second overlapping image to obtain a plurality of second The dividing the sub-image includes: calculating a second difference image of the second overlapping image on each color channel, performing Laplacian filtering transformation and smoothing processing on the second difference image to obtain a second mole Function value;
The second Morse function value determines the second divided sub-image on the second overlay image.
结合第一方面的第三种可能的实现方式,在第一方面的第四种可能的实现方式中,所述根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像,具体包括:根据所述第一重叠图像上的像素点的第一摩尔斯函数值确定所述第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点;根据每一个所述第一局部最小值点、一个第一局部最大值点和两个所述第一鞍点确定一个第一分割子图像;所述根据所述第二摩尔斯函数值确定所述第二重叠图像上的所述第二分割子图像,具体包括:根据所述第二重叠图像上的像素点的第二摩尔斯函数值确定所述第二重叠图像上的第二局部最小值点,第二局部最大值点和第二鞍点;根据每一个所述第二局部最小值点,一个第二局部最大值点和两个所述第二鞍点确定一个第二分割子图像。In conjunction with the third possible implementation of the first aspect, in a fourth possible implementation of the first aspect, the determining, by the first Morse function value, the first on the first overlapping image The method of determining the first partial minimum point, the first local maximum point, and the first overlapping point on the first overlapping image according to the first Morse function value of the pixel point on the first overlapping image a saddle point; determining a first segmentation sub-image according to each of the first local minimum point, a first local maximum point, and two of the first saddle points; the determining according to the second Morse function value The second divided sub-image on the second overlapping image specifically includes: determining a second local minimum on the second overlapping image according to a second Morse function value of a pixel point on the second overlapping image a value point, a second local maximum point and a second saddle point; a second segmentation sub-image is determined according to each of the second local minimum points, a second local maximum point, and two of the second saddle points.
结合第一方面的第四种可能的实现方式,在第一方面的第五种可能的实现方式中,所述根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像之后,还包括:确定每个所述第一分割子图像的第一最大值点,使用区域算法,根据所述第一最大值点确定所述第一重叠图像对应的第一对偶分割子图像;所述根据所述第二摩尔斯函数值确定所述第二重叠图像上的所述第二分割子图像之后,还包括:确定每个所述第二分割子图像的第二最大值点,使用区域算法,根据所述第二最大值点确定所述第二重叠图像对应的第二对偶分割子图像。In conjunction with the fourth possible implementation of the first aspect, in a fifth possible implementation manner of the first aspect, the determining, by the first Morse function value, the first on the first overlapping image After dividing the sub-image, the method further includes: determining a first maximum point of each of the first divided sub-images, and determining, by using a region algorithm, a first duality corresponding to the first overlapping image according to the first maximum point Dividing the sub-image; after determining the second divided sub-image on the second overlapping image according to the second Morse function value, further comprising: determining a second maximum of each of the second divided sub-images a value point, using a region algorithm, determining a second dual segmentation sub-image corresponding to the second overlapping image according to the second maximum point.
结合第一方面的第五种可能的实现方式,在第一方面的第六种可能的实现方式中,所述为每个所述第一分割子图像在至少两个所述第一图像的背景图像中选取对应第一拼接子图像,具体包括:根据相连的所述第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据所述第一重叠图像的图像中心到所述第一对偶分割子图像的距离计算数据开销;根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像;所述为每个所述第二分割子图像在至少两个所述第一图像的前景图像中选取对应的第二拼接子图像,具体包括:根据相连的所述第二对偶分割子图像的最大颜色差的平均值计算光
滑开销,根据所述第二重叠图像的图像中心到所述第二对偶分割子图像的距离计算数据开销;根据所述光滑开销和所述数据开销为每个所述第二对偶分割子图像选取对应的所述第二拼接子图像。In conjunction with the fifth possible implementation of the first aspect, in a sixth possible implementation manner of the first aspect, the background of each of the first divided sub-images in at least two of the first images Selecting a corresponding first spliced sub-image in the image, the method comprising: calculating a smoothing overhead according to an average value of the maximum color difference of the connected first dual-divided sub-images, according to the image center of the first overlapping image to the first Calculating a data overhead for the distance of the dual-divided sub-image; selecting the corresponding first splice sub-image for each of the first dual-divided sub-images according to the smoothing overhead and the data overhead; The second divided sub-image selects the corresponding second splice sub-image in the foreground image of the at least two of the first images, and specifically includes: calculating light according to an average value of maximum color differences of the connected second dual-divided sub-images
a sliding overhead, calculating a data overhead according to a distance from an image center of the second overlapping image to the second dual-divided sub-image; selecting each of the second dual-divided sub-images according to the smoothing overhead and the data overhead Corresponding to the second splice sub-image.
结合第一方面的第六种可能的实现方式,在第一方面的第七种可能的实现方式中,所述根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像,包括:根据所述第一对偶分割子图像构造第一图;根据所述光滑开销和所述数据开销确定所述第一图每条边的权值;根据所述第一图每条边的权值采用图切割方法对所述第一对偶分割子图像进行分类,根据分类结果为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像;所述根据所述光滑开销和所述数据开销为每个所述第二对偶分割子图像选取对应的所述第二拼接子图像,包括:根据所述第二对偶分割子图像构造第二图;根据所述光滑开销和所述数据开销确定所述第二图每条边的权值;根据所述第二图每条边的权值采用图切割方法对所述第二对偶分割子图像进行分类,根据分类结果为每个所述第二对偶分割子图像选取对应的所述第二拼接子图像。With reference to the sixth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, the performing the first dual-divided sub-image according to the smoothing overhead and the data overhead Selecting the corresponding first spliced sub-image includes: constructing a first map according to the first dual-divided sub-image; determining a weight of each side of the first graph according to the smoothing overhead and the data overhead; And the first dual-divided sub-image is classified according to the weight of each edge of the first figure by using a graph cutting method, and the first stitching corresponding to each of the first dual-divided sub-images is selected according to the classification result. a sub-image; the selecting the corresponding second splice sub-image for each of the second dual-divided sub-images according to the smoothing overhead and the data overhead, comprising: constructing according to the second dual-divided sub-image structure a second graph; determining a weight of each edge of the second graph according to the smoothing overhead and the data overhead; and using the graph cutting method to the second dual segment according to a weight of each edge of the second graph Figure Classified according to the classification result of each of said second dual segmented sub-images corresponding to the selected second sub-image mosaic.
第二方面,本发明实施例提供一种全景图像的获取装置,包括:获取模块,用于获取至少两张第一图像;提取模块,用于提取所述第一图像的背景图像和前景图像;确定模块,用于确定所述至少两张第一图像第一图像的背景图像的第一重叠图像和第一非重叠图像,以及确定所述至少两张第一图像第一图像的前景图像的第二重叠图像和第二非重叠图像;分割模块,用于对所述第一重叠图像进行分割得到多个第一分割子图像,对所述第二重叠图像进行分割得到多个第二分割子图像;选取模块,用于为每个所述第一分割子图像在至少两个所述第一图像的背景图像中选取对应第一拼接子图像,为每个所述第二分割子图像在至少两个所述第一图像的前景图像中选取对应的第二拼接子图像;拼接模块,用于对多个所述第一分割子图像对应的第一拼接子图像进行拼接得到目标背景图像,对多个所述第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像;合成模块,用于对所述目标背景图像、所述目标前景图像、所述第一非重叠图像和所述第二非重叠图像进行合成,得到目
标全景图像。In a second aspect, an embodiment of the present invention provides a device for acquiring a panoramic image, including: an acquiring module, configured to acquire at least two first images; and an extracting module, configured to extract a background image and a foreground image of the first image; a determining module, configured to determine a first overlapping image and a first non-overlapping image of the background image of the at least two first image first images, and to determine a foreground image of the at least two first image first images a second overlapping image and a second non-overlapping image; a segmentation module, configured to segment the first overlapping image to obtain a plurality of first divided sub-images, and divide the second overlapping image to obtain a plurality of second divided sub-images a selection module, configured to select, for each of the first divided sub-images, a corresponding first splice sub-image in a background image of at least two of the first images, and at least two for each of the second divided sub-images Corresponding second splicing sub-images are selected from the foreground image of the first image; a splicing module is configured to perform the first splicing sub-image corresponding to the plurality of the first divided sub-images And obtaining a target background image, and splicing the second spliced sub-image corresponding to the plurality of the second divided sub-images to obtain a target foreground image; and a synthesizing module, configured to: the target background image, the target foreground image, and the Combining the first non-overlapping image and the second non-overlapping image to obtain a mesh
Panoramic image.
结合第二方面,在第二方面的第一种可能的实现方式中,根据所述第一图像的深度值确定所述第一图像的前景模板;所述第一前景模板为0,1组成的矩阵,在同一像素点处,提取前景模板1所对应的所述第一图像的像素点,构成所述第一图像的前景图像;在同一像素点处,提取前景模板的补中1所对应的所述第一图像的像素点,构成所述第一图像的背景图像。With reference to the second aspect, in a first possible implementation manner of the second aspect, the foreground template of the first image is determined according to the depth value of the first image; the first foreground template is composed of 0, 1. a matrix, at a same pixel, extracting pixel points of the first image corresponding to the foreground template 1 to form a foreground image of the first image; and extracting a corresponding one of the foreground templates at the same pixel point The pixel of the first image constitutes a background image of the first image.
结合第二方面或第二方面的第一种可能的实现方式,在第二方面的第二种可能的实现方式中,所述确定模块具体用于:通过单应转换将每个所述第一背景图像转换为第一背景虚拟图像,通过单应转换将每个所述第一前景图像转换为第一前景虚拟图像;根据所述第一背景虚拟图像计算各个所述第一背景图像的所述第一重叠图像和所述第一非重叠图像,根据所述第一前景虚拟图像计算各个所述第一前景图像的所述第二重叠图像和所述第二非重叠图像。With reference to the second aspect, or the first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the determining module is specifically configured to: Converting the background image into a first background virtual image, converting each of the first foreground images into a first foreground virtual image by homography conversion; calculating the respective of the first background images according to the first background virtual image a first overlapping image and the first non-overlapping image, and calculating the second overlapping image and the second non-overlapping image of each of the first foreground images according to the first foreground virtual image.
结合第二方面或第二方面的第一种可能的实现方式或第二方面的第二种可能的实现方式,在第二方面的第三种可能的实现方式中,所述分割模块具体用于:计算所述第一重叠图像在各个颜色通道上的第一差值,对所述第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值;根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像;计算所述第二重叠图像在各个颜色通道上的第二差值图像,对所述第二差值图像依次进行拉普拉斯滤波变换和平滑处理,得到第二摩尔斯函数值;根据所述第二摩尔斯函数值确定所述第二重叠图像上的所述第二分割子图像。With reference to the second aspect or the first possible implementation manner of the second aspect, or the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the segmentation module is specifically used to Calculating a first difference value of the first overlapping image on each color channel, performing a Laplace filter transformation and smoothing processing on the first difference value to obtain a first Morse function value; a Morse function value determining the first divided sub-image on the first overlapping image; calculating a second difference image of the second overlapping image on each color channel, sequentially for the second difference image Performing a Laplacian transform transform and smoothing to obtain a second Morse function value; and determining the second divided sub-image on the second overlapping image according to the second Morse function value.
结合第二方面的第三种可能的实现方式,在第二方面的第四种可能的实现方式中,所述分割模块还用于:根据所述第一重叠图像上的像素点的第一摩尔斯函数值确定所述第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点;根据每一个所述第一局部最小值点、一个第一局部最大值点和两个所述第一鞍点确定一个第一分割子图像;根据所述第二重叠图像上的像素点的第二摩尔斯函数值确定所述第二重叠图像上的第二局部最小值点,第二局部最大值点和第二鞍点;根据每一
个所述第二局部最小值点,一个第二局部最大值点和两个所述第二鞍点确定一个第二分割子图像。In conjunction with the third possible implementation of the second aspect, in a fourth possible implementation of the second aspect, the segmentation module is further configured to: determine, according to a first mole of pixels on the first overlapping image The sigma function value determines a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image; according to each of the first local minimum point, a first local maximum point, and Determining, by the two first saddle points, a first divided sub-image; determining a second local minimum point on the second overlapping image according to a second Morse function value of the pixel on the second overlapping image, Two local maximum points and a second saddle point; according to each
The second partial minimum point, a second local maximum point, and the two second saddle points determine a second segmentation sub-image.
结合第二方面的第四种可能的实现方式,在第二方面的第五种可能的实现方式中,所述分割模块还用于:确定每个所述第一分割子图像的第一最大值点,使用区域算法,根据所述第一最大值点确定所述第一重叠图像对应的第一对偶分割子图像;确定每个所述第二分割子图像的第二最大值点,使用区域算法,根据所述第二最大值点确定所述第二重叠图像对应的第二对偶分割子图像。In conjunction with the fourth possible implementation of the second aspect, in a fifth possible implementation of the second aspect, the segmentation module is further configured to: determine a first maximum value of each of the first segmentation sub-images Point, using a region algorithm, determining a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point; determining a second maximum point of each of the second divided sub-images, using a region algorithm And determining, according to the second maximum point, a second dual-divided sub-image corresponding to the second overlapping image.
结合第二方面的第五种可能的实现方式,在第二方面的第六种可能的实现方式中,所述选取模块具体用于:根据相连的所述第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据所述第一重叠图像的图像中心到所述第一对偶分割子图像的距离计算数据开销;根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像;根据相连的所述第二对偶分割子图像的最大颜色差的平均值计算光滑开销,根据所述第二重叠图像的图像中心到所述第二对偶分割子图像的距离计算数据开销;根据所述光滑开销和所述数据开销为每个所述第二对偶分割子图像选取对应的所述第二拼接子图像。With reference to the fifth possible implementation of the second aspect, in a sixth possible implementation manner of the second aspect, the selecting module is specifically configured to: determine, according to a maximum color difference of the connected first dual-divided sub-images Calculating a smoothing overhead, calculating a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image; and each of the first duals according to the smoothing overhead and the data overhead Dividing the sub-image to select the corresponding first splice sub-image; calculating a smoothing overhead according to the average of the maximum color differences of the connected second dual-divided sub-images, according to the image center of the second overlapping image to the Calculating a data overhead of the distance of the two dual-divided sub-images; selecting the corresponding second splice sub-image for each of the second dual-divided sub-images according to the smoothing overhead and the data overhead.
结合第二方面的第六种可能的实现方式,在第二方面的第七种可能的实现方式中,所述选取模块还用于:根据所述第一对偶分割子图像构造第一图,所述第一图包括:多个所述第一对偶分割子图像;根据所述光滑开销和所述数据开销确定所述第一图每条边的权值;根据所述第一图每条边的权值采用图切割方法对所述第一对偶分割子图像进行分类,根据分类结果为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像;所述根据所述光滑开销和所述数据开销为每个所述第二对偶分割子图像选取对应的所述第二拼接子图像,包括:根据所述第二对偶分割子图像构造第二图,所述第二图包括:多个所述第二对偶分割子图像;根据所述光滑开销和所述数据开销确定所述第二图每条边的权值;根据所述第二图每条边的权值采用图切割方法对所述第二对偶分割子图像进行分类,根据分类结果为每个所述第二对偶分割子图像选取对应的所述第二拼接子图像。
In conjunction with the sixth possible implementation of the second aspect, in a seventh possible implementation manner of the second aspect, the selecting module is further configured to: construct a first image according to the first dual-divided sub-image, The first map includes: a plurality of the first dual-divided sub-images; determining a weight of each side of the first graph according to the smoothing overhead and the data overhead; and each edge according to the first graph The weighting is performed by using a graph cutting method to classify the first dual-divided sub-image, and selecting the corresponding first splice sub-image for each of the first dual-divided sub-images according to the classification result; And selecting, by the data overhead, the second splice sub-image corresponding to each of the second dual-divided sub-images, comprising: constructing a second map according to the second dual-divided sub-image, where the second map includes: a plurality of the second dual-divided sub-images; determining a weight of each side of the second graph according to the smoothing overhead and the data overhead; and using a graph cutting method according to the weight of each edge of the second graph For the second pair Classifying the divided sub-images, each of said second dual segmented image of the second sub-splicing corresponding selected sub-image based on the classification result.
本发明实施例提供的全景图像的获取方法和装置,通过对至少两个相机拍摄的第一图像的背景图像的第一重叠图像和前景图像的第二重叠图像进行分割,为每个第一分割子图像在至少两个第一图像的背景图像中选取对应第一拼接子图像,为每个第二分割子图像在至少两个第一图像的前景图像中选取对应的第二拼接子图像,将第一拼接子图像进行拼接得到目标背景图像,将第二拼接子图像进行拼接得到目标前景图像,最后对目标背景图像、目标前景图像、第一非重叠图像和第二非重叠图像进行合成,从而得到更大视场范围的高分辨率目标全景图像。The method and device for acquiring a panoramic image according to an embodiment of the present invention divides a first overlapping image of a background image of a first image captured by at least two cameras and a second overlapping image of a foreground image, for each first segmentation The sub-image selects a corresponding first splice sub-image in the background image of the at least two first images, and selects a corresponding second splice sub-image in the foreground image of the at least two first images for each second divided sub-image, The first spliced sub-image is spliced to obtain a target background image, the second spliced sub-image is spliced to obtain a target foreground image, and finally the target background image, the target foreground image, the first non-overlapping image and the second non-overlapping image are synthesized, thereby Get a high resolution target panoramic image with a larger field of view.
图1为本发明一实施例提供的一种全景图像的获取方法的流程图;FIG. 1 is a flowchart of a method for acquiring a panoramic image according to an embodiment of the present invention;
图2为本发明另一实施例提供的一种全景图像的获取方法的流程图;2 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention;
图3A为本发明一实施例提供的第一分割子图像示意图;3A is a schematic diagram of a first divided sub-image according to an embodiment of the present invention;
图3B为本发明另一实施例提供的第一分割子图像示意图;FIG. 3B is a schematic diagram of a first segmentation sub-image according to another embodiment of the present invention; FIG.
图4A为本发明一实施例提供的目标全景图像示意图;4A is a schematic diagram of a target panoramic image according to an embodiment of the present invention;
图4B为本发明另一实施例提供的目标全景图像示意图;FIG. 4B is a schematic diagram of a target panoramic image according to another embodiment of the present invention; FIG.
图5为本发明再一实施例提供的一种全景图像的获取方法的流程图;FIG. 5 is a flowchart of a method for acquiring a panoramic image according to still another embodiment of the present invention;
图6A为本发明再一实施例提供的图切割示意图;FIG. 6A is a schematic diagram of cutting according to still another embodiment of the present invention; FIG.
图6B为本发明再一实施例提供的图切割的分类结果示意图;6B is a schematic diagram showing a classification result of a graph cut according to still another embodiment of the present invention;
图6C为本发明又一实施例提供的目标全景图像的示意图;6C is a schematic diagram of a target panoramic image according to another embodiment of the present invention;
图7为本发明又一实施例提供的一种全景图像的获取方法的流程图;FIG. 7 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention;
图8为本发明又一实施例提供的相机阵列的示意图;FIG. 8 is a schematic diagram of a camera array according to still another embodiment of the present invention; FIG.
图9为本发明又一实施例提供的背景图像拼接示意图;FIG. 9 is a schematic diagram of a background image mosaic according to another embodiment of the present invention; FIG.
图10为本发明一实施例提供的一种全景图像的获取装置的结构示意图。FIG. 10 is a schematic structural diagram of a device for acquiring a panoramic image according to an embodiment of the present invention.
为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的
实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Is a part of the embodiment of the invention, not all
Example. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
图1为本发明一实施例提供的一种全景图像的获取方法的流程图,该方法适用于图像处理领域,该方法的执行主体可以为全景图像的获取装置,其中该装置可以为计算机等智能设备,全景图像的获取方法的具体步骤包括:FIG. 1 is a flowchart of a method for acquiring a panoramic image according to an embodiment of the present invention. The method is applicable to the field of image processing, and the execution body of the method may be a device for acquiring a panoramic image, where the device may be a computer or the like. The specific steps of the device and the method for acquiring the panoramic image include:
S101:获取至少两张第一图像。S101: Acquire at least two first images.
具体地,计算机获得至少两张第一图像,其中每张第一图像为对同一物体不同角度的拍摄图像,由于拍摄角度的不同,因此第一图像的视场范围不同,并且获得拍摄上述第一图像的相机的标定数据,其中标定数据包括相机的内参矩阵,外参矩阵,内参矩阵包括相机的分辨率、焦距等参数,外参矩阵包括相机的平移、旋转等位置参数。Specifically, the computer obtains at least two first images, wherein each of the first images is a captured image at different angles to the same object, and the field of view of the first image is different due to different shooting angles, and the first image is obtained. The calibration data of the camera of the image, wherein the calibration data includes an internal parameter matrix of the camera, an external parameter matrix, the internal reference matrix includes parameters such as resolution and focal length of the camera, and the external parameter matrix includes positional parameters such as translation and rotation of the camera.
S102:分别提取第一图像的背景图像和前景图像。S102: Extract a background image and a foreground image of the first image, respectively.
可选地,根据第一图像的深度值确定第一图像的前景模板,其中第一图像的前景模板为0,1组成的矩阵,在同一像素点处,提取前景模板1所对应的第一图像的像素点,构成第一图像的前景图像,进一步地,还可在同一像素点处,提取前景模板的补中1所对应的第一图像的像素点,构成第一图像的背景图像。Optionally, determining a foreground template of the first image according to the depth value of the first image, where the foreground template of the first image is a matrix consisting of 0, 1 , and at the same pixel point, extracting the first image corresponding to the foreground template 1 The pixel points constitute a foreground image of the first image. Further, at the same pixel point, the pixel points of the first image corresponding to the complement 1 of the foreground template may be extracted to form a background image of the first image.
具体地,对第一图像的深度值进行划分,定义一个阈值范围,深度值在阈值范围之内和在阈值范围之外分别对应数字“1”和“0”,则第一图像可以根据深度值转换为一个由“0”和“1”构成的矩阵,形成的矩阵为前景模板,在同一像素点处,提取前景模板1所对应的第一图像的像素点,构成第一图像的前景图像,进一步地,还可在同一像素点处,提取前景模板的补中1所对应的第一图像的像素点,构成第一图像的背景图像。获取第一图像的前景图像和背景图像的方法还可为其它前景、背景图像检测方法。Specifically, the depth value of the first image is divided to define a threshold range, and the depth value is within the threshold range and the numbers “1” and “0” are respectively outside the threshold range, and the first image may be according to the depth value. Converted into a matrix consisting of "0" and "1", the formed matrix is a foreground template, and at the same pixel point, the pixel of the first image corresponding to the foreground template 1 is extracted to form a foreground image of the first image. Further, at the same pixel point, the pixel points of the first image corresponding to the complement 1 of the foreground template may be extracted to form a background image of the first image. The method of acquiring the foreground image and the background image of the first image may also be other foreground and background image detecting methods.
S103:确定至少两张第一图像的背景图像形成的第一重叠图像和第一非重叠图像,确定至少两张第一图像的前景图像形成的第二重叠图像和第二非重叠图像。S103: Determine a first overlapping image and a first non-overlapping image formed by the background images of the at least two first images, and determine a second overlapping image and a second non-overlapping image formed by the foreground images of the at least two first images.
具体地,通过单应变换将每个背景图像转换为背景虚拟图像,通过
单应变换将每个前景图像转换为前景虚拟图像,根据背景虚拟图像计算各个背景图像的第一重叠图像和第一非重叠图像,然后根据前景虚拟图像计算各个前景图像的第二重叠图像和第二非重叠图像。其中单应变换的计算方式可以根据每个相机的内参矩阵、外参矩阵以及深度值确定,该深度值可以为某一第一图像的背景图像的深度值,也可以为自定义的深度值。假设背景虚拟图像所在的平面的法向量n平行于第i个相机所在坐标系的z轴,并且它与该相机之间的距离为zbg,那么该相机所拍摄的第一图像的背景图像到对应的背景虚拟图像之间的单应变换为这里d表示[001],Ki表示第i个相机的3x3内参矩阵,其中Kv可以取所有相机内参的平均值,Rv和tv可以通过对所有相机外参进行插值得到。单应变换的方法不局限与上述方法。最后,根据第一背景虚拟图像计算第一重叠图像和第一非重叠图像,同样的方法,根据第一前景虚拟图像计算第二重叠图像和第二非重叠图像。Specifically, each background image is converted into a background virtual image by a homography transformation, each foreground image is converted into a foreground virtual image by a homography transformation, and a first overlapping image and a first non-image of each background image are calculated according to the background virtual image. The images are superimposed, and then the second superimposed image and the second non-overlapping image of the respective foreground images are calculated from the foreground virtual images. The calculation method of the homography transformation may be determined according to an internal parameter matrix, an outer parameter matrix, and a depth value of each camera, and the depth value may be a depth value of a background image of a certain first image, or may be a customized depth value. Assuming that the normal vector n of the plane in which the background virtual image is located is parallel to the z-axis of the coordinate system in which the i-th camera is located, and the distance between it and the camera is z bg , then the background image of the first image captured by the camera is The correspondence between the corresponding background virtual images is converted to Where d denotes [001], K i denotes the 3x3 internal reference matrix of the i-th camera, where K v can take the average of all camera internal parameters, and R v and t v can be obtained by interpolating all camera external parameters. The method of homography transformation is not limited to the above method. Finally, the first overlapping image and the first non-overlapping image are calculated according to the first background virtual image. In the same manner, the second overlapping image and the second non-overlapping image are calculated according to the first foreground virtual image.
S104:对第一重叠图像进行分割得到多个第一分割子图像,对第二重叠图像进行分割得到多个第二分割子图像。S104: Segmenting the first superimposed image to obtain a plurality of first divided sub-images, and dividing the second superimposed image to obtain a plurality of second divided sub-images.
具体地,首先计算第一重叠图像在各个颜色通道上的第一差值,对第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值,然后根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像,同样对第二重叠图像进行分割得到多个第二分割子图像,具体包括:计算第二重叠图像在各个颜色通道上的第二差值图像,对第二差值图像依次进行拉普拉斯滤波变换和平滑处理,得到第二摩尔斯函数值,然后根据第二摩尔斯函数值确定第二重叠图像上的第二分割子图像。其中根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像具体包括:根据第一重叠图像上的像素点的第一摩尔斯函数值确定第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点,根据每一个第一局部最小值点、一个第一局部最大值点和两个第一鞍点确定一个第一分割子图像,也就是说每一个第一分割子图像均包括上述四个点。同样,根据第二摩尔斯函数值确定第二重叠图像上的第二分割子图像,具体包括:根据第二重叠图像上的像素点的第二摩尔斯函数值确定第二重叠图像上的第二局部最小值点,第二局部最大值点和第二鞍点,然后根
据每一个第二局部最小值点,一个第二局部最大值点和两个第二鞍点确定一个第二分割子图像。Specifically, first, a first difference value of the first overlapping image on each color channel is calculated, and a Laplace filter transform and a smoothing process are sequentially performed on the first difference value to obtain a first Morse function value, and then according to the first mole. The function value determines the first divided sub-image on the first overlapping image, and the second overlapping image is also divided to obtain a plurality of second divided sub-images, specifically comprising: calculating a second difference of the second overlapping image on each color channel The value image is subjected to Laplacian transform transform and smoothing processing on the second difference image to obtain a second Morse function value, and then the second divided sub-image on the second overlapping image is determined according to the second Morse function value. Determining, according to the first Morse function value, the first divided sub-image on the first overlapping image comprises: determining a first local minimum on the first overlapping image according to a first Morse function value of the pixel point on the first overlapping image a value point, a first local maximum point, and a first saddle point, and determining a first segmentation sub-image according to each of the first local minimum point, a first local maximum point, and two first saddle points, that is, each The first divided sub-images each include the above four points. Similarly, determining the second divided sub-image on the second overlapping image according to the second Morse function value, specifically: determining, according to the second Morse function value of the pixel point on the second overlapping image, the second on the second overlapping image Local minimum point, second local maximum point and second saddle point, then root
A second segmentation sub-image is determined according to each of the second local minimum points, a second local maximum point, and two second saddle points.
进一步地,由于第一分割子图像的边界可能存在与第一重叠图像无法对齐的现象,因此在根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像之后,还包括:确定每个第一分割子图像的第一最大值点,使用区域算法,根据第一最大值点确定第一重叠图像对应的第一对偶分割子图像,同样,根据第二摩尔斯函数值确定第二重叠图像上的所述第二分割子图像之后,还包括:确定每个第二分割子图像的第二最大值点,使用区域算法,根据第二最大值点确定第二重叠图像对应的第二对偶分割子图像。Further, after the boundary of the first divided sub-image may be incapable of being aligned with the first overlapping image, after determining the first divided sub-image on the first overlapping image according to the first Morse function value, the method further includes: determining a first maximum point of each first divided sub-image, using a region algorithm, determining a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point, and determining a second according to the second Morse function value After the second divided sub-image on the superimposed image, the method further includes: determining a second maximum point of each second divided sub-image, and determining, by using a region algorithm, a second corresponding to the second overlapping image according to the second maximum point Dual split sub-images.
S105:为每个第一分割子图像在至少两个第一图像的背景图像中选取对应第一拼接子图像,为每个第二分割子图像在至少两个第一图像的前景图像中选取对应的第二拼接子图像。S105: Select, for each first segmentation sub-image, a corresponding first tiled sub-image in the background images of the at least two first images, and select, in each of the foreground images of the at least two first images, for each second segmentation sub-image. The second splice image.
具体地,首先根据相连的第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第一重叠图像的图像中心到第一对偶分割子图像的距离计算数据开销,然后根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像,为每个第二分割子图像在至少两个第一图像的前景图像中选取对应的第二拼接子图像,具体包括:根据相连的第二对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第二重叠图像的图像中心到第二对偶分割子图像的距离计算数据开销,根据光滑开销和数据开销为每个第二对偶分割子图像选取对应的第二拼接子图像。Specifically, first, calculating a smoothing overhead according to an average value of maximum color differences of the connected first dual-divided sub-images, calculating a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image, and then according to the smoothing The overhead and the data overhead are selected for each of the first dual-divided sub-images, and the corresponding first splice sub-images are selected for each of the second divided sub-images in the foreground image of the at least two first images. The second splicing sub-image includes: calculating a smoothing overhead according to an average value of maximum color differences of the connected second dual-divided sub-images, and calculating a data overhead according to a distance from the image center of the second overlapping image to the second dual-divided sub-image, Corresponding second splice sub-images are selected for each second dual-divided sub-image according to smoothing overhead and data overhead.
进一步地,根据光滑开销和数据开销为每个第一对偶分割子图像选取对应的第一拼接子图像,包括:首先,根据第一对偶分割子图像构造第一图,第一图包括:多个第一对偶分割子图像,其次,根据光滑开销和数据开销确定第一图每条边的权值,最后,根据第一图每条边的权值采用图切割方法对第一对偶分割子图像进行分类,根据分类结果为每个第一对偶分割子图像选取对应的第一拼接子图像,同样地,根据光滑开销和数据开销为每个第二对偶分割子图像选取对应的第二拼接子图像,包括:根据第二对偶分割子图像构造第二图,第二图包括:多个第二对
偶分割子图像,根据光滑开销和数据开销确定第二图每条边的权值,根据第二图每条边的权值采用图切割方法对第二对偶分割子图像进行分类,根据分类结果为每个第二对偶分割子图像选取对应的第二拼接子图像。Further, the corresponding first splice sub-image is selected for each first dual-divided sub-image according to the smoothing overhead and the data overhead, including: first, constructing the first map according to the first dual-divided sub-image, the first graph includes: multiple The first dual-divided sub-image, and secondly, the weight of each edge of the first graph is determined according to the smoothing overhead and the data overhead. Finally, the first dual-divided sub-image is performed by using a graph cutting method according to the weight of each edge of the first graph. Sorting, according to the classification result, selecting a corresponding first splice sub-image for each first dual-divided sub-image, and similarly selecting a corresponding second splice sub-image for each second dual-divided sub-image according to smoothing overhead and data overhead, The method includes: constructing a second map according to the second dual segmentation sub-image, and the second image includes: a plurality of second pairs
Evenly dividing the sub-image, determining the weight of each edge of the second graph according to the smoothing overhead and the data overhead, and classifying the second dual-divided sub-image according to the weight of each edge of the second graph, according to the classification result Each second dual-divided sub-image selects a corresponding second splice sub-image.
S106:对多个第一分割子图像对应的第一拼接子图像进行拼接得到目标背景图像,对多个第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像。S106: splicing the first spliced sub-images corresponding to the plurality of first divided sub-images to obtain a target background image, and splicing the second spliced sub-images corresponding to the plurality of second divided sub-images to obtain a target foreground image.
具体地,在选取完第一拼接子图像之后,对所有的第一拼接子图像进行拼接,从而得到目标背景图像,对多个第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像。Specifically, after the first spliced sub-image is selected, all the first spliced sub-images are spliced to obtain a target background image, and the second spliced sub-image corresponding to the plurality of second divided sub-images is spliced to obtain a target foreground. image.
S107:对目标背景图像、目标前景图像、第一非重叠图像和第二非重叠图像进行合成,得到目标全景图像。S107: Synthesize the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a target panoramic image.
具体地,对目标背景图像和目标前景图像通过合成算法形成目标图像,然后对目标图像和第一非重叠图像和第二非重叠图像进行合成,得到目标全景图像。Specifically, the target image is formed by the synthesis algorithm on the target background image and the target foreground image, and then the target image and the first non-overlapping image and the second non-overlapping image are combined to obtain a target panoramic image.
本发明提供了一种全景图像的获取方法,其中通过对至少两张第一图像的背景图像的第一重叠图像和前景图像的第二重叠图像进行分割,为每个第一分割子图像在至少两个第一图像的背景图像中选取对应第一拼接子图像,为每个第二分割子图像在至少两个第一图像的前景图像中选取对应的第二拼接子图像,将第一拼接子图像进行拼接得到目标背景图像,将第二拼接子图像进行拼接得到目标前景图像,最后对目标背景图像、目标前景图像、第一非重叠图像和第二非重叠图像进行合成,从而得到更大视场范围的高分辨率目标全景图像。The present invention provides a method for acquiring a panoramic image, wherein at least a first overlapping image of a background image of at least two first images and a second overlapping image of a foreground image are segmented, at least for each of the first divided sub-images Selecting a corresponding first splice sub-image in the background image of the two first images, and selecting a corresponding second splice sub-image in the foreground image of the at least two first images for each second divided sub-image, the first splice The image is spliced to obtain a target background image, the second spliced sub-image is spliced to obtain a target foreground image, and finally the target background image, the target foreground image, the first non-overlapping image and the second non-overlapping image are combined to obtain a larger view. High resolution target panoramic image of the field range.
图2为本发明另一实施例提供的一种全景图像的获取方法的流程图,该方法可适用于图像处理技术领域,执行主体可以为计算机等智能设备,其中在上一实施例基础之上,该方法主要是对步骤S104的细化,具体包括如下步骤:FIG. 2 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention. The method may be applied to the field of image processing technology, and the execution subject may be a smart device such as a computer, where the previous embodiment is based on the previous embodiment. The method is mainly the refinement of step S104, and specifically includes the following steps:
S1041:计算第一重叠图像在各个颜色通道上的第一差值,对第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值。S1041: Calculate a first difference value of the first overlapping image on each color channel, perform a Laplace filter transformation and a smoothing process on the first difference, and obtain a first Morse function value.
具体地,计算第一重叠图像在红、黄、蓝颜色通道上的第一差值,
其中差值公式为I1和I2分别表示两张第一图像所对应的颜色函数,(x,y)表示像素点的位置坐标,i表示第i个颜色通道,通过上述的差值公式来衡量第一重叠图像的错位情况,然后对第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值。Specifically, calculating a first difference of the first overlapping image on the red, yellow, and blue color channels, wherein the difference formula is I 1 and I 2 respectively represent the color functions corresponding to the two first images, (x, y) represents the position coordinates of the pixel points, i represents the i-th color channel, and the first overlapping image is measured by the difference formula described above. The misalignment condition is then subjected to Laplacian filter transformation and smoothing processing on the first difference to obtain a first Morse function value.
S1042:根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像。S1042: Determine a first divided sub-image on the first overlapping image according to the first Morse function value.
具体地,根据第一重叠图像上的像素点的第一摩尔斯函数值确定第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点,根据每一个所述第一局部最小值点、一个第一局部最大值点和两个第一鞍点确定一个第一分割子图像,图3A为本发明一实施例提供的第一分割子图像示意图,图3B为本发明另一实施例提供的第一分割子图像示意图,如图3A和图3B所示,每四个点确定一个第一分割子图像,这四个点分别为:第一局部最小值点、一个第一局部最大值点和两个第一鞍点。其中图3A和图3B在平滑处理过程中所选择的平滑参数不相同,因此所得到的第一分割子图像的大小并不相同,例如图3A采用的平滑参数为10,图3B采用的平滑参数为20,一个更大的平滑参数—一个高斯平滑的更大的标准差会得到更大的第一分割子图像。Specifically, determining, according to a first Morse function value of a pixel point on the first overlapping image, a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image, according to each of the A partial minimum point, a first local maximum point, and two first saddle points determine a first segmentation sub-image. FIG. 3A is a schematic diagram of a first segmentation sub-image according to an embodiment of the present invention, and FIG. A first divided sub-image diagram provided by an embodiment, as shown in FIG. 3A and FIG. 3B, each of the four points defines a first divided sub-image, wherein the four points are: a first local minimum point, and a first Local maximum point and two first saddle points. In FIG. 3A and FIG. 3B, the smoothing parameters selected in the smoothing process are different, so the size of the obtained first divided sub-images is not the same. For example, the smoothing parameter used in FIG. 3A is 10, and the smoothing parameters used in FIG. 3B are used. A larger smoothing parameter of 20, a larger standard deviation of Gaussian smoothing, results in a larger first segmented sub-image.
进一步地,图4A为本发明一实施例提供的目标全景图像示意图,如图4A所示,在得到第一分割子图像之后没有再采用区域算法得到第一对偶分割子图像,因此存在第一分割子图像的边界与第一重叠图像无法对齐的情况,因此可能存在拼接缝隙,图4B为本发明另一实施例提供的目标全景图像示意图,由于根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像之后,还包括:确定每个第一分割子图像的第一最大值点,使用区域算法,根据第一最大值点确定第一重叠图像对应的第一对偶分割子图像,该方法可以解决第一分割子图像的边界与第一重叠图像无法对齐的问题,因此获得到的目标全景图像不存在拼接缝隙。Further, FIG. 4A is a schematic diagram of a target panoramic image according to an embodiment of the present invention. As shown in FIG. 4A, after obtaining the first divided sub-image, the first dual-divided sub-image is obtained by using the region algorithm, so there is a first segmentation. The boundary of the sub-image is not aligned with the first overlapping image, so there may be a splicing slot. FIG. 4B is a schematic diagram of a target panoramic image according to another embodiment of the present invention, since the first overlapping image is determined according to the first Morse function value. After the first divided sub-image, the method further includes: determining a first maximum point of each first divided sub-image, and determining, by using a region algorithm, a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point, The method can solve the problem that the boundary of the first divided sub-image cannot be aligned with the first overlapping image, and thus the obtained target panoramic image does not have a stitching gap.
进一步地,还可以计算第二重叠图像在各个颜色通道上的第二差值图像,对第二差值图像依次进行拉普拉斯滤波变换和平滑处理,得到第二摩尔斯函数值。Further, a second difference image of the second overlapping image on each color channel may be calculated, and the second difference image is sequentially subjected to Laplacian transform transform and smoothing processing to obtain a second Morse function value.
具体地,计算第二重叠图像在红、黄、绿颜色通道上的第二差值,
其中差值公式为I1和I2分别表示提供第二图像的两个相机所对应的颜色函数,(x,y)表示像素点的坐标位置,i表示第i个颜色通道,通过上述的差值公式来衡量第二重叠图像的错位情况,然后对第二差值依次进行拉普拉斯滤波变换和平滑处理,得到第二摩尔斯函数值。然后,根据第二摩尔斯函数值确定第二重叠图像上的第二分割子图像。根据第二重叠图像上的像素点的第二摩尔斯函数值确定第二重叠图像上的第二局部最小值点,第二局部最大值点和第二鞍点,根据每一个第二局部最小值点,一个第二局部最大值点和两个第二鞍点确定一个第二分割子图像。根据第二摩尔斯函数值确定第二重叠图像上的第二分割子图像之后,还包括:确定每个第二分割子图像的第二最大值点,使用区域算法,根据第二最大值点确定第二重叠图像对应的第二对偶分割子图像。Specifically, calculating a second difference of the second overlapping image on the red, yellow, and green color channels, where the difference formula is I 1 and I 2 respectively represent the color functions corresponding to the two cameras providing the second image, (x, y) represents the coordinate position of the pixel point, and i represents the i-th color channel, which is measured by the difference formula described above. The two overlapping images are misaligned, and then the second difference is sequentially subjected to Laplacian transform and smoothing to obtain a second Morse function value. Then, the second divided sub-image on the second overlapping image is determined according to the second Morse function value. Determining, according to a second Morse function value of the pixel point on the second overlapping image, a second local minimum point on the second overlapping image, a second local maximum point and a second saddle point, according to each second local minimum point A second partial maximum point and two second saddle points determine a second divided sub-image. After determining the second divided sub-image on the second overlapping image according to the second Morse function value, further comprising: determining a second maximum point of each second divided sub-image, using a region algorithm, determining according to the second maximum point The second dual segmentation sub-image corresponding to the second overlapping image.
本实施例提供了一种全景图像的获取方法,其中通过计算第一摩尔斯函数值和第二摩尔斯函数值,从而确定第一重叠图像上的第一分割子图像和第二重叠图像上的第二分割子图像,进一步地,采用区域算法对第一重叠图像,第二重叠图像进行二次分割,从而实现对第一重叠图像和第二重叠图像的对齐分割,再通过拼接方法进而得到更大视场范围的高分辨率目标全景图像。The embodiment provides a method for acquiring a panoramic image, wherein the first and second overlapping images on the first overlapping image are determined by calculating a first Morse function value and a second Morse function value. The second divided sub-image is further divided into a first overlapping image and a second overlapping image by using a region algorithm, so that the first overlapping image and the second overlapping image are aligned and segmented, and then the stitching method is further obtained. High resolution target panoramic image of large field of view.
图5为本发明再一实施例提供的一种全景图像的获取方法的流程图,该方法可适用于图像处理技术领域,执行主体可以为计算机等智能设备,其中在实施例一的基础之上,该方法主要是对步骤S105的细化,具体包括如下步骤:FIG. 5 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention. The method may be applied to the field of image processing technology, and the execution subject may be a smart device such as a computer, where the first embodiment is based on the first embodiment. The method is mainly for the refinement of step S105, and specifically includes the following steps:
S1051:根据相连的第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第一重叠图像的图像中心到第一对偶分割子图像的距离计算数据开销。S1051: Calculate a smoothing overhead according to an average value of maximum color differences of the connected first dual-divided sub-images, and calculate a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image.
具体地,对相连的第一对偶分割子图像中的每个像素点在红黄蓝三个颜色通道上求颜色差,该颜色差表示两个相机所拍摄的第一背景虚拟图像在每一个像素点上的颜色差,获取最大颜色差,最后对三个颜色通道上的最大颜色差求平均值,根据平均值确定光滑开销,另外,根据第一重叠图像的图像中心到第一对偶分割子图像的距离计算数据开销。
Specifically, for each pixel point in the connected first dual-divided sub-image, a color difference is obtained on the red, yellow and blue color channels, and the color difference indicates that the first background virtual image captured by the two cameras is in each pixel. The color difference on the point, the maximum color difference is obtained, and finally the maximum color difference on the three color channels is averaged, the smoothing overhead is determined according to the average value, and, according to the image center of the first overlapping image to the first dual segmentation sub-image The distance calculates the data overhead.
S1052:根据光滑开销和数据开销为每个第一对偶分割子图像选取对应的第一拼接子图像。S1052: Select a corresponding first splice sub-image for each first dual-divided sub-image according to the smoothing overhead and the data overhead.
可选地,根据第一对偶分割子图像构造第一图,根据光滑开销和数据开销确定第一图每条边的权值,根据第一图每条边的权值采用图切割方法对第一对偶分割子图像进行分类,根据分类结果为每个第一对偶分割子图像选取对应的第一拼接子图像。例如:假设存在两个相机TL和TR,则采用图切割方法确定第一对偶分割子图像的切割线,由于第一图中除了包括第一对偶切割片,还包括终端节点,该终端节点用来表示某一相机,图6A为本发明再一实施例提供的图切割示意图,节点s和节点t分别表示两个相机,若某一第一对偶分割子图像与节点s在切割线的同一侧,则为该第一对偶分割子图像选节点s对应的相机拍摄的第一背景图像中相应的区域图像为第一拼接子图像,图6B为本发明再一实施例提供的图切割的分类结果示意图,子图像601由属于相机TL拍摄的第一图像所形成的第一分割子图像构成的,子图像602由属于相机TR拍摄的第一图像所形成的第一分割子图像构成的,根据该分割结果可以选择第一拼接子图像,例如子图像601中的某一个第一分割子图像对应的第一拼接子图像取自于TL拍摄的第一图像中同一位置的部分图像,对第一拼接子图像进行拼接最终形成目标背景图像,同样的方法可以选择第二拼接子图像,最终形成目标背景图像,图6C为本发明又一实施例提供的目标全景图像的示意图,如图6C所示,它是经过将对目标背景图像、目标前景图像、第一非重叠图像和第二非重叠图像进行合成,得到目标全景图像。Optionally, constructing the first map according to the first dual-divided sub-image, determining the weight of each edge of the first graph according to the smoothing overhead and the data overhead, and using the graph cutting method to the first according to the weight of each edge of the first graph The dual-divided sub-images are classified, and corresponding first splice sub-images are selected for each first dual-divided sub-image according to the classification result. For example, if there are two cameras TL and TR, the cutting line of the first dual-divided sub-image is determined by the graph cutting method. Since the first graph includes the first dual-cut slice, the terminal node is further included, and the terminal node is used for FIG. 6A is a schematic diagram of a graph cut according to still another embodiment of the present invention. The node s and the node t respectively represent two cameras. If a first dual split sub-image and the node s are on the same side of the cutting line, The corresponding area image in the first background image captured by the camera corresponding to the first dual-divided sub-image selection node s is the first splicing sub-image, and FIG. 6B is a schematic diagram of the classification result of the dicing cut according to another embodiment of the present invention. The sub-image 601 is composed of a first divided sub-image formed by a first image captured by the camera TL, and the sub-image 602 is composed of a first divided sub-image formed by the first image captured by the camera TR, according to the segmentation As a result, the first splice sub-image may be selected. For example, the first splice sub-image corresponding to one of the first divided sub-images in the sub-image 601 is taken from the first image captured by the TL. Part of the image of the position, the first stitching sub-image is stitched to form a target background image, and the second stitching sub-image can be selected in the same manner to finally form a target background image. FIG. 6C is a target panoramic image according to another embodiment of the present invention. The schematic diagram, as shown in FIG. 6C, is to obtain a target panoramic image by synthesizing the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image.
进一步地,还可以根据相连的第二对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第二重叠图像的图像中心到第二对偶分割子图像的距离计算数据开销。Further, the smoothing overhead may also be calculated according to the average of the maximum color differences of the connected second dual-divided sub-images, and the data overhead is calculated according to the distance from the image center of the second overlapping image to the second dual-divided sub-image.
具体地,对相连的第二对偶分割子图像中的每个像素点在红黄绿三个颜色通道上求颜色差,该颜色差表示两个相机所拍摄的第一前景虚拟图像在每一个像素点上的颜色差,获取最大颜色差,最后对三个颜色通道上的最大颜色差求平均值,根据平均值确定光滑开销,另外,根据第二重叠图像的图像中心到第二对偶分割子图像的距离计算数据开销。然后,根据光滑开销和数据开销为每个第二对偶分割子图像选取对应的第
二拼接子图像。可选地,根据第二对偶分割子图像构造第二图,根据光滑开销和数据开销确定第二图每条边的权值,根据第二图每条边的权值采用图切割方法对第二对偶分割子图像进行分类,根据分类结果为每个第二对偶分割子图像选取对应的第二拼接子图像。Specifically, for each pixel point in the connected second dual-divided sub-image, a color difference is obtained on the red, yellow and green color channels, and the color difference indicates that the first foreground virtual image captured by the two cameras is in each pixel. The color difference at the point, the maximum color difference is obtained, and finally the maximum color difference on the three color channels is averaged, the smoothing overhead is determined according to the average value, and, according to the image center of the second overlapping image to the second dual segmentation sub-image The distance calculates the data overhead. Then, according to the smoothing overhead and the data overhead, the corresponding first is selected for each second dual-divided sub-image.
Two stitched sub-images. Optionally, constructing a second map according to the second dual-divided sub-image, determining a weight of each edge of the second graph according to the smoothing overhead and the data overhead, and using the graph cutting method to the second according to the weight of each edge of the second graph The dual-divided sub-images are classified, and corresponding second splice sub-images are selected for each second dual-divided sub-image according to the classification result.
本实施例提供了一种全景图像的获取方法,其中主要包括:通过计算光滑开销和数据开销,从而为每个第一对偶分割子图像选取对应的第一拼接子图像和每个第二对偶分割子图像选取对应的第二拼接子图像,在上述图切割方法的基础之上实现有效拼接,从而得到更大视场范围的高分辨率目标全景图像。The embodiment provides a method for acquiring a panoramic image, which mainly includes: selecting a corresponding first splice sub-image and each second dual split for each first dual-divided sub-image by calculating a smooth overhead and a data overhead. The sub-image selects the corresponding second splice sub-image, and achieves effective splicing on the basis of the above-described graph cutting method, thereby obtaining a high-resolution target panoramic image with a larger field of view range.
图7为本发明又一实施例提供的一种全景图像的获取方法的流程图,该方法可适用于图像处理技术领域,执行主体可以为计算机等智能设备,在上述实施例的基础上,本实施例举出实例说明全景图像的获取方法,图8为本发明又一实施例提供的相机阵列的示意图,假设存在四个相机TL、TR、BL和BR组成2*2的相机阵列,对同一物体进行拍摄,它们的拍摄角度不相同,该深度值可以为某一第一图像的背景图像的深度值,也可以为自定义的深度值,全景图像的获取方法具体包括如下步骤:FIG. 7 is a flowchart of a method for acquiring a panoramic image according to another embodiment of the present invention. The method may be applied to the technical field of image processing, and the execution subject may be a smart device such as a computer. On the basis of the foregoing embodiment, The embodiment exemplifies a method for acquiring a panoramic image. FIG. 8 is a schematic diagram of a camera array according to another embodiment of the present invention. It is assumed that there are four cameras TL, TR, BL, and BR, and the camera array of 2*2 is the same. The object is photographed, and the shooting angles thereof are different. The depth value may be a depth value of the background image of a certain first image, or may be a customized depth value. The method for acquiring the panoramic image specifically includes the following steps:
S701:获取四张第一图像和对应的四个前景模板。S701: Acquire four first images and corresponding four foreground templates.
具体地,计算机获得四张第一图像,其中这四张第一图像分别由四个相机TL、TR、BL和BR组成2*2的相机阵列对同一物体所拍摄的图像,它们的拍摄角度不相同,因此第一图像的视场范围不同,并且获得拍摄上述第一图像的相机的标定数据,其中标定数据包括相机的内参矩阵,外参矩阵,内参矩阵包括相机的分辨率、焦距等参数,外参矩阵包括相机的平移、旋转等位置参数。对第一图像的深度值进行划分,定义一个阈值范围,深度值在阈值范围之内和在阈值范围之外分别对应二进制“1”和“0”,则第一图像可以根据深度值转换为一个由“0”和“1”构成的矩阵Specifically, the computer obtains four first images, wherein the four first images are respectively composed of four cameras TL, TR, BL, and BR, and the camera array of 2*2 is taken by the same object, and the shooting angle thereof is not The same, so the field of view of the first image is different, and the calibration data of the camera for capturing the first image is obtained, wherein the calibration data includes an internal parameter matrix of the camera, an external parameter matrix, and the internal reference matrix includes parameters such as resolution and focal length of the camera. The outer parameter matrix includes positional parameters such as translation, rotation, and the like of the camera. Dividing the depth value of the first image to define a threshold range, where the depth value is within the threshold range and outside the threshold range respectively correspond to binary "1" and "0", then the first image may be converted into one according to the depth value a matrix consisting of "0" and "1"
S702:采用前景模板提取每张第一图像的前景图像和背景图像。S702: Extract a foreground image and a background image of each first image by using a foreground template.
具体地,对第一图像的深度值进行划分,定义一个阈值范围,深度值在阈值范围之内和在阈值范围之外分别对应二进制“1”和“0”,则
第一图像可以根据深度值转换为一个由“0”和“1”构成的矩阵,形成的矩阵为前景模板,在同一像素点处,提取前景模板1所对应的第一图像的像素点,构成第一图像的前景图像,进一步地,还可在同一像素点处,提取前景模板的补中1所对应的第一图像的像素点,构成第一图像的背景图像,其中获取第一图像的前景图像和背景图像的方法还可为其它前景、背景图像检测方法。Specifically, the depth value of the first image is divided to define a threshold range, where the depth value is within the threshold range and outside the threshold range respectively correspond to binary “1” and “0”, then
The first image may be converted into a matrix consisting of “0” and “1” according to the depth value, and the formed matrix is a foreground template, and at the same pixel point, the pixel of the first image corresponding to the foreground template 1 is extracted, and the pixel is formed. a foreground image of the first image, and further, at the same pixel point, extracting pixel points of the first image corresponding to the complement 1 of the foreground template to form a background image of the first image, wherein the foreground of the first image is acquired The image and background image methods can also be other foreground, background image detection methods.
S703:确定背景图像的深度值,以及背景图像到虚拟背景图像的单应变换。S703: Determine a depth value of the background image, and a homography transformation of the background image to the virtual background image.
确定深度值方法可以为以某一第一图像的背景图像的深度值作为背景图像的深度值,也可以为自定义的深度值。另外,单应变换的确定方法为:假设背景虚拟图像所在的平面的法向量n平行于第i个相机所在坐标系的z轴,并且它与该相机之间的距离为zbg,那么该相机所拍摄的第一图像的背景图像到对应的第一背景虚拟图像之间的单应变换为这里d表示[001],Ki表示第i个相机的3x3内参矩阵,其中Kv可以取所有相机内参的平均值,Rv和tv可以通过对所有相机外参进行插值得到。单应变换的方法不局限与上述方法。The method for determining the depth value may be a depth value of the background image of a certain first image as a depth value of the background image, or may be a customized depth value. In addition, the homography transformation is determined by: assuming that the normal vector n of the plane in which the background virtual image is located is parallel to the z-axis of the coordinate system of the i-th camera, and the distance between the camera and the camera is z bg , then the camera The homography between the captured background image of the first image and the corresponding first background virtual image is converted to Where d denotes [001], K i denotes the 3x3 internal reference matrix of the i-th camera, where K v can take the average of all camera internal parameters, and R v and t v can be obtained by interpolating all camera external parameters. The method of homography transformation is not limited to the above method.
S704:基于相机TL、TR分别拍摄的第一图像的第一重叠图像进行分割,并基于分割结果进行背景图像的拼接,选取第一拼接子图像,形成目标背景图像。S704: Perform segmentation based on the first overlapping image of the first image captured by the cameras TL and TR respectively, and perform splicing of the background image based on the segmentation result, and select the first spliced sub-image to form a target background image.
具体地,首先计算第一重叠图像在各个颜色通道上的第一差值,对第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值,然后根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像,其中根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像具体包括:根据第一重叠图像上的像素点的第一摩尔斯函数值确定第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点,根据每一个第一局部最小值点、一个第一局部最大值点和两个第一鞍点确定一个第一分割子图像,也就是说每一个第一分割子图像均包括上述四个点。进一步地,由于第一分割子图像的边界可能存在与第一重叠图像无法对齐的现象,因此在根据第一摩尔斯函数值确定第一重叠图像上的第一分割子图像之后,还包括:确定每个第一分割子图像的第一最大值
点,使用区域算法,根据第一最大值点确定第一重叠图像对应的第一对偶分割子图像。然后,为每个第一分割子图像在至少两个第一图像的背景图像中选取对应第一拼接子图像,形成目标背景图像,包括:根据相连的第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第一重叠图像的图像中心到第一对偶分割子图像的距离计算数据开销,然后根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像,Specifically, first, a first difference value of the first overlapping image on each color channel is calculated, and a Laplace filter transform and a smoothing process are sequentially performed on the first difference value to obtain a first Morse function value, and then according to the first mole. The sigma function value determines a first segmentation sub-image on the first superimposed image, wherein determining the first segmentation sub-image on the first superimposed image according to the first Morse function value specifically includes: according to the pixel point on the first superimposed image a Morse function value determining a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image, according to each of the first local minimum point, a first local maximum point, and two The first saddle point determines a first segmentation sub-image, that is to say each of the first segmentation sub-images includes the above four points. Further, after the boundary of the first divided sub-image may be incapable of being aligned with the first overlapping image, after determining the first divided sub-image on the first overlapping image according to the first Morse function value, the method further includes: determining First maximum of each first segmented sub-image
Point, using a region algorithm, determining a first dual segmentation sub-image corresponding to the first overlapping image according to the first maximum point. Then, selecting a corresponding first splice sub-image in the background images of the at least two first images for each of the first divided sub-images, forming a target background image, including: according to a maximum color difference of the connected first dual-divided sub-images Calculating a smoothing overhead by an average value, calculating a data overhead according to a distance from the image center of the first overlapping image to the first dual-divided sub-image, and then selecting, for each of the first dual-divided sub-images, according to the smoothing overhead and the data overhead Corresponding to the first splice sub-image,
进一步地,根据光滑开销和数据开销为每个第一对偶分割子图像选取对应的第一拼接子图像,包括:首先,根据第一对偶分割子图像构造第一图,第一图包括:多个第一对偶分割子图像,其次,根据光滑开销和数据开销确定第一图每条边的权值,最后,根据第一图每条边的权值采用图切割方法对第一对偶分割子图像进行分类,根据分类结果为每个第一对偶分割子图像选取对应的第一拼接子图像。最后,对上述的第一拼接子图像进行拼接形成目标背景图像。Further, the corresponding first splice sub-image is selected for each first dual-divided sub-image according to the smoothing overhead and the data overhead, including: first, constructing the first map according to the first dual-divided sub-image, the first graph includes: multiple The first dual-divided sub-image, and secondly, the weight of each edge of the first graph is determined according to the smoothing overhead and the data overhead. Finally, the first dual-divided sub-image is performed by using a graph cutting method according to the weight of each edge of the first graph. Classification, selecting a corresponding first splice sub-image for each first dual-divided sub-image according to the classification result. Finally, the first splice sub-images described above are spliced to form a target background image.
S705:基于相机BL、BR分别拍摄的第一图像的第一重叠图像进行分割,并基于分割结果进行背景图像的拼接,选取第一拼接子图像,形成目标背景图像。S705: Perform segmentation based on the first overlapping image of the first image captured by the cameras BL and BR respectively, and perform splicing of the background image based on the segmentation result, and select the first spliced sub-image to form a target background image.
该步骤与S704类似,在此不再赘述。This step is similar to S704 and will not be described here.
S706:对TL、TR所拍摄的第一图像形成的目标背景图像以及BL、BR所拍摄的第一图像形成的目标背景图像的重叠图像进行分割,并基于分割结果形成最终的目标背景图像。S706: segment the superimposed image of the target background image formed by the first image captured by the TL and the TR and the target background image formed by the first image captured by the BL and the BR, and form a final target background image based on the segmentation result.
具体地,图9为本发明又一实施例提供的背景图像拼接示意图,如图9所示,第一行图像分别为相机TL、TR、BL和BR所拍摄的第一图像,第二行左边的图像是TL、TR所拍摄的第一图像形成的目标背景图像,右侧的图像是BL、BR所拍摄的第一图像形成的目标背景图像,第三行图像则为对TL、TR所拍摄的第一图像形成的目标背景图像以及BL、BR所拍摄的第一图像形成的目标背景图像的重叠图像进行分割,并基于分割结果形成最终的目标背景图像。Specifically, FIG. 9 is a schematic diagram of background image mosaic according to another embodiment of the present invention. As shown in FIG. 9, the first line of images is the first image captured by cameras TL, TR, BL, and BR, and the second line is left. The image is the target background image formed by the first image captured by TL and TR, the image on the right is the target background image formed by the first image captured by BL and BR, and the image on the third line is taken on TL and TR. The target background image formed by the first image and the overlapping image of the target background image formed by the first image captured by the BL and the BR are segmented, and a final target background image is formed based on the segmentation result.
S707:确定前景图像的深度值,以及前景图像到虚拟前景图像的单应变换。
S707: Determine a depth value of the foreground image, and a homography transformation of the foreground image to the virtual foreground image.
S708:基于相机TL、TR分别拍摄的第一图像的第二重叠图像进行分割,并基于分割结果进行前景图像的拼接,形成目标前景图像。S708: Perform segmentation based on the second overlapping image of the first image captured by the cameras TL and TR respectively, and perform splicing of the foreground image based on the segmentation result to form a target foreground image.
由于前景图像的拼接并不能显著增加视场的大小,因此可以只对上面任意两个相机所拍摄的第一图像的前景图像进行拼接即可。Since the splicing of the foreground image does not significantly increase the size of the field of view, it is possible to splicing only the foreground image of the first image taken by any two of the above cameras.
S709:根据上述第一拼接子图像和第二拼接子图像确定目标全景图像。S709: Determine a target panoramic image according to the first splice sub image and the second splice sub image.
对上述的目标背景图像、目标前景图像、以及第一图像的背景图像所确定的第一非重叠图像、第一图像的前景图形所确定的第二非重叠图像进行合成,最后得到目标全景图像。如图9所示的背景图像拼接过程得到的目标背景图像,同时类似于图9所示的背景图像拼接过程的前景拼接过程,得到的目标前景图像,并且同时对第一图像的背景图像所确定的第一非重叠图像、第一图像的前景图形所确定的第二非重叠图像进行合成,最后得到目标全景图像。The target non-overlapping image determined by the target background image, the target foreground image, and the background image of the first image, and the second non-overlapping image determined by the foreground image of the first image are synthesized, and finally the target panoramic image is obtained. The target background image obtained by the background image stitching process shown in FIG. 9 is similar to the foreground stitching process of the background image stitching process shown in FIG. 9, the obtained target foreground image, and simultaneously determined for the background image of the first image. The first non-overlapping image, the second non-overlapping image determined by the foreground image of the first image are synthesized, and finally the target panoramic image is obtained.
本发明提供了一种全景图像的获取方法,其中包括对第一重叠图像进行分割,形成第一分割子图像,同样对第二重叠图像进行分割,形成第二分割子图像,为每个第一分割子图像选取对应第一拼接子图像,为每个第二分割子图像选取对应的第二拼接子图像,将第一拼接子图像进行拼接得到目标背景图像,将第二拼接子图像进行拼接得到目标前景图像,最后对目标背景图像、目标前景图像、第一非重叠图像和第二非重叠图像进行合成,从而得到更大视场范围的高分辨率目标全景图像。The present invention provides a method for acquiring a panoramic image, which includes dividing a first overlapping image to form a first divided sub-image, and equally dividing the second overlapping image to form a second divided sub-image for each first The segmented sub-image is selected corresponding to the first spliced sub-image, and the corresponding second spliced sub-image is selected for each second sub-sub-image, the first spliced sub-image is spliced to obtain the target background image, and the second spliced sub-image is spliced The target foreground image is finally combined with the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a high-resolution target panoramic image of a larger field of view.
图10为本发明一实施例提供的一种全景图像的获取装置的结构示意图,该装置可以为计算机等智能设备,其中全景图像的获取装置包括:获取模块111,用于获取至少两张第一图像;提取模块112,用于提取第一图像的背景图像和前景图像;确定模块113,用于确定至少两张第一图像的背景图像形成的第一重叠图像和第一非重叠图像,以及确定至少两张第一图像的前景图像形成的第二重叠图像和第二非重叠图像;其中提取模块112具体用于:根据第一图像的深度值确定第一图像的前景模板;第一前景模板为0,1组成的矩阵,在同一像素点处,提取前景模板1所对应的第一图像的像素点,构成第一图像的前景图像;在同一像素点处,提取前景模板的补中1所对应的第一图像的像素点,构成第一图像的背景
图像。确定模块113具体用于:通过单应转换将每个第一背景图像转换为第一背景虚拟图像,通过单应转换将每个第一前景图像转换为第一前景虚拟图像;根据第一背景虚拟图像计算各个第一背景图像的第一重叠图像和第一非重叠图像,根据第一前景虚拟图像计算各个第一前景图像的第二重叠图像和第二非重叠图像。分割模块114,用于对第一重叠图像进行分割得到多个第一分割子图像,对第二重叠图像进行分割得到多个第二分割子图像;选取模块115,用于为每个第一分割子图像在至少两个第一图像的背景图像中选取对应第一拼接子图像,为每个第二分割子图像在至少两个第一图像的前景图像中选取对应的第二拼接子图像;拼接模块116,用于对多个第一分割子图像对应的第一拼接子图像进行拼接得到目标背景图像,对多个第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像;合成模块117,用于对目标背景图像、目标前景图像、第一非重叠图像和第二非重叠图像进行合成,得到目标全景图像。FIG. 10 is a schematic structural diagram of a device for acquiring a panoramic image according to an embodiment of the present invention. The device may be a smart device such as a computer. The device for acquiring a panoramic image includes: an acquiring module 111, configured to acquire at least two first An image extraction module 112, configured to extract a background image and a foreground image of the first image; a determining module 113, configured to determine a first overlapping image and a first non-overlapping image formed by the background images of the at least two first images, and determine a second overlapping image and a second non-overlapping image formed by the foreground image of the at least two first images; wherein the extracting module 112 is configured to: determine a foreground template of the first image according to the depth value of the first image; the first foreground template is a matrix consisting of 0,1, at the same pixel point, extracting pixel points of the first image corresponding to the foreground template 1 to form a foreground image of the first image; at the same pixel point, extracting the complement 1 of the foreground template The pixel of the first image, which constitutes the background of the first image
image. The determining module 113 is specifically configured to: convert each first background image into a first background virtual image by a homography conversion, convert each first foreground image into a first foreground virtual image by a homography conversion; The image calculates a first overlap image and a first non-overlapping image of each of the first background images, and calculates a second overlap image and a second non-overlapping image of each of the first foreground images according to the first foreground virtual image. The segmentation module 114 is configured to segment the first overlapping image to obtain a plurality of first divided sub-images, and divide the second overlapping image to obtain a plurality of second divided sub-images; and the selecting module 115 is configured to The sub-image selects a corresponding first splice sub-image in the background images of the at least two first images, and selects a corresponding second splice image in the foreground image of the at least two first images for each second divided sub-image; The module 116 is configured to splicing a first spliced sub-image corresponding to the plurality of first divided sub-images to obtain a target background image, and splicing the second spliced sub-image corresponding to the plurality of second divided sub-images to obtain a target foreground image; The module 117 is configured to synthesize the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a target panoramic image.
本实施例的全景图像的获取装置,可以用于执行图1所示方法实施例的技术方案,其实现原理和技术效果类似,此处不再赘述。The device for acquiring the panoramic image of the embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 1 , and the implementation principle and technical effects thereof are similar, and details are not described herein again.
在上一实施例的基础之上,进一步地,分割模块114具体用于:计算第一重叠图像在各个颜色通道上的第一差值,对第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值;根据所述第一摩尔斯函数值确定第一重叠图像上的第一分割子图像;还用于计算第二重叠图像在各个颜色通道上的第二差值图像,对第二差值图像依次进行拉普拉斯滤波变换和平滑处理,得到第二摩尔斯函数值;根据第二摩尔斯函数值确定第二重叠图像上的第二分割子图像。进一步地,分割模块114还用于:根据第一重叠图像上的像素点的第一摩尔斯函数值确定第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点;根据每一个第一局部最小值点、一个第一局部最大值点和两个第一鞍点确定一个第一分割子图像;还用于根据第二重叠图像上的像素点的第二摩尔斯函数值确定第二重叠图像上的第二局部最小值点,第二局部最大值点和第二鞍点;根据每一个第二局部最小值点,一个第二局部最大值点和两个第二鞍点确定一个第二分割子图像。分割模块114还用于:确定每个第一分割子图像的第一最大值点,使用区域算法,根据第一最大值点确定
第一重叠图像对应的第一对偶分割子图像;确定每个第二分割子图像的第二最大值点,使用区域算法,根据第二最大值点确定第二重叠图像对应的第二对偶分割子图像。On the basis of the previous embodiment, the segmentation module 114 is specifically configured to: calculate a first difference value of the first overlapping image on each color channel, and perform a Laplace filter transformation on the first difference sequentially. Smoothing to obtain a first Morse function value; determining a first segmentation sub-image on the first overlay image according to the first Morse function value; and further for calculating a second difference of the second overlay image on each color channel The value image is subjected to Laplacian transform transform and smoothing processing on the second difference image to obtain a second Morse function value; and the second divided sub-image on the second superimposed image is determined according to the second Morse function value. Further, the segmentation module 114 is further configured to: determine, according to a first Morse function value of the pixel point on the first overlapping image, a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image Determining a first segmentation sub-image according to each of the first local minimum point, a first local maximum point, and the two first saddle points; and further for using a second Morse function of the pixel points on the second overlapping image The value determines a second local minimum point on the second overlapping image, a second local maximum point and a second saddle point; and a second local maximum point and two second saddle points are determined according to each of the second local minimum points A second segmented sub-image. The segmentation module 114 is further configured to: determine a first maximum point of each first segmentation sub-image, and determine, according to the first maximum point, using a region algorithm
Determining a first dual-divided sub-image corresponding to the first overlapping image; determining a second maximum point of each second divided sub-image, determining a second dual-divided sub-correlation corresponding to the second overlapping image according to the second maximum point using a region algorithm image.
本实施例的全景图像的获取装置,可以用于执行图2所示方法实施例的技术方案,其实现原理和技术效果类似,此处不再赘述。The device for acquiring the panoramic image in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 2, and the implementation principle and technical effects are similar, and details are not described herein again.
进一步地,在图10所示装置实施例的基础之上,选取模块115具体用于:根据相连的第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第一重叠图像的图像中心到第一对偶分割子图像的距离计算数据开销;根据光滑开销和数据开销为每个第一对偶分割子图像选取对应的第一拼接子图像;根据相连的第二对偶分割子图像的最大颜色差的平均值计算光滑开销,根据第二重叠图像的图像中心到第二对偶分割子图像的距离计算数据开销;根据光滑开销和数据开销为每个第二对偶分割子图像选取对应的第二拼接子图像。此外,选取模块115还用于:根据第一对偶分割子图像构造第一图,第一图包括:多个第一对偶分割子图像;根据光滑开销和数据开销确定第一图每条边的权值;根据第一图每条边的权值采用图切割方法对第一对偶分割子图像进行分类,根据分类结果为每个第一对偶分割子图像选取对应的第一拼接子图像;根据光滑开销和数据开销为每个第二对偶分割子图像选取对应的第二拼接子图像,包括:根据第二对偶分割子图像构造第二图,第二图包括:多个第二对偶分割子图像;根据光滑开销和数据开销确定第二图每条边的权值;根据第二图每条边的权值采用图切割方法对第二对偶分割子图像进行分类,根据分类结果为每个第二对偶分割子图像选取对应的第二拼接子图像。Further, based on the apparatus embodiment shown in FIG. 10, the selecting module 115 is specifically configured to: calculate a smoothing overhead according to an average value of maximum color differences of the connected first dual-divided sub-images, according to the image of the first overlapping image Calculating a data overhead from a distance from the center to the first dual-divided sub-image; selecting a corresponding first splice sub-image for each first dual-divided sub-image according to smoothing overhead and data overhead; according to a maximum color of the connected second dual-divided sub-image The average value of the difference is used to calculate a smoothing overhead, and the data overhead is calculated according to the distance from the image center of the second overlapping image to the second dual-divided sub-image; and the second stitching is selected for each second dual-divided sub-image according to the smoothing overhead and the data overhead. Sub image. In addition, the selecting module 115 is further configured to: construct a first map according to the first dual-divided sub-image, where the first image includes: a plurality of first dual-divided sub-images; and determining, according to the smoothing overhead and the data overhead, the right of each side of the first graph The first dual-divided sub-image is classified according to the weight of each edge of the first graph by using a graph cutting method, and the corresponding first splice sub-image is selected for each first dual-divided sub-image according to the classification result; according to the smoothing overhead And the data overhead is: selecting a second second sub-segment image for each second dual-divided sub-image, comprising: constructing a second image according to the second dual-divided sub-image, the second image comprising: a plurality of second dual-divided sub-images; Smoothing overhead and data overhead determine the weight of each edge of the second graph; according to the weight of each edge of the second graph, the second dual-divided sub-image is classified by a graph cutting method, and each second dual partition is segmented according to the classification result. The sub image selects a corresponding second splice sub image.
本实施例的全景图像的获取装置,可以用于执行图5所示方法实施例的技术方案,其实现原理和技术效果类似,此处不再赘述。The apparatus for acquiring the panoramic image of the embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 5, and the implementation principle and technical effects are similar, and details are not described herein again.
最后应说明的是:以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。
It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not limited thereto; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (16)
- 一种全景图像的获取方法,其特征在于,包括:A method for acquiring a panoramic image, comprising:获取至少两张第一图像;Obtaining at least two first images;分别提取所述第一图像的背景图像和前景图像;Extracting a background image and a foreground image of the first image, respectively;确定所述至少两张第一图像的背景图像形成的第一重叠图像和第一非重叠图像;Determining a first overlapping image and a first non-overlapping image formed by the background images of the at least two first images;确定所述至少两张第一图像的前景图像形成的第二重叠图像和第二非重叠图像;Determining a second overlapping image and a second non-overlapping image formed by the foreground image of the at least two first images;对所述第一重叠图像进行分割得到多个第一分割子图像,对所述第二重叠图像进行分割得到多个第二分割子图像;Dividing the first overlapping image to obtain a plurality of first divided sub-images, and dividing the second overlapping image to obtain a plurality of second divided sub-images;为每个所述第一分割子图像在所述至少两张第一图像的背景图像中选取对应第一拼接子图像,为每个所述第二分割子图像在所述至少两张第一图像的前景图像中选取对应的第二拼接子图像;Selecting, for each of the first divided sub-images, a corresponding first splice sub-image in the background images of the at least two first images, and the at least two first images for each of the second divided sub-images Selecting a corresponding second splice image in the foreground image;对多个所述第一分割子图像对应的第一拼接子图像进行拼接得到目标背景图像,对多个所述第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像;And splicing the first spliced sub-images corresponding to the plurality of the first sub-sub-images to obtain a target background image, and splicing the second spliced sub-images corresponding to the plurality of the second sub-sub-images to obtain a target foreground image;对所述目标背景图像、所述目标前景图像、所述第一非重叠图像和所述第二非重叠图像进行合成,得到目标全景图像。The target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image are combined to obtain a target panoramic image.
- 根据权利要求1所述的方法,其特征在于,所述分别提取所述第一图像的背景图像和前景图像,具体包括:The method according to claim 1, wherein the extracting the background image and the foreground image of the first image respectively comprises:根据所述第一图像的深度值确定所述第一图像的前景模板;Determining a foreground template of the first image according to a depth value of the first image;所述第一图像的前景模板为0,1组成的矩阵,在同一像素点处,提取前景模板1所对应的所述第一图像的像素点,构成所述第一图像的前景图像;The foreground template of the first image is a matrix composed of 0,1, and at the same pixel point, the pixel points of the first image corresponding to the foreground template 1 are extracted to form a foreground image of the first image;在同一像素点处,提取前景模板的补中1所对应的所述第一图像的像素点,构成所述第一图像的背景图像。At the same pixel point, the pixel points of the first image corresponding to the complement 1 of the foreground template are extracted to form a background image of the first image.
- 根据权利要求1或2所述的方法,其特征在于,所述确定所述至少两张第一图像形成的背景图像的第一重叠图像和第一非重叠图像;确定所述至少两张第一图像的前景图像形成的第二重叠图像和第二非重叠图像,具体包括: The method according to claim 1 or 2, wherein the determining the first overlapping image and the first non-overlapping image of the background image formed by the at least two first images; determining the at least two first The second overlapping image and the second non-overlapping image formed by the foreground image of the image specifically include:通过单应变换将每个所述背景图像转换为背景虚拟图像,通过所述单应变换将每个所述前景图像转换为前景虚拟图像;Converting each of the background images into a background virtual image by a homography transformation, converting each of the foreground images into a foreground virtual image by the homography transformation;根据所述背景虚拟图像计算各个所述背景图像的所述第一重叠图像和所述第一非重叠图像,根据所述前景虚拟图像计算各个所述前景图像的所述第二重叠图像和所述第二非重叠图像。Calculating the first overlapping image and the first non-overlapping image of each of the background images according to the background virtual image, calculating the second overlapping image of each of the foreground images according to the foreground virtual image, and the The second non-overlapping image.
- 根据权利要求1-3任一项所述的方法,其特征在于,所述对所述第一重叠图像进行分割得到多个第一分割子图像,包括:The method according to any one of claims 1-3, wherein the segmenting the first overlapping image to obtain a plurality of first divided sub-images comprises:计算所述第一重叠图像在各个颜色通道上的第一差值,对所述第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值;Calculating a first difference value of the first overlapping image on each color channel, performing Laplacian filtering transformation and smoothing processing on the first difference, to obtain a first Morse function value;根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像。Determining the first divided sub-image on the first overlapping image according to the first Morse function value.
- 根据权利要求4所述的方法,其特征在于,所述根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像,具体包括:The method according to claim 4, wherein the determining the first divided sub-image on the first overlapping image according to the first Morse function value comprises:根据所述第一重叠图像上的像素点的第一摩尔斯函数值确定所述第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点;Determining, according to a first Morse function value of the pixel point on the first overlapping image, a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image;根据每一个所述第一局部最小值点、一个第一局部最大值点和两个所述第一鞍点确定一个第一分割子图像。A first segmentation sub-image is determined according to each of the first local minimum point, a first local maximum point, and two of the first saddle points.
- 根据权利要求5所述的方法,其特征在于,所述根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像之后,还包括:The method according to claim 5, wherein after determining the first divided sub-image on the first overlapping image according to the first Morse function value, the method further comprises:确定每个所述第一分割子图像的第一最大值点,使用区域算法,根据所述第一最大值点确定所述第一重叠图像对应的第一对偶分割子图像。Determining a first maximum point of each of the first divided sub-images, and determining, by using a region algorithm, a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point.
- 根据权利要求6所述的方法,其特征在于,所述为每个所述第一分割子图像在至少两个所述第一图像的背景图像中选取对应第一拼接子图像,具体包括:The method according to claim 6, wherein the selecting the first splice sub-image in the background image of the at least two of the first images for each of the first divided sub-images comprises:根据相连的所述第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据所述第一重叠图像的图像中心到所述第一对偶分割子图像的距离计算数据开销; Calculating a smoothing overhead according to an average value of the maximum color differences of the connected first dual-divided sub-images, and calculating a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image;根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像。And corresponding to the first splice sub-image for each of the first dual-divided sub-images according to the smoothing overhead and the data overhead.
- 根据权利要求7所述的方法,其特征在于,所述根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像,包括:The method according to claim 7, wherein the selecting the corresponding first splice sub-image for each of the first dual-divided sub-images according to the smoothing overhead and the data overhead comprises:根据所述第一对偶分割子图像构造第一图;Constructing a first map according to the first dual segmentation sub-image;根据所述光滑开销和所述数据开销确定所述第一图每条边的权值;Determining weights of each side of the first graph according to the smoothing overhead and the data overhead;根据所述第一图每条边的权值采用图切割方法对所述第一对偶分割子图像进行分类,根据分类结果为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像。And the first dual-divided sub-image is classified according to the weight of each edge of the first figure by using a graph cutting method, and the first stitching corresponding to each of the first dual-divided sub-images is selected according to the classification result. Sub image.
- 一种全景图像的获取装置,其特征在于,包括:A device for acquiring a panoramic image, comprising:获取模块,用于获取至少两张第一图像;Obtaining a module, configured to acquire at least two first images;提取模块,用于提取所述第一图像的背景图像和前景图像;An extraction module, configured to extract a background image and a foreground image of the first image;确定模块,用于确定所述至少两张第一图像的背景图像形成的第一重叠图像和第一非重叠图像;a determining module, configured to determine a first overlapping image and a first non-overlapping image formed by the background image of the at least two first images;所述确定模块,还用于确定所述至少两张第一图像的前景图像的第二重叠图像和第二非重叠图像;The determining module is further configured to determine a second overlapping image and a second non-overlapping image of the foreground image of the at least two first images;分割模块,用于对所述第一重叠图像进行分割得到多个第一分割子图像,对所述第二重叠图像进行分割得到多个第二分割子图像;a segmentation module, configured to divide the first overlapping image to obtain a plurality of first divided sub-images, and divide the second overlapping image to obtain a plurality of second divided sub-images;选取模块,用于为每个所述第一分割子图像在至少两个所述第一图像的背景图像中选取对应第一拼接子图像,为每个所述第二分割子图像在至少两个所述第一图像的前景图像中选取对应的第二拼接子图像;a selection module, configured to select, for each of the first divided sub-images, a corresponding first splice sub-image in at least two background images of the first image, and at least two for each of the second divided sub-images Selecting a corresponding second splice sub-image in the foreground image of the first image;拼接模块,用于对多个所述第一分割子图像对应的第一拼接子图像进行拼接得到目标背景图像,对多个所述第二分割子图像对应的第二拼接子图像进行拼接得到目标前景图像;a splicing module, configured to splicing a first splicing sub-image corresponding to the plurality of the first sub-sub-images to obtain a target background image, and splicing the second splicing sub-image corresponding to the second sub-sub-images to obtain a target Foreground image合成模块,用于对所述目标背景图像、所述目标前景图像、所述第一非重叠图像和所述第二非重叠图像进行合成,得到目标全景图像。And a synthesizing module, configured to synthesize the target background image, the target foreground image, the first non-overlapping image, and the second non-overlapping image to obtain a target panoramic image.
- 根据权利要求9所述的装置,其特征在于,所述提取模块具体用于:The device according to claim 9, wherein the extraction module is specifically configured to:根据所述第一图像的深度值确定所述第一图像的前景模板; Determining a foreground template of the first image according to a depth value of the first image;所述第一前景模板为0,1组成的矩阵,在同一像素点处,提取前景模板1所对应的所述第一图像的像素点,构成所述第一图像的前景图像;The first foreground template is a matrix composed of 0, 1. At the same pixel point, the pixel points of the first image corresponding to the foreground template 1 are extracted to form a foreground image of the first image;在同一像素点处,提取前景模板的补中1所对应的所述第一图像的像素点,构成所述第一图像的背景图像。At the same pixel point, the pixel points of the first image corresponding to the complement 1 of the foreground template are extracted to form a background image of the first image.
- 根据权利要求9或10所述的装置,其特征在于,所述确定模块具体用于:The device according to claim 9 or 10, wherein the determining module is specifically configured to:通过单应转换将每个所述第一背景图像转换为第一背景虚拟图像,通过单应转换将每个所述第一前景图像转换为第一前景虚拟图像;Converting each of the first background images into a first background virtual image by a homography conversion, converting each of the first foreground images into a first foreground virtual image by a homography conversion;根据所述第一背景虚拟图像计算各个所述第一背景图像的所述第一重叠图像和所述第一非重叠图像,根据所述第一前景虚拟图像计算各个所述第一前景图像的所述第二重叠图像和所述第二非重叠图像。Calculating the first overlapping image and the first non-overlapping image of each of the first background images according to the first background virtual image, and calculating, by the first foreground virtual image, each of the first foreground images The second overlapping image and the second non-overlapping image are described.
- 根据权利要求9-11任一项所述的装置,其特征在于,所述分割模块具体用于:The device according to any one of claims 9-11, wherein the segmentation module is specifically configured to:计算所述第一重叠图像在各个颜色通道上的第一差值,对所述第一差值依次进行拉普拉斯滤波变换和平滑处理,得到第一摩尔斯函数值;Calculating a first difference value of the first overlapping image on each color channel, performing Laplacian filtering transformation and smoothing processing on the first difference, to obtain a first Morse function value;根据所述第一摩尔斯函数值确定所述第一重叠图像上的所述第一分割子图像。Determining the first divided sub-image on the first overlapping image according to the first Morse function value.
- 根据权利要求12所述的装置,其特征在于,所述分割模块还用于:The device according to claim 12, wherein the segmentation module is further configured to:根据所述第一重叠图像上的像素点的第一摩尔斯函数值确定所述第一重叠图像上的第一局部最小值点、第一局部最大值点和第一鞍点;Determining, according to a first Morse function value of the pixel point on the first overlapping image, a first local minimum point, a first local maximum point, and a first saddle point on the first overlapping image;根据每一个所述第一局部最小值点、一个第一局部最大值点和两个所述第一鞍点确定一个第一分割子图像。A first segmentation sub-image is determined according to each of the first local minimum point, a first local maximum point, and two of the first saddle points.
- 根据权利要求13所述的装置,其特征在于,所述分割模块还用于:The device according to claim 13, wherein the segmentation module is further configured to:确定每个所述第一分割子图像的第一最大值点,使用区域算法,根据所述第一最大值点确定所述第一重叠图像对应的第一对偶分割子图像。Determining a first maximum point of each of the first divided sub-images, and determining, by using a region algorithm, a first dual-divided sub-image corresponding to the first overlapping image according to the first maximum point.
- 根据权利要求14所述的装置,其特征在于,所述选取模块具体用于: The device according to claim 14, wherein the selection module is specifically configured to:根据相连的所述第一对偶分割子图像的最大颜色差的平均值计算光滑开销,根据所述第一重叠图像的图像中心到所述第一对偶分割子图像的距离计算数据开销;Calculating a smoothing overhead according to an average value of the maximum color differences of the connected first dual-divided sub-images, and calculating a data overhead according to a distance from an image center of the first overlapping image to the first dual-divided sub-image;根据所述光滑开销和所述数据开销为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像。And corresponding to the first splice sub-image for each of the first dual-divided sub-images according to the smoothing overhead and the data overhead.
- 根据权利要求15所述的装置,其特征在于,所述选取模块还用于:The device according to claim 15, wherein the selection module is further configured to:根据所述第一对偶分割子图像构造第一图,所述第一图包括:多个所述第一对偶分割子图像;Constructing a first map according to the first dual-divided sub-image, the first map comprising: a plurality of the first dual-divided sub-images;根据所述光滑开销和所述数据开销确定所述第一图每条边的权值;Determining weights of each side of the first graph according to the smoothing overhead and the data overhead;根据所述第一图每条边的权值采用图切割方法对所述第一对偶分割子图像进行分类,根据分类结果为每个所述第一对偶分割子图像选取对应的所述第一拼接子图像。 And the first dual-divided sub-image is classified according to the weight of each edge of the first figure by using a graph cutting method, and the first stitching corresponding to each of the first dual-divided sub-images is selected according to the classification result. Sub image.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410178945.2A CN105096283B (en) | 2014-04-29 | 2014-04-29 | The acquisition methods and device of panoramic picture |
CN201410178945.2 | 2014-04-29 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015165222A1 true WO2015165222A1 (en) | 2015-11-05 |
Family
ID=54358102
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2014/088726 WO2015165222A1 (en) | 2014-04-29 | 2014-10-16 | Method and device for acquiring panoramic image |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN105096283B (en) |
WO (1) | WO2015165222A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108038820A (en) * | 2017-11-14 | 2018-05-15 | 深圳岚锋创视网络科技有限公司 | A kind of method, apparatus and panorama camera for realizing bullet time shooting effect |
CN111461985A (en) * | 2020-03-31 | 2020-07-28 | 维沃移动通信有限公司 | Picture processing method and electronic equipment |
US11373300B2 (en) * | 2020-01-31 | 2022-06-28 | International Business Machines Corporation | Image stitching using geometry-based image orientation |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106023125B (en) * | 2016-05-06 | 2019-01-04 | 安徽伟合电子科技有限公司 | It is a kind of to cover and obscure the image split-joint method reappeared based on image |
JP6429829B2 (en) * | 2016-05-25 | 2018-11-28 | キヤノン株式会社 | Image processing system, image processing apparatus, control method, and program |
CN108616733B (en) * | 2016-12-28 | 2020-01-10 | 深圳市掌网科技股份有限公司 | Panoramic video image splicing method and panoramic camera |
CN106991645B (en) | 2017-03-22 | 2018-09-28 | 腾讯科技(深圳)有限公司 | Image split-joint method and device |
CN107395974B (en) * | 2017-08-09 | 2019-09-13 | Oppo广东移动通信有限公司 | Image processing system and method |
CN107370951B (en) * | 2017-08-09 | 2019-12-27 | Oppo广东移动通信有限公司 | Image processing system and method |
CN107493411B (en) * | 2017-08-09 | 2019-09-13 | Oppo广东移动通信有限公司 | Image processing system and method |
CN107493412B (en) * | 2017-08-09 | 2019-09-13 | Oppo广东移动通信有限公司 | Image processing system and method |
CN109348114A (en) * | 2018-11-26 | 2019-02-15 | Oppo广东移动通信有限公司 | Imaging device and electronic equipment |
CN111047510B (en) * | 2019-12-17 | 2023-02-14 | 大连理工大学 | Large-field-angle image real-time splicing method based on calibration |
CN114283155B (en) * | 2021-11-23 | 2023-07-04 | 赣州好朋友科技有限公司 | Ore image segmentation method, device and computer readable storage medium |
WO2024119309A1 (en) * | 2022-12-05 | 2024-06-13 | 深圳华大生命科学研究院 | Method for stitching images, and device therefor |
CN117876538B (en) * | 2024-03-11 | 2024-05-31 | 成都环境工程建设有限公司 | Visual management method and system for garbage power plant |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6018349A (en) * | 1997-08-01 | 2000-01-25 | Microsoft Corporation | Patch-based alignment method and apparatus for construction of image mosaics |
US20060268130A1 (en) * | 2005-05-26 | 2006-11-30 | Williams Karen E | In-camera panorama stitching method and apparatus |
CN101859433A (en) * | 2009-04-10 | 2010-10-13 | 日电(中国)有限公司 | Image mosaic device and method |
CN101923709A (en) * | 2009-06-16 | 2010-12-22 | 日电(中国)有限公司 | Image splicing method and equipment |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7006709B2 (en) * | 2002-06-15 | 2006-02-28 | Microsoft Corporation | System and method deghosting mosaics using multiperspective plane sweep |
CN100337473C (en) * | 2004-09-30 | 2007-09-12 | 中国科学院计算技术研究所 | Panorama composing method for motion video |
CN100355272C (en) * | 2005-06-24 | 2007-12-12 | 清华大学 | Synthesis method of virtual viewpoint in interactive multi-viewpoint video system |
CN101621634B (en) * | 2009-07-24 | 2010-12-01 | 北京工业大学 | Method for splicing large-scale video with separated dynamic foreground |
-
2014
- 2014-04-29 CN CN201410178945.2A patent/CN105096283B/en not_active Expired - Fee Related
- 2014-10-16 WO PCT/CN2014/088726 patent/WO2015165222A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6018349A (en) * | 1997-08-01 | 2000-01-25 | Microsoft Corporation | Patch-based alignment method and apparatus for construction of image mosaics |
US20060268130A1 (en) * | 2005-05-26 | 2006-11-30 | Williams Karen E | In-camera panorama stitching method and apparatus |
CN101859433A (en) * | 2009-04-10 | 2010-10-13 | 日电(中国)有限公司 | Image mosaic device and method |
CN101923709A (en) * | 2009-06-16 | 2010-12-22 | 日电(中国)有限公司 | Image splicing method and equipment |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108038820A (en) * | 2017-11-14 | 2018-05-15 | 深圳岚锋创视网络科技有限公司 | A kind of method, apparatus and panorama camera for realizing bullet time shooting effect |
US11373300B2 (en) * | 2020-01-31 | 2022-06-28 | International Business Machines Corporation | Image stitching using geometry-based image orientation |
CN111461985A (en) * | 2020-03-31 | 2020-07-28 | 维沃移动通信有限公司 | Picture processing method and electronic equipment |
Also Published As
Publication number | Publication date |
---|---|
CN105096283A (en) | 2015-11-25 |
CN105096283B (en) | 2017-12-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2015165222A1 (en) | Method and device for acquiring panoramic image | |
US10540806B2 (en) | Systems and methods for depth-assisted perspective distortion correction | |
CN105957007B (en) | Image split-joint method based on characteristic point plane similarity | |
CN111066065B (en) | System and method for hybrid depth regularization | |
JP5908599B2 (en) | Multispectral imaging system | |
CN108600576B (en) | Image processing apparatus, method and system, and computer-readable recording medium | |
WO2017076106A1 (en) | Method and device for image splicing | |
KR101175097B1 (en) | Panorama image generating method | |
US20150278996A1 (en) | Image processing apparatus, method, and medium for generating color image data | |
EP3308323B1 (en) | Method for reconstructing 3d scene as 3d model | |
JP5672112B2 (en) | Stereo image calibration method, stereo image calibration apparatus, and computer program for stereo image calibration | |
Mistry et al. | Image stitching using Harris feature detection | |
AU2020203790B2 (en) | Transformed multi-source content aware fill | |
JP2011134012A (en) | Image processor, image processing method for the same and program | |
KR101853269B1 (en) | Apparatus of stitching depth maps for stereo images | |
WO2018147059A1 (en) | Image processing device, image processing method, and program | |
US11044452B2 (en) | Imaging system and method | |
WO2016113805A1 (en) | Image processing method, image processing apparatus, image pickup apparatus, program, and storage medium | |
KR20120071286A (en) | Apparatus for image matting using multi camera, and method for generating alpha map | |
CN111630569A (en) | Binocular matching method, visual imaging device and device with storage function | |
KR101718309B1 (en) | The method of auto stitching and panoramic image genertation using color histogram | |
JP2006293600A (en) | Image processing method, image processor, and image composition processor | |
JP6306494B2 (en) | Shape estimation device, shape estimation method, and shape estimation program | |
GB2555585A (en) | Multiple view colour reconstruction | |
Yashika et al. | Image Fusion in Multi-view Videos Using SURF Algorithm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14890774 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14890774 Country of ref document: EP Kind code of ref document: A1 |