CN107392848A - Panoramic image display method and device - Google Patents
Panoramic image display method and device Download PDFInfo
- Publication number
- CN107392848A CN107392848A CN201710449844.8A CN201710449844A CN107392848A CN 107392848 A CN107392848 A CN 107392848A CN 201710449844 A CN201710449844 A CN 201710449844A CN 107392848 A CN107392848 A CN 107392848A
- Authority
- CN
- China
- Prior art keywords
- overhead view
- image
- panorama
- source images
- view image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000009466 transformation Effects 0.000 claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 27
- 230000004927 fusion Effects 0.000 claims abstract description 12
- 238000012937 correction Methods 0.000 claims description 17
- 238000013507 mapping Methods 0.000 claims description 15
- 230000010354 integration Effects 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 description 12
- 230000000694 effects Effects 0.000 description 10
- 230000008859 change Effects 0.000 description 6
- 241000251468 Actinopterygii Species 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 210000003733 optic disk Anatomy 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20068—Projection on vertical or horizontal image axis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides a kind of panoramic image display method and device, methods described includes：4 frame source images of the vehicle camera collection of four direction all around are obtained, and antibarreling processing is carried out to source images described in every frame；Establish panoramic picture model, and the pixel of the source images after every frame is corrected by projective transformation project to the panoramic picture model one by one corresponding to pixel position, obtain overhead view image corresponding to source images described in every frame；Two common points in the public view field of two adjacent overhead view images are marked, and overhead view image progress image mosaic fusion treatment is obtained closely panorama overhead view image by the standard using the straight line that two common points are formed as image mosaic gap；Barrel distortion is carried out to the closely panorama overhead view image to handle to obtain remote panorama overhead view image.The present invention can provide a kind of clear, close to actual scene panoramic picture.
Description
Technical field
The present invention relates to automobile technical field, more particularly to a kind of panoramic image display method and device.
Background technology
With the fast development of social economy, automobile has turned into the indispensable vehicles in average family life.
In all parts of automobile and design concept, security is to occupy center-stage all the time, except automobile is had in itself
It is various in the case of safety guarantee configuration, how to be guided to a kind of controllable safe driving of driver is automotive safety all the time
This field is constantly pursued and innovation.
Existing panorama auxiliary park mode be mainly using the image around vehicle Shang Ge roads camera collection vehicle body,
And obtain panoramic picture after display device for mounting on vehicle carries out image projection, splicing.But panorama sketch is established in the prior art
The processing procedure of picture is computationally intensive, and the overlapping region line of demarcation of the image of each road camera shooting is obvious, and can not adjust panorama
The distance of image, panoramic picture clear, close to actual scene can not be obtained, influences judgement of the user to vehicle body surrounding environment.
The content of the invention
In view of above-mentioned condition, it is necessary to for panoramic picture clear, close to actual scene can not be obtained in the prior art
The problem of, there is provided a kind of panoramic image display method and device.
The invention provides a kind of panoramic image display method, including：
4 frame source images of the vehicle camera collection of four direction all around are obtained, and source images described in every frame are entered
Row antibarreling is handled；
Panoramic picture model is established, and is thrown the pixel of the source images after the correction of every frame one by one by projective transformation
Pixel position corresponding to shadow to the panoramic picture model, obtains overhead view image corresponding to source images described in every frame；
Two common points in the public view field of two adjacent overhead view images are marked, and with two common points
The straight line of formation obtains overhead view image progress image mosaic fusion treatment closely as the standard in image mosaic gap
Panorama overhead view image；
Barrel distortion is carried out to the closely panorama overhead view image to handle to obtain remote panorama overhead view image.
Above-mentioned panoramic image display method, wherein, it is described that antibarreling processing is carried out to source images described in every frame
Step includes：
Using default concentric circles at equal intervals as correction model, antibarreling processing is carried out to described image.
Above-mentioned panoramic image display method, wherein, it is described every frame is corrected by projective transformation after the source images
Pixel project to the panoramic picture model one by one corresponding to pixel position, the step of obtaining overhead view image includes：
In the panoramic picture model, 8 calibration points, the source images bag of each camera collection are marked along ground
4 calibration points are included, according to the coordinate of 4 calibration points, the parameter of DLT algorithms is solved, obtains the Transformation Relation of Projection formula；
According to the Transformation Relation of Projection formula, the picture corresponding to the source images after every frame is corrected in quadrilateral area
Vegetarian refreshments carries out mapping solution successively, obtains overhead view image.
Above-mentioned panoramic image display method, wherein, the straight line formed using two common points is used as image mosaic
After the step of overhead view image progress image mosaic fusion treatment is obtained closely panorama overhead view image by the standard in gap
Also include：
Determine 8~20 pixels of integration region and the coefficient of opacity of each pixel；
Two adjacent overhead view images are corresponded to by the fusion by the proportionate relationship that the coefficient of opacity determines
The pixel in region is overlapped processing, obtains smooth panorama top view.
Above-mentioned panoramic image display method, wherein, it is described that the closely panorama overhead view image is carried out at barrel distortion
The step of reason obtains remote panorama overhead view image includes：
The closely panorama overhead view image is carried out using template of the equidistant concentric circles as barrel distortion model
Barrel distortion handles to obtain remote panorama overhead view image, and the barrel distortion model is：
Sr(r)=akr4+bkr3+ckr2+ dkr+e, wherein, srIt is respectively model parameter for pantograph ratio, a, b, c, d, e, r is
The radius of the concentric circles, k are distortion parameter changed factor.
The present invention also provides a kind of panoramic image display device, including：
Acquisition module, for obtaining 4 frame source images of the vehicle camera collection of four direction all around；
Correction module, for carrying out antibarreling processing to source images described in every frame；
Modeling module, for establishing panoramic picture model；
Projective transformation module, the pixel for the source images after every frame is corrected by projective transformation project one by one
To pixel position corresponding to the panoramic picture model, overhead view image corresponding to source images described in every frame is obtained；
Image mosaic module, two common points in public view field for marking two adjacent overhead view images,
And the overhead view image is carried out image mosaic by the standard using the straight line that two common points are formed as image mosaic gap
Fusion treatment obtains closely panorama overhead view image；
Barrel distortion module, handle to obtain at a distance entirely for carrying out the closely panorama overhead view image barrel distortion
Scape overhead view image.
Above-mentioned panoramic image display device, wherein, the correction module is specifically used for：With default equidistant concentric circles
As correction model, antibarreling processing is carried out to described image.
Above-mentioned panoramic image display device, wherein, the projective transformation module includes：
The Transformation Relation of Projection formula computing module, in the panoramic picture model, 8 calibration points to be marked along ground,
The source images of each camera collection include 4 calibration points, according to the coordinate of 4 calibration points, solve DLT and calculate
The parameter of method, to obtain the Transformation Relation of Projection formula；
Module is solved, for according to the Transformation Relation of Projection formula, four corresponding to the source images after every frame is corrected
Pixel in the shape region of side carries out mapping solution successively, obtains overhead view image.
Above-mentioned panoramic image display device, in addition to：
Determining module, for determining 8~20 pixels of integration region and the coefficient of opacity of each pixel；
Overlap-add procedure module, for by the proportionate relationship that the coefficient of opacity determines by two adjacent vertical views
The pixel that image corresponds to the integration region is overlapped processing, obtains smooth panorama top view.
Above-mentioned panoramic image display device, wherein, the barrel distortion module is specifically used for：
The closely panorama overhead view image is carried out using template of the equidistant concentric circles as barrel distortion model
Barrel distortion handles to obtain remote panorama overhead view image, and the barrel distortion model is：
Sr(r)=akr4+bkr3+ckr2+ dkr+e, wherein, srIt is respectively model parameter for pantograph ratio, a, b, c, d, e, r is
The radius of the concentric circles, k are distortion parameter changed factor.
The embodiment of the present invention marks two common points in the public view field of adjacent source images according to panoramic picture model, and
Using the connection of two common points as the splicing standard of overlapping region, the closely vertical view panorama of boundary transitions smooth is obtained
Figure, provides the user the panorama sketch for being more nearly actual scene.And by carrying out barrel distortion to closely panorama overhead view image
Processing obtains remote panorama overhead view image, to change visual range, so as to obtain panorama sketch clear, close to actual scene
Picture.
Brief description of the drawings
Fig. 1 is the panoramic image display method flow chart in first embodiment of the invention；
Fig. 2 is the panoramic image display method flow chart in second embodiment of the invention；
Fig. 3 is equidistantly concentric circle template schematic diagram；
Fig. 4 a are the source images before barrel aberrance emendation；
Fig. 4 b are the source images after barrel aberrance emendation；
Fig. 5 is panoramic picture model schematic；
Fig. 6 is the schematic diagram of the ground calibration point of panoramic picture model；
Fig. 7 is closely panorama overhead view image；
Fig. 8 is remote panorama overhead view image；
Fig. 9 is the structured flowchart of the panoramic image display device in third embodiment of the invention.
Embodiment
Embodiments of the invention are described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or the element with same or like function.Below with reference to attached
The embodiment of figure description is exemplary, is only used for explaining the present invention, and is not considered as limiting the invention.
With reference to following description and accompanying drawing, it will be clear that these and other aspects of embodiments of the invention.In these descriptions
In accompanying drawing, some particular implementations in embodiments of the invention are specifically disclosed, to represent to implement the implementation of the present invention
Some modes of the principle of example, but it is to be understood that the scope of embodiments of the invention is not limited.On the contrary, the present invention
Embodiment includes all changes, modification and the equivalent fallen into the range of the spirit and intension of attached claims.
Fig. 1 is referred to as the panoramic image display method in first embodiment of the invention, including step S11~S14.
Step S11,4 frame source images of the vehicle camera collection of four direction all around are obtained, and to described in every frame
Source images carry out antibarreling processing.
In the present embodiment, one camera is all around respectively installed in automobile, to cover 360 degree of scope ground of vehicle field
Scape.The camera uses wide-angle camera, can increase coverage, but because the angle of visual field increase of wide-angle camera causes
Source images deform, therefore are corrected for the barrel-shaped distortion of original image.The present embodiment can be by any one in the prior art
Antibarreling algorithm is corrected.
Step S12, establishes panoramic picture model, and the pixel of the source images after every frame is corrected by projective transformation
Point projects to pixel position corresponding to the panoramic picture model one by one, obtains top view corresponding to source images described in every frame
Picture.
Four cameras are installed in the surrounding of vehicle, and what it shot is all from the side view of respective different angle, to allow
Driver watches more intuitively image, side elevation image need to be made into a projective transformation, side elevation image is transformed into and is similar to
The look down on the rendering of camera shooting directly over vehicle.
Step S13, two common points in the public view field of two adjacent overhead view images are marked, and with two institutes
The straight line for stating common point formation obtains overhead view image progress image mosaic fusion treatment as the standard in image mosaic gap
To closely panorama overhead view image.
There is overlapping region in the source images of adjacent camera shooting, it is necessary to by image mosaic technology splice at fusion
Reason.In projective transformation processing, the straight line of the different pixels point composition of the source images of camera shooting, after conversion, corresponding picture
Vegetarian refreshments is still on same straight line.The overlapping region mark for the source images that the present embodiment is shot according to this characteristic in adjacent camera
Two common points are remembered as the foundation that image seamless splices to handle visual field overlapping region part.After image mosaic processing, as long as
Record the mapping relations between the panoramic pixel point after source images and processing, it is possible to form panorama look down on the rendering, obtain near
Apart from panorama overhead view image.
Step S14, barrel distortion is carried out to the closely panorama overhead view image and handles to obtain remote panorama top view
Picture.
Closely for panorama overhead view image according to the fixation of resolution ratio, visual range is very limited, general visual range 2~
4 meters or so.The visual field of wide-angle lens is bigger than the visual field of common lens, and main cause is to add barrel-shaped distortion therebetween, according to this
Principle, reprojection's conversion is carried out in obtained closely panoramic picture, i.e., closely panorama overhead view image is carried out
Barrel distortion is handled, and it substantially can be regarded as the inverse process of antibarreling, it is possible to obtain the remote of wide-angle effect
Panoramic design sketch.By adjusting the amount of distortion of panoramic picture, change visual range.
The present embodiment marks two common points in the public view field of adjacent source images according to panoramic picture model, and with two
Splicing standard of the connection of individual common point as overlapping region, obtains the closely vertical view panorama sketch of boundary transitions smooth, is
User provides the panorama sketch for being more nearly actual scene.And handled by carrying out barrel distortion to closely panorama overhead view image
To remote panorama overhead view image, to change visual range.
Referring to Fig. 2, be the panoramic image display method in second embodiment of the invention, including step S21~S27.
Step S21,4 frame source images of the vehicle camera collection of four direction all around are obtained, and with default etc.
The concentric circles of spacing carries out antibarreling processing as correction model to described image.
Has more ripe antibarreling algorithm both at home and abroad at present, wherein the algorithm that two classes are commonly used is：Based on known
Parametric method and based on template.Using circle template is mainly as correction model, its foundation with one heart at equal intervals in the present embodiment：
(1) with one heart after the equidistant concentric circles imaging in circle template, it should be still equidistant concentric circles；(2) it is smaller from picture centre, figure
Image distortion amount is smaller, and for the minimum concentric circles in template close to optical axis, approximation is thought of as abnormal vanishing；(3) distortion of image has
Continuity, the barrel distortion of the annulus in same Radius are symmetrical.Equidistantly circle template is as shown in Figure 3 with one heart.
The cross center of concentric circle template is placed in the center of source images, to reduce to round identification error,
Image processing tool, such as photoshop are utilized to current source picture, it is several to utilizing color discontinuity to mark on each circle
Point, then according to colouring information, circle is fitted by least square method.The radius value for each circle tried to achieve is expressed as rm=(r1,
r2,...,rn), wherein r1It is smallest circle radius value, r1Close to picture centre, undistorted, the then preferable radius of each circle is approximately considered
Value is represented by rt=(r1,3r1,...,(2n-1)r1).Calculate the measured value and theory of each radius of circle of concentric circle template
The ratio of value is：
With measured value rmFor independent variable, srFor because becoming
Amount, establishes relationship sr(r)=f (r).According to the continuous principle of image deformation, measurement is fitted using quartic polynomial
The ratio rule of radius value and principal radii value, quartic polynomial are represented by：
sr(r)=ar4+br3+cr2+ dr+e, wherein, 0≤r≤max (r), wherein having r in the radius of minimum annulus1=0,
sr1=1, quartic polynomial is with ((0,1), (r2,sr2),...,(rn,srn)) put to be fitted barrel-shaped distortion Distortion Law.
With the centre coordinate (m in current source picture0,n0) it is origin, rectangular coordinate system is established, then is appointed in current source picture
The pixel coordinate (m, n) of meaning point is represented by x=n-n0, y=m-m0, (m, n) and the positive angle of X-axis are represented by α=tan-1(y/x), therefore the coordinate (M, N) in point (m, n) theory is：
As y >=0,
Work as y<When 0
Wherein ceil rounds for forward direction.After source image pixels point and antibarreling being can obtain according to relational expression one and two
Pixel mapping relations.By the solution of source images to 4 vertex positions of image, new image array is built, then will
The pixel point value of source images mapping is mapped to ideal image position, it is possible to recovers the image under normal vision, barrel distortion
It is as shown in Figs. 4a and 4b to correct comparison diagram.
Step S22, panoramic picture model is established, in the panoramic picture model, mark 8 calibration points along ground, often
The source images of the individual camera collection include 4 calibration points, using the deformation method of arbitrary quadrilateral, solve DLT and calculate
The parameter of method, obtain the Transformation Relation of Projection formula.
Calculated in the present embodiment with carton simulating vehicle, the length and width of auto model measured using survey tool,
L and W are designated as, the total resolution of setting panoramic picture model is CN*RN, as shown in figure 5, model is divided into five pieces of regions, its
The corresponding front camera shooting image mapping area in middle region 0, the corresponding rear camera shooting image mapping area in region 1, together
Reason region 2 and 3 corresponds to camera the left and right shooting image mapping area respectively, and 4 corresponding car body area of region.Region 4
Positioned at the centre of model, if the regional extent in region 4 is fixed, other region units are also with regard to automatic fixed allocation.
Then, if the length and width in region 4 are length_4 and width_4 respectively, then length_4 and width_4 value should
With the length and width direct proportionality of auto model, that is, have：
K values are freely set, general setting causes the width in region 4 according to the demand size in the region 4 of panoramic picture model
Account for panoramic picture model it is wide 1/3rd or so.Also fixed allocation, wherein a~h divide the image-region of such four cameras
It is not the line of demarcation point between camera.According to the proportionate relationship value between auto model and region 4, several are demarcated along ground
Point divides the image range value per road source images, as shown in Figure 6 to be mapped with panoramic picture model cut points.
Because the length and width ratio in the region 4 of panoramic picture model is equal with the length and width ratio of auto model, then two
Ratio relation between person, H1 and H0 is represented by figure：
After H1 and H0 length computation comes out, it is possible to point A position is obtained, and point C, E, G also may be used according to symmetric relation
To be calculated.So on auto model A, B, C, D it is corresponding in front of camera the corresponding pixel value of source images, respectively
It is mapped to a, b, c, d of panoramic picture model.A, to be that source images are to be mapped arrive the side of a, b, c, d tetra- to the quadrangle that B, C, D are formed
The image-region of shape, the shooting image mapping of other cameras is similarly.In addition AB, CD, EF, GH are respectively as adjacent camera
The splicing standard of shooting image overlapping region.
The camera scope of front, rear, left and right four is limited respectively along eight point A~H of ground mark, its every 4 structure
Into regional extent just it is corresponding with panoramic picture 4 points composition regions produce mapping relations.
The fundamental formular of DLT algorithms (Direct linear transformation, straight linear conversion) is：
(X, Y, Z) is the coordinate of physical space point in formula, and (u, v) is the seat that physical space point corresponds to pixel in source images
Punctuate, A, B ... K are 11 unknown quantitys, therefore 6 groups of space coordinates known to only needing and image coordinate value can solve mapping and close
It is formula.When the region of camera shooting is horizontal plane, it is believed that Z is constant, for simplified operation, makes Z=0, then：
To solve the unknown quantity in relational expression, it is only necessary to known 4 pairs and the space coordinates of the above and the correspondence of image coordinate
Value.By establishing panoramic picture model, 8 points are labelled with along ground：A~H.The source images of each camera capture have 4
Calibration point, then using the deformation method of arbitrary quadrilateral, if the quadrangle apex coordinate of source images is (x respectively all the way1,y1)、
(x2,y2)、(x3,y3)、(x4,y4), the apex coordinate being mapped in panoramic picture model is (u respectively1,v1)、(u2,v2)、(u3,
v3)、(u4,v4).Two groups of quadrangle apex coordinates are updated in formula four and obtain 8 yuan of systems of linear equations.
8 parameter A~J are solved to obtain using Gaussian elimination method by the augmented matrix of equation group, obtain projective transformation pass
It is formula.
Step S23, according to the Transformation Relation of Projection formula, quadrangle area corresponding to the source images after every frame is corrected
Pixel in domain carries out mapping solution successively, obtains overhead view image corresponding to source images described in every frame.
Step S24, two common points in the public view field of two adjacent overhead view images are marked, and with two institutes
The straight line for stating common point formation obtains overhead view image progress image mosaic fusion treatment as the standard in image mosaic gap
To closely panorama overhead view image.
In above-mentioned steps, two common points of mark in the public view field per two adjacent overhead view images, connect adjacent
Two common points of mark form straight line in the public view field of overhead view image, and four overhead view images just exist straight as 4
Line, reasonably it is assigned with the pixel region of each road source images.Standard using every straight line as image mosaic gap, has synthesized picture
Look down on the rendering picture corresponding to element.
Step S25, determine 8~20 pixels of integration region and the coefficient of opacity of each pixel.
Step S26, it is by the proportionate relationship that the coefficient of opacity determines that two adjacent overhead view images are corresponding
The pixel of the integration region is overlapped processing, obtains smooth panorama top view.
In order to obtain more preferable panoramic design sketch, image co-registration processing is carried out at line of demarcation so that boundary transition is put down
It is sliding.Using simple linear algorithm, 8~20 pixels are set, determine the coefficient of opacity α values of each pixel of integration region,
Pixel point value overlap-add procedure corresponding to overhead view image after adjacent camera is projected by the α proportionate relationships determined, so as to
To the smooth closely panorama overhead view image in unobvious line of demarcation, as shown in Figure 7.
Step S27, the closely panorama overhead view image is entered as barrel distortion model using equidistant concentric circles
Row barrel distortion handles to obtain remote panorama overhead view image.
In order to increase the visual range of panorama, a reprojection is carried out to closely panorama overhead view image in the present embodiment
Processing, that is, the amount of distortion of closely panorama overhead view image is adjusted, be substantially interpreted as the inverse process of antibarreling, to obtain
The remote panorama sketch of fish mesh effect.The template and antibarreling identical that its barrel distortion model uses are equidistantly concentric
Circle, the barrel distortion model are：
Sr(r)=akr4+bkr3+ckr2+ dkr+e, wherein, srIt is respectively model parameter for pantograph ratio, a, b, c, d, e, r is
The radius of the concentric circles, k are distortion parameter changed factor.
By setting distortion parameter changed factor k, while adjust and 4 non-constant term systems are obtained in barrel-shaped distortion distortion model
Number, you can realize the remote panorama overhead view image of the fish mesh effect of the different distortion degrees of distortion degree consecutive variations.When k is held
When continuous reduction approaches or is 0, the image that reprojection obtains is undistorted closely panorama overhead view image.
Distortion parameter changed factor k is bigger, then the degree to distort is bigger, and resulting remote panorama overhead view image shrinks
Degree is more serious, and image viewing effect directly perceived is also bigger closer to spherical, equivalent vertical view visual angle.K is gradually reduced, then image is abnormal
The degree of change is also gradually smaller, and remote panorama overhead view image shrinkage degree reduces with k values to be also gradually reduced, and equivalent vertical view regards
Angle is also gradually reduced.When k is close to 0, obtained remote panorama overhead view image there is no obvious distortion effect.
When k is in a proper median, a more satisfactory fish mesh effect panorama overhead view image can be obtained.Experiment
In data, when setting K to change in the range of 0.8~1.2, more satisfactory fish mesh effect overhead view image, this figure have been obtained
The equivalent depression angle of picture is about 180 degree, and in overhead view image, horizon is retracted in image range, and distortion degree also compares
More uniform, viewing effect is more satisfactory, and its design sketch is as shown in Figure 8.
The present embodiment is carried out at a reprojection to increase the visual range of panorama to panoramic picture closely
Reason, by adjusting the amount of distortion of panoramic picture, to obtain the remote panorama sketch of fish mesh effect.
Referring to Fig. 9, present invention also offers a kind of panoramic image display device, including：
Acquisition module 100, for obtaining 4 frame source images of the vehicle camera collection of four direction all around；
Correction module 200, for carrying out antibarreling processing to source images described in every frame；
Modeling module 300, for establishing panoramic picture model；
Projective transformation module 400, the pixel for the source images after every frame is corrected by projective transformation is one by one
Pixel position corresponding to projecting to the panoramic picture model, obtains overhead view image corresponding to source images described in every frame；
Image mosaic module 500, two in public view field for marking two adjacent overhead view images are public
Point, and the overhead view image is carried out image spelling by the standard using the straight line that two common points are formed as image mosaic gap
Connect fusion treatment and obtain closely panorama overhead view image；
Barrel distortion module 600, handle to obtain long distance for carrying out barrel distortion to the closely panorama overhead view image
From panorama overhead view image.
Further, the correction module 200 is specifically used for：Using default concentric circles at equal intervals as correction model,
Antibarreling processing is carried out to described image.
Further, the projective transformation module 400 includes：
The Transformation Relation of Projection formula computing module, in the panoramic picture model, 8 calibration points to be marked along ground,
The source images of each camera collection include 4 calibration points, according to the coordinate of 4 calibration points, solve DLT and calculate
The parameter of method, to obtain the Transformation Relation of Projection formula；
Module is solved, for according to the Transformation Relation of Projection formula, four corresponding to the source images after every frame is corrected
Pixel in the shape region of side carries out mapping solution successively, obtains overhead view image.
Further, the panoramic image display device also includes：
Determining module 700, for determining 8~20 pixels and the opaque system of each pixel of integration region
Number；
Overlap-add procedure module 800, for by the proportionate relationship that the coefficient of opacity determines by described in adjacent two
The pixel that overhead view image corresponds to the integration region is overlapped processing, obtains smooth panorama top view.
Further, the barrel distortion module 600 is specifically used for：
It is barrel-shaped abnormal to closely panorama overhead view image progress as barrel distortion model using equidistant concentric circles
Change handles to obtain remote panorama overhead view image, and the barrel distortion model is：
Sr(r)=akr4+bkr3+ckr2+ dkr+e, wherein, srIt is respectively model parameter for pantograph ratio, a, b, c, d, e, r is
The radius of the concentric circles, k are distortion parameter changed factor.
The device of the present embodiment, it can be used for the technical scheme for performing any shown embodiments of the method for Fig. 1 to Fig. 2, in fact
Existing principle is similar with technique effect, and here is omitted.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) use, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress used with reference to these instruction execution systems, device or equipment
Put.
The more specifically example (non-exhaustive list) of computer-readable medium includes following：Connected up with one or more
Electrical connection section (electronic installation), portable computer diskette box (magnetic device), random access memory (RAM), read-only storage
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device, and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium, which can even is that, to print the paper of described program thereon or other are suitable
Medium, because can then enter edlin, interpretation or if necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each several part of the present invention can be realized with hardware, software, firmware or combinations thereof.Above-mentioned
In embodiment, software that multiple steps or method can be performed in memory and by suitable instruction execution system with storage
Or firmware is realized.If, and in another embodiment, can be with well known in the art for example, realized with hardware
Any one of row technology or their combination are realized：With the logic gates for realizing logic function to data-signal
Discrete logic, have suitable combinational logic gate circuit application specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment or example of the present invention.In this manual, to the schematic representation of above-mentioned term not
Necessarily refer to identical embodiment or example.Moreover, specific features, structure, material or the feature of description can be any
One or more embodiments or example in combine in an appropriate manner.
Embodiment described above only expresses the several embodiments of the present invention, and its description is more specific and detailed, but simultaneously
Therefore the limitation to the scope of the claims of the present invention can not be interpreted as.It should be pointed out that for one of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention
Protect scope.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (10)
- A kind of 1. panoramic image display method, it is characterised in that including：4 frame source images of the vehicle camera collection of four direction all around are obtained, and bucket is carried out to source images described in every frame Shape distortion correction treatment；Panoramic picture model is established, and is projected to the pixel of the source images after the correction of every frame one by one by projective transformation Pixel position corresponding to the panoramic picture model, obtains overhead view image corresponding to source images described in every frame；Two common points in the public view field of two adjacent overhead view images are marked, and are formed with two common points Straight line as image mosaic gap standard by the overhead view image progress image mosaic fusion treatment obtain closely panorama Overhead view image；Barrel distortion is carried out to the closely panorama overhead view image to handle to obtain remote panorama overhead view image.
- 2. panoramic image display method as claimed in claim 1, it is characterised in that described that bucket is carried out to source images described in every frame The step of shape distortion correction treatment, includes：Using default concentric circles at equal intervals as correction model, antibarreling processing is carried out to described image.
- 3. panoramic image display method as claimed in claim 1, it is characterised in that described to be corrected by projective transformation per frame The pixel of the source images afterwards project to the panoramic picture model one by one corresponding to pixel position, obtain overhead view image The step of include：In the panoramic picture model, 8 calibration points are marked along ground, the source images of each camera collection include 4 The individual calibration point, according to the coordinate of 4 calibration points, the parameter of DLT algorithms is solved, obtains the Transformation Relation of Projection formula；According to the Transformation Relation of Projection formula, the pixel corresponding to the source images after every frame is corrected in quadrilateral area Mapping solution is carried out successively, obtains overhead view image.
- 4. panoramic image display method as claimed in claim 1, it is characterised in that described to be formed with two common points Overhead view image progress image mosaic fusion treatment is obtained closely panorama as the standard in image mosaic gap and bowed by straight line Also include after the step of visible image：Determine 8~20 pixels of integration region and the coefficient of opacity of each pixel；Two adjacent overhead view images are corresponded to by the integration region by the proportionate relationship that the coefficient of opacity determines Pixel be overlapped processing, obtain smooth panorama top view.
- 5. panoramic image display method as claimed in claim 1, it is characterised in that described to the closely panorama top view Handle to include the step of obtaining remote panorama overhead view image as carrying out barrel distortion：It is barrel-shaped to closely panorama overhead view image progress as the template of barrel distortion model using equidistant concentric circles Distortion handles to obtain remote panorama overhead view image, and the barrel distortion model is：Sr(r)=akr4+bkr3+ckr2+ dkr+e, wherein, srIt is respectively model parameter for pantograph ratio, a, b, c, d, e, r is described The radius of concentric circles, k are distortion parameter changed factor.
- A kind of 6. panoramic image display device, it is characterised in that including：Acquisition module, for obtaining 4 frame source images of the vehicle camera collection of four direction all around；Correction module, for carrying out antibarreling processing to source images described in every frame；Modeling module, for establishing panoramic picture model；Projective transformation module, the pixel for the source images after every frame is corrected by projective transformation project to institute one by one Pixel position corresponding to panoramic picture model is stated, obtains overhead view image corresponding to source images described in every frame；Image mosaic module, two common points in public view field for marking two adjacent overhead view images, and with The overhead view image is carried out image mosaic fusion by the straight line that two common points are formed as the standard in image mosaic gap Processing obtains closely panorama overhead view image；Barrel distortion module, handle to obtain remote panorama for carrying out barrel distortion to the closely panorama overhead view image and bow Visible image.
- 7. panoramic image display device as claimed in claim 6, it is characterised in that the correction module is specifically used for：With pre- If equidistant concentric circles as correction model, antibarreling processing is carried out to described image.
- 8. panoramic image display device as claimed in claim 6, it is characterised in that the projective transformation module includes：The Transformation Relation of Projection formula computing module, in the panoramic picture model, marking 8 calibration points along ground, each The source images of the camera collection include 4 calibration points, according to the coordinate of 4 calibration points, solve DLT algorithms Parameter, to obtain the Transformation Relation of Projection formula；Module is solved, for according to the Transformation Relation of Projection formula, quadrangle corresponding to the source images after every frame is corrected Pixel in region carries out mapping solution successively, obtains overhead view image.
- 9. panoramic image display device as claimed in claim 6, it is characterised in that also include：Determining module, for determining 8~20 pixels of integration region and the coefficient of opacity of each pixel；Overlap-add procedure module, for by the proportionate relationship that the coefficient of opacity determines by two adjacent overhead view images The pixel of the corresponding integration region is overlapped processing, obtains smooth panorama top view.
- 10. panoramic image display device as claimed in claim 6, it is characterised in that the barrel distortion module is specifically used for：It is barrel-shaped to closely panorama overhead view image progress as the template of barrel distortion model using equidistant concentric circles Distortion handles to obtain remote panorama overhead view image, and the barrel distortion model is：Sr(r)=akr4+bkr3+ckr2+ dkr+e, wherein, srIt is respectively model parameter for pantograph ratio, a, b, c, d, e, r is described The radius of concentric circles, k are distortion parameter changed factor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710449844.8A CN107392848A (en) | 2017-06-14 | 2017-06-14 | Panoramic image display method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710449844.8A CN107392848A (en) | 2017-06-14 | 2017-06-14 | Panoramic image display method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107392848A true CN107392848A (en) | 2017-11-24 |
Family
ID=60331840
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710449844.8A Pending CN107392848A (en) | 2017-06-14 | 2017-06-14 | Panoramic image display method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107392848A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109410140A (en) * | 2018-10-24 | 2019-03-01 | 京东方科技集团股份有限公司 | A kind of distortion correction method, device, system and computer readable storage medium |
| CN109978926A (en) * | 2018-12-29 | 2019-07-05 | 深圳市行知达科技有限公司 | A kind of automatic fusion method of image, device and terminal device |
| CN110006420A (en) * | 2018-05-31 | 2019-07-12 | 上海快仓智能科技有限公司 | Build drawing method, image acquisition and processing system and localization method |
| CN111008985A (en) * | 2019-11-07 | 2020-04-14 | 贝壳技术有限公司 | Panorama picture seam detection method and device, readable storage medium and electronic equipment |
| CN111284503A (en) * | 2020-03-10 | 2020-06-16 | 宁波远景汽车零部件有限公司 | Automobile safety trip device |
| CN111476716A (en) * | 2020-04-03 | 2020-07-31 | 深圳力维智联技术有限公司 | Real-time video splicing method and device |
| CN112150367A (en) * | 2019-06-26 | 2020-12-29 | 比亚迪股份有限公司 | Display control method and device of vehicle-mounted head-up display device and vehicle |
| CN112581523A (en) * | 2020-12-01 | 2021-03-30 | 广东世宇科技股份有限公司 | Method and device for determining ball cake position, electronic equipment and readable storage medium |
| CN113066003A (en) * | 2021-03-12 | 2021-07-02 | 深圳市唯特视科技有限公司 | Panoramic image generation method and device, electronic equipment and storage medium |
| CN115379122A (en) * | 2022-10-18 | 2022-11-22 | 鹰驾科技（深圳）有限公司 | Video content dynamic splicing method, system and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164274A (en) * | 2011-04-26 | 2011-08-24 | 石黎 | Vehicle-mounted virtual panoramic system with variable field of view |
| CN102629372A (en) * | 2012-02-22 | 2012-08-08 | 北京工业大学 | 360 degree panoramic aerial view generation method used for assisting vehicle driving |
| CN103617606A (en) * | 2013-11-26 | 2014-03-05 | 中科院微电子研究所昆山分所 | Vehicle multi-angle panorama generating method for aided driving |
| CN105894549A (en) * | 2015-10-21 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Panorama assisted parking system and device and panorama image display method |
-
2017
- 2017-06-14 CN CN201710449844.8A patent/CN107392848A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164274A (en) * | 2011-04-26 | 2011-08-24 | 石黎 | Vehicle-mounted virtual panoramic system with variable field of view |
| CN102629372A (en) * | 2012-02-22 | 2012-08-08 | 北京工业大学 | 360 degree panoramic aerial view generation method used for assisting vehicle driving |
| CN103617606A (en) * | 2013-11-26 | 2014-03-05 | 中科院微电子研究所昆山分所 | Vehicle multi-angle panorama generating method for aided driving |
| CN105894549A (en) * | 2015-10-21 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Panorama assisted parking system and device and panorama image display method |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110006420A (en) * | 2018-05-31 | 2019-07-12 | 上海快仓智能科技有限公司 | Build drawing method, image acquisition and processing system and localization method |
| CN110006420B (en) * | 2018-05-31 | 2024-04-23 | 上海快仓智能科技有限公司 | Picture construction method, image acquisition and processing system and positioning method |
| CN109410140B (en) * | 2018-10-24 | 2021-02-26 | 京东方科技集团股份有限公司 | Distortion correction method, device and system and computer readable storage medium |
| CN109410140A (en) * | 2018-10-24 | 2019-03-01 | 京东方科技集团股份有限公司 | A kind of distortion correction method, device, system and computer readable storage medium |
| CN109978926A (en) * | 2018-12-29 | 2019-07-05 | 深圳市行知达科技有限公司 | A kind of automatic fusion method of image, device and terminal device |
| CN112150367A (en) * | 2019-06-26 | 2020-12-29 | 比亚迪股份有限公司 | Display control method and device of vehicle-mounted head-up display device and vehicle |
| CN112150367B (en) * | 2019-06-26 | 2023-11-14 | 比亚迪股份有限公司 | Display control method and device of vehicle-mounted head-up display device and vehicle |
| CN111008985A (en) * | 2019-11-07 | 2020-04-14 | 贝壳技术有限公司 | Panorama picture seam detection method and device, readable storage medium and electronic equipment |
| CN111008985B (en) * | 2019-11-07 | 2021-08-17 | 贝壳找房(北京)科技有限公司 | Panorama picture seam detection method and device, readable storage medium and electronic equipment |
| CN111284503A (en) * | 2020-03-10 | 2020-06-16 | 宁波远景汽车零部件有限公司 | Automobile safety trip device |
| CN111284503B (en) * | 2020-03-10 | 2021-12-10 | 宁波远景汽车零部件有限公司 | Automobile safety trip device |
| CN111476716B (en) * | 2020-04-03 | 2023-09-26 | 深圳力维智联技术有限公司 | Real-time video stitching method and device |
| CN111476716A (en) * | 2020-04-03 | 2020-07-31 | 深圳力维智联技术有限公司 | Real-time video splicing method and device |
| CN112581523A (en) * | 2020-12-01 | 2021-03-30 | 广东世宇科技股份有限公司 | Method and device for determining ball cake position, electronic equipment and readable storage medium |
| CN113066003B (en) * | 2021-03-12 | 2024-03-29 | 深圳市唯特视科技有限公司 | Method and device for generating panoramic image, electronic equipment and storage medium |
| CN113066003A (en) * | 2021-03-12 | 2021-07-02 | 深圳市唯特视科技有限公司 | Panoramic image generation method and device, electronic equipment and storage medium |
| CN115379122B (en) * | 2022-10-18 | 2023-01-31 | 鹰驾科技（深圳）有限公司 | Video content dynamic splicing method, system and storage medium |
| CN115379122A (en) * | 2022-10-18 | 2022-11-22 | 鹰驾科技（深圳）有限公司 | Video content dynamic splicing method, system and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107392848A (en) | Panoramic image display method and device | |
| CN104851076B (en) | Panoramic looking-around parking assisting system and camera installation method for commercial car | |
| CN109741455B (en) | Vehicle-mounted stereoscopic panoramic display method, computer readable storage medium and system | |
| CN108263283B (en) | Method for calibrating and splicing panoramic all-round looking system of multi-marshalling variable-angle vehicle | |
| CN112224132B (en) | Vehicle panoramic all-around obstacle early warning method | |
| CN103177439B (en) | A kind of automatic calibration method based on black and white lattice corners Matching | |
| CN110381255A (en) | Using the Vehicular video monitoring system and method for 360 panoramic looking-around technologies | |
| CN109948398A (en) | The image processing method and panorama parking apparatus that panorama is parked | |
| CN108805801A (en) | A kind of panoramic picture bearing calibration and system | |
| CN106060493A (en) | Multi-source projection seamless edge stitching method and system | |
| CN105354796B (en) | Image processing method and system for auxiliary of driving a vehicle | |
| CN106846410A (en) | Based on three-dimensional environment imaging method and device | |
| CN103136720A (en) | Vehicle-mounted 360-degree panorama mosaic method | |
| CN102164274B (en) | Vehicle-mounted virtual panoramic system with variable field of view | |
| CN103871071A (en) | Method for camera external reference calibration for panoramic parking system | |
| CN110288527B (en) | Panoramic aerial view generation method of vehicle-mounted panoramic camera | |
| CN102881016A (en) | Vehicle 360-degree surrounding reconstruction method based on internet of vehicles | |
| CN105825475A (en) | 360-degree panorama image generation method based on single pick-up head | |
| CN106994936A (en) | A kind of 3D panoramic parking assist systems | |
| CN110443771B (en) | Method for adjusting consistency of brightness and color of annular view in vehicle-mounted annular view camera system | |
| CN113362228A (en) | Method and system for splicing panoramic images based on improved distortion correction and mark splicing | |
| CN109087251A (en) | A kind of vehicle-mounted panoramic image display method and system | |
| JP2010018102A (en) | Driving support device | |
| CN103295231A (en) | Method for geometrically correcting vertically mapped images of fisheye lenses in fisheye image mosaic | |
| JP4872890B2 (en) | Image distortion correction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171124 |
|
| RJ01 | Rejection of invention patent application after publication |