CN101814181A - Unfolding method for restoration of fisheye image - Google Patents

Unfolding method for restoration of fisheye image Download PDF

Info

Publication number
CN101814181A
CN101814181A CN 201010125523 CN201010125523A CN101814181A CN 101814181 A CN101814181 A CN 101814181A CN 201010125523 CN201010125523 CN 201010125523 CN 201010125523 A CN201010125523 A CN 201010125523A CN 101814181 A CN101814181 A CN 101814181A
Authority
CN
China
Prior art keywords
image
fish eye
coordinate
pixel
unfolded
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 201010125523
Other languages
Chinese (zh)
Other versions
CN101814181B (en
Inventor
曹作良
刘庆杰
王颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University of Technology
Original Assignee
Tianjin University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University of Technology filed Critical Tianjin University of Technology
Priority to CN2010101255230A priority Critical patent/CN101814181B/en
Publication of CN101814181A publication Critical patent/CN101814181A/en
Application granted granted Critical
Publication of CN101814181B publication Critical patent/CN101814181B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The invention provides an unfolding method for restoration of a fisheye image, and relates to the restoration and correction of a fisheye image based on the equidistant projection principle of a spherical model. The method comprises the following steps of: establishing a spherical imaging model, consistent with the equidistant projection principle, of the fisheye image(including the establishment of a fisheye image coordinate system and a camera coordinate system); establishing an unfolded window coordinate system and an unfolded image coordinate system, and calculating the relationship between the two coordinate systems; establishing the relationship between the unfolded image coordinate system and the camera coordinate system; calculating the relationship between the unfolded image coordinate system and the fisheye image coordinate system; and finally obtaining a whole image unfolded in any specified direction. The method can overcome the defect of the methods that can unfold only the top part information of fisheye lenses and can not only restore the top part information, but also restore all information in any direction, The method can be applied to the equidistant projection principle based fisheye lenses as well as other projection principle based fisheye lenses, so that the method has a wide application prospect in fields of image tracking, monitoring, video conference and the like.

Description

A kind of method of deploying of restoration of fisheye image
[technical field]:
The present invention relates to recovery and correction, and, belong to technical field of image processing with fish-eye any image stretching that incident ray is an optical axis based on the fish eye images of the equidistant projection principle of sphere model.
[background technology]:
Omni-directional visual is meant and is equipped with the piece image that video camera once absorbs that super large wide-angle fish eye lens is placed straight up, can write down the whole scenes in the visual angle, 360 ° of orientation, promptly around scenery " one soon all over " in the whole all around zone.The machinery rotation that need not the separate unit video camera that obtains of this " panoramic picture " carries out " shaking bat ", or adopt multiple cameras to take multiple image simultaneously and carry out " stickup ", thereby simplified system architecture and handling procedure, helped the real-time requirement of machine vision applications.It is caught and application scenarios such as camera and object interaction in maneuvering target tracking, multiple goal, has special advantage and important use and is worth.This panorama observation with super large wide-angle view can remedy the deficiency of narrow viewing angle, at safety monitoring, pipeline detection, driver assistance, field monitoring, vehicle-mountedly patrol and examine, all there is direct or potential application prospect aspects such as aircraft guidance and robot for space.
The corner connection of fish eye lens camera field of view is near even surpass 180 degree, can realize closely or the stereoscopic vision perception of the big visual field of super close distance.Fish eye lens is to set up one of effective method of fully-directional visual system, and with respect to reflective omni-directional visual camera lens, fish eye lens has advantages such as compact conformation, volume be less, not fragile.The fish eye lens field angle meets or exceeds 180 degree usually, and single lens can be caught the hemisphere image, uses two fish eye lenses then can once obtain the complete image in whole space.Adopt single fish eye lens and image acquisition card structure fully-directional visual system, its relevant rudimentary theory is carried out preliminary exploration, to a certain extent research is carried out in its application.The Flame Image Process problem of omni-directional visual in the main research image tracking system.
Because the particular design of fish-eye camera, focal length is short more, and the visual angle is big more; The visual angle is big more, because of the distortion that optical principle produces also just strong more, the captured image of fish eye lens remains unchanged except the scenery at picture center, other should level or vertical scenery corresponding variation has all taken place, there is serious twisted phenomena in the omnidirectional images that obtains of flake video camera like this.To effectively utilize the image information tracking of navigating, need to recover go back original image.Study fish-eye imaging mechanism, fish eye lens projection imaging rule, the pattern distortion origin cause of formation, and then develop a kind of algorithm, make fish eye images obtain in real time, recover accurately.Can accurately proofread and correct the image of any range in the fish eye images, and then realize the reconstructing restored of three-dimensional spatial information.
[summary of the invention]:
The purpose of this invention is to provide a kind of fish eye images method of deploying that restores according to the equidistant projection principle based on the sphere imaging model, restore the image of any range in the fish eye images accurately with realization, and then splice correction and the recovery that realizes the distortion fish eye images; Can launch simultaneously the pairing fish eye images of expansion window at the arbitrary orientation angle, space, fish eye lens place and the elevation angle.
The present invention is a kind of method of deploying of the fish eye images based on sphere equidistant projection model, and it is realized on following hardware carrier: PC, flake video camera, support.The flake video camera is made up of super large wide-angle fish eye lens and high resolution camera.Support is used for supporting and fixing flake camera; The flake camera is gathered the spatial information of structured environment in real time, and PC launches to handle by method of the present invention in real time, utilizes the information after launching can carry out other effective processing, as car body navigation, monitoring, video conference etc.
The present invention is a kind of method of deploying of the fish eye images based on sphere equidistant projection model, makes fixed fish eye lens realize being equivalent to the function of mobile camera.Its implementation flow process as shown in Figure 2, its concrete steps are as follows:
1st, set up the sphere imaging model of square projection theories such as meeting of fish eye images;
The the 1.1st: set up the fish eye images coordinate system.Center o (x with fish eye images 0, y 0) set up fish eye images coordinate system xoy for initial point.
The the 1.2nd: set up camera coordinates system.Camera coordinates is x cy cz cIt is a three-dimensional system of coordinate.x c, y cOverlap z with the x axle y axle of fish eye images coordinate system respectively cAxle and fish-eye optical axis coincidence, and meet at the O point with the fish eye images coordinate system.
The 1.3rd: with the O point is the center, and the radius R of fish eye images is that radius is done hemisphere, hemisphere and z cThe axle meet at O ' point (0,0, R).
The the 1.4th: suppose 1 P in the camera coordinates system, be ray OP, meet at P ' (x with hemisphere face Cp ', y Cp ', z Cp ') point, the coordinate of this intersection point in camera coordinates system is:
(x cp′,y cp′,z cp′)=(Rsinθcosφ,Rsinθsinφ,Rcosθ) (1)
Wherein, OP and z cThe angle theta of axle is an incident angle; The projection OP of OP on xoy " is defined as the position angle with the included angle of x axle; The elevation angle of OP is β=90 °-θ.Then according to waiting square projection theory:
||OP″||=Kθ (2)
Wherein k is for waiting distortion factor of square projection.
2nd, set up expansion window coordinates system and unfolded image coordinate system, and obtain the relation between these two coordinate systems.
The the 2.1st: cross the tangent plane U that P ' does sphere wV w, this plane is and launches window coordinates is the plane at place, the visual angle of launching window coordinates system is γ; P ' V wWith OZ, OP ' coplane and its positive dirction are perpendicular to OP ' sensing OZ; P ' U wPositive dirction by P ' V wCross product decision with OP '; The position angle that defines this tangent plane is the position angle φ of OP, and the elevation angle is same as the elevation angle of OP:
β=90°-θ (3)
2.2nd, the unfolded image coordinate is U wV w, unfolded image coordinate system U wV wInitial point (0,0) in the lower left corner of unfolded image; If the wide and height of unfolded image is respectively w, h.
2.3rd, launch the point that window coordinates fasten and have one-to-one relationship with the pixel on the unfolded image coordinate system, so a pixel wide on the unfolded image corresponding to the size on the expansion plane is:
w pixel = R × 2 × tan ( β 2 ) / w - - - ( 4 )
Pixel coordinate on the unfolded image and the point coordinate that launches on the plane can be mapped one by one according to the 2.2nd; As the point coordinate on the unfolded image is that (m, n), the then corresponding coordinate launching on the window plane is Q ' (mw to Q Pixel, nw Pixel);
3rd, set up the relation of unfolded image coordinate system and camera coordinates system.
The the 3.1st: set up the relation of launching window coordinates system and camera coordinates system.Launching window coordinates system along P ' U wThe whenever mobile w of positive dirction PixelThe time variable quantity fastened in camera coordinates be:
X direction variable quantity
dxu=-w pixel*sin(φ) (5)
Y direction variable quantity
dyu=w pixel*cos(φ) (6)
The variable quantity of z direction is
dzu=0 (7)
In like manner launching window coordinates system along P ' V wThe whenever mobile w of positive dirction PixelThe time variable quantity fastened in camera coordinates be:
At x direction variable quantity
dxv=w pixel*cos(θ)*cos(φ) (8)
Y direction variable quantity is
dyv=w pixel*cos(θ)*sin(φ) (9)
Variable quantity on the z direction is
dzv=w pixel*sin(θ) (10)
The 3.2nd: obtain unfolded image mid point Q (m, n) the corresponding coordinate Q in camera coordinates system ":
(x cp+mdxu+ndxv,y cp+mdyu+ndyv,z cp+mdzu+ndzv)
=R×w pixel×(sinθcosφ-msinφ+ncosθcosφ (11)
,sinθsin+mcosφ+ncosθsinφ
,cosθ+nsinθ)
4th, obtain relation between unfolded image coordinate system and the fish eye images coordinate system.
4.1st, we have obtained the coordinate Q of some Q coordinate in camera coordinates system in the unfolded image in the 3rd step (11) formula ".We can obtain Q " the imaging Q in fish eye images " ' according to the equidistant projection principle.Q " incident angle be:
α = tan - 1 ( dis z ) = tan - 1 x 2 + y 2 z
(12)
= tan - 1 ( - m sin φ + n cos θ cos φ ) 2 + ( m cos φ + n cos θ sin φ ) 2 n sin θ
The position angle is:
λ = tan - 1 y x = tan - 1 m cos φ + n cos θ sin φ - m sin φ + n cos θ cos φ (13)
With getting: Q according to (2) formula equidistant projection principle " ' be r=k α to the length of central point.So Q " ' coordinate be:
x Q″′=rcosλ+x 0
(14)
y Q″′=rsinλ+y 0
X wherein 0, y 0Be the centre coordinate of fish eye images, with α and λ substitution can try to achieve any point Q that recovers on the image of back (m, n) corresponding point coordinate on fish eye images:
Q″′(rcosλ+x 0,rsinλ+y 0) (15)
4.2nd, the " ' (rcos λ+x that will put Q 0, rsin λ+y 0) pixel value compose that (m n), draws unfolded image after the group to Q.
ValueQ(m,n)=ValueQ″′(rcosλ+x 0,rsinλ+y 0) (16)
5th, pointwise obtains whole unfolded images
According to the method in the 2nd to the 4th step, pointwise obtains the image of each point in the unfolded image, obtains whole unfolded images at last.
6th, the expansion of fish eye lens place space three-dimensional scene.
At first, according to the cube strategy, as launching window, the definition on these several planes is as follows respectively with hemispherical external half cubical five faces in the 1st step model:
6.1st, fish-eye top: get sphere top and z cThe axle intersection point (0,0, R) cook tangent plane P UpCan recover the image at fish eye lens top for launching window.The size of launching window is 2R * 2R (wide * height), and the visual angle is 90 °, and no position angle, the elevation angle are 90 °.Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (16) formula.
6.2nd, fish-eye the place ahead: get sphere and y cThe intersection point (0, R, 0) of axle is cooked tangent plane P FrontFor the expansion window in the place ahead can recover the image in fish eye lens the place ahead, the size on plane is 2R * R, and the visual angle is 90 °, and the position angle is 90 °, and the elevation angle is 0 °.Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (16) formula.
6.3rd, fish-eye rear: get sphere y cThe intersection point (0 ,-R, 0) of axle is cooked tangent plane P BackFor the expansion window at rear can recover the image at fish eye lens rear, the size on plane is 2R * R, and the visual angle is 90 °, and the position angle is 270 °, and the elevation angle is 0 °.Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (16) formula.
6.4th, the fish-eye left side: get sphere and x cThe intersection point (R, 0,0) of axle is cooked tangent plane P LeftFor the expansion window on the left side can recover the image on the fish eye lens left side, the size on plane is R * 2R, and the visual angle is 90 °, and the position angle is 180 °, and the elevation angle is 0 °.Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (16) formula.
6.5th, the fish-eye right side: get sphere and x cThe intersection point (R, 0,0) of axle is cooked tangent plane P RightFor the expansion window on the right side can recover the image on the fish eye lens right side, the size on plane is R * 2R, and the visual angle is 90 °, and the position angle is 0 °, and the elevation angle is 0 °.Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (16) formula.
7th, specify the unfolded image of any direction.
With any one direction in the space, fish eye lens place is the center, and fish eye images is launched, and the expansion of any direction needs 5 parameters, the elevation angle, and the position angle, the wide and high of back image recovered at the visual angle.Given these 5 parameters can be launched according to the 1st step to the 4th step.
8th, launch the roaming of window.
To launch window and on sphere, move, and launch, can obtain the roaming figure of fish eye images according to the 1st method that went on foot for the 5th step.
Advantage of the present invention and good effect:
● the present invention is based on fish-eye sphere imaging model, promptly be adapted to wait the fish eye lens of square projection theory, also be applicable to the fish eye lens of other projection theory.
● the present invention has remedied other can only launch the defective of fish eye lens top information approach, not only can restore the information at top, also can restore the information of side any direction, as two sides, the left and right sides.
● the present invention can launch " the expansion window " on any direction axle, and the extracting of sensitive information in the image is had good effect.
● expansion effect of the present invention has been eliminated the existing picture that other restoration methods edge " is pullled ", also can obtain the effect near real world at the edge.
● the present invention is having positive effect and application prospects based on fields such as fish-eye image tracking, monitoring and video conferences.
[description of drawings]:
Fig. 1 is a sphere equidistant projection model synoptic diagram.
Fig. 2 is the process flow diagram of the method for the invention.
Fig. 3 is the stretch-out view that " back " launches window.
Fig. 4 is that position angle and incident angle are 45 ° stretch-out view.
Fig. 5 is cube strategy stretch-out view and splicing.
[embodiment]:
Below in conjunction with drawings and Examples the multidimensional method of deploying that fish eye lens of the present invention restores is elaborated.
Embodiment 1
Image resolution ratio is 1280 * 1024, and the center of fish eye images is (504,508), and calibrated radius is R=498, and distortion is k=5.9.Given position angle is φ=45 °, elevation angle β=45 °, and visual angle γ=90 °, recovering back image size requirements is 400 * 400.
The first step: set up fish eye images coordinate system xy, camera coordinates is x cy cz c, the initial point of coordinate system is the central point (504,508) of fish eye images; Do the hemisphere face of radius R=498, hemisphere face and z cIntersection point be (0,0,498).
Second step: obtain 45 ° at position angle, the ray at 45 ° at the elevation angle and the point of intersection S of sphere:
S(R?sin(45)*cos(45),R?sin(45)*sin(45),R?cos(45))=S(249.0,249.0,352.086)。
The 3rd step: S selects and cooks tangent plane P excessively SFor launching window, the visual angle is 90 °, and coordinate axis is U S, V SU SPositive dirction point to x cThe positive dirction of axle, V SPositive dirction point to z cPositive dirction, initial point is S, so the S point is at P SOn coordinate be S (0,0).Calculate on the unfolded image pixel corresponding to P SOn size be:
w pixel = R × 2 × tan ( β 2 ) / w = 2.49 .
The 4th step: calculate P SOn whenever mobile w PixelThe variation that in camera coordinates system, causes.
No U SPositive dirction moves w PixelThe time in camera coordinates system, cause be changed to:
x cDirection: dxu=w PixelSin φ=1.760
y cDirection: dyu=-w PixelCos φ=-1.760
z cDirection: dzu=0
Along V SPositive dirction moves w PixelThe time in camera coordinates system, cause be changed to:
x cDirection: dxv=-w Pixel* cos (θ) * cos (φ)=-1.245
y cDirection: dyv=-w Pixel* cos (θ) * sin (φ)=-1.245
z cDirection: dzv=w Pixel* sin (θ)=1.760
The 5th step: set up the unfolded image coordinate system.Launching back image size is 400 * 400, the initial point of coordinate system is located at the center (200,200) of image, so first pixel coordinate on the image is (200 ,-200).
The 6th step: calculate (200 ,-200) at P SOn coordinate.According to formula (11) be:
(-200w pixel,-200w pixel)=(-498,-498)
The 7th step: calculate (200 ,-200) coordinate in camera coordinates system.Calculate according to the 3rd in summary of the invention step.
x=x s-200×dxu+(-200)×dxv=249.0+103.0=352.0
y=y s-200×dyu+(-200)×dyv=249.0+601.0=850.0
z=z s-200×dxu+(-200)×dxv=352.086-352=0.086
The 8th step: according to waiting square projection theory to calculate the coordinate that is projected on the fish eye images.According to the calculating of the step of the 4th in the summary of the invention.
Incident angle
The position angle
Figure GSA00000055311200072
The square that arrives mid point is from r=k α=5.9 * 90=531
So coordinates computed is x Q " '=rcos λ+x 0=531 * cos67.5+504=203.2+504=707
y Q″′=rsinλ+y 0=531×sin67.5+508=490.6+508=999
To just having calculated in the unfolded image first pixel (0,0) here corresponding to the coordinate in the fish eye images.
The 9th step: other pixel is for the coordinate in the fish eye images in the calculating unfolded image, and the rgb value of this point is (109,204,63), so the value of first pixel on the unfolded image is (109,204,63) just.Coordinate pointwise in the unfolded image is calculated, just can obtain whole unfolded image.Fig. 4 is image and original image after launching.
Fig. 3 is the stretch-out view that " back " launches window.Fig. 5 is cube strategy stretch-out view and splicing.
By the specific descriptions of above embodiment, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential hardware platform, can certainly pass through hardware, but the former is better embodiment under a lot of situation.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in the storage medium, comprises that some instructions are in order to realize method of the present invention.
More than disclosed only be a specific embodiment of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.

Claims (3)

1. method of deploying based on the restoration of fisheye image of sphere equidistant projection model is characterized in that the concrete steps of this method are as follows:
1st, set up the sphere imaging model of square projection theories such as meeting of fish eye images
1.1st, set up the fish eye images coordinate system: with the center o (x of fish eye images 0, y 0) set up fish eye images coordinate system xoy for initial point;
1.2nd, set up camera coordinates system: camera coordinates is x cy cz cIt is a three-dimensional system of coordinate; x c, y cOverlap z with x axle, the y axle of fish eye images coordinate system respectively cAxle and fish-eye optical axis coincidence, and meet at the O point with the fish eye images coordinate system;
1.3rd, be the center with the O point, the radius R of fish eye images is that radius is done hemisphere, hemisphere and z cThe axle meet at O ' point (0,0, R);
1.4th, 1 P in the camera coordinates system is ray OP, meets at P ' (x with hemisphere face Cp ', y Cp ', z Cp ') point, the coordinate of this intersection point in camera coordinates system is:
(x cp′,y cp′,z cp′)=(Rsinθcosφ,Rsinθsinφ,Rcosθ) (1)
Wherein, OP and z cThe angle theta of axle is an incident angle; The projection OP of OP on xoy " is defined as the position angle with the included angle of x axle; The elevation angle of OP is β=90 °-θ, then according to waiting square projection theory:
‖OP″‖=kθ (2)
Wherein k is for waiting distortion factor of square projection;
2nd, set up expansion window coordinates system and unfolded image coordinate system, and obtain the relation between these two coordinate systems
2.1st, cross the tangent plane U that P ' does sphere wV w, this plane is and launches window coordinates is the plane at place, the visual angle of launching window coordinates system is γ; P ' V wWith OZ, OP ' coplane and its positive dirction are perpendicular to OP ' sensing OZ; P ' U wPositive dirction by P ' V wCross product decision with OP '; The position angle that defines this tangent plane is the position angle φ of OP, and the elevation angle is same as the elevation angle of OP:
β=90°-θ (3);
2.2nd, the unfolded image coordinate is UwVw, and the initial point (0,0) of unfolded image coordinate system UwVw is in the lower left corner of unfolded image; If the wide and height of unfolded image is respectively w, h;
2.3rd, launch the point that window coordinates fasten and have one-to-one relationship with the pixel on the unfolded image coordinate system, so a pixel wide on the unfolded image corresponding to the size on the expansion plane is:
w pixel = R × 2 × tan ( β 2 ) / w - - - ( 4 )
Pixel coordinate on the unfolded image and the point coordinate that launches on the plane can be mapped one by one according to the 2.2nd; As the point coordinate on the unfolded image is that (m, n), the then corresponding coordinate launching on the window plane is Q ' (mw to Q Pixel, nw Pixel);
3rd, set up the relation of unfolded image coordinate system and camera coordinates system
3.1st, set up the relation of launching window coordinates system and camera coordinates system: launching window coordinates system along P ' U wThe whenever mobile w of positive dirction PixelThe time variable quantity fastened in camera coordinates be:
X direction variable quantity
dxu=-w pixel*sin(φ) (5)
Y direction variable quantity
dyu=w pixel*cos(φ) (6)
The variable quantity of z direction is
dzu=0 (7)
In like manner launching window coordinates system along P ' V wThe whenever mobile w of positive dirction PixelThe time variable quantity fastened in camera coordinates be:
At x direction variable quantity
dxv=w pixel*cos(θ)*cos(φ) (8)
Y direction variable quantity is
dyv=w pixel*cos(θ)*sin(φ) (9)
Variable quantity on the z direction is
dzv=w pixel*sin(θ) (10)
3.2nd, obtain unfolded image mid point Q (m, n) the corresponding coordinate Q in camera coordinates system ":
(x cp+mdxu+ndxv,y cp+mdyu+ndyv,z cp+mdzu+ndzv)
=R×w pixel×(sinθcosφ-msinφ+ncosθcosφ(11)
,sinθsinφ+mcosφ+ncosθsinφ
,cosθ+nsinθ)
4th, obtain relation between unfolded image coordinate system and the fish eye images coordinate system
4.1st, in the 3rd step formula (11), obtained the coordinate Q of some Q coordinate correspondence in camera coordinates system in the unfolded image ", can obtain Q " the imaging Q ' in fish eye images " according to the equidistant projection principle; Q " incident angle be:
α = tan - 1 ( dis z ) = tan - 1 x 2 + y 2 z
= tan - 1 ( - m sin φ + n cos θ cos φ ) 2 + ( m cos φ + n cos θ sin φ ) 2 n sin θ - - - ( 12 ) The position angle is:
λ = tan - 1 y x = tan - 1 m cos φ + n cos θ sin φ - m sin φ + n cos θ cos φ - - - ( 13 )
" length to central point is r=k α according to equidistant projection principle (2) Shi Kede: Q '; So Q ' " coordinate be:
x Q ′ ′ ′ = r cos λ + x 0 (14)
y Q′″=rsinλ+y 0
X wherein 0, y 0Be the centre coordinate of fish eye images, with α and λ substitution can try to achieve any point Q that recovers on the image of back (m, n) corresponding point coordinate on fish eye images:
Q″′(rcosλ+x 0,rsinλ+y 0) (15)
4.2nd, will put Q ' " (rcos λ+x 0, rsin λ+y 0) pixel value compose that (m n), draws unfolded image after the group to Q;
ValueQ(m,n)=ValueQ′″(rcosλ+x 0,rsinλ+y 0) (16)
5th, obtain whole unfolded images
According to the method in the 2nd to the 4th step, pointwise obtains the image value of each point in the unfolded image, obtains whole unfolded images at last.
2. method according to claim 1 is characterized in that obtaining the unfolded image of fish eye lens place space three-dimensional scene
6th, according to the cube strategy, as launching window, the definition on these several planes is as follows respectively with hemispherical external half cubical five faces in the 1st step model:
6.1st, fish-eye top: get sphere top and z cThe axle intersection point (0,0, R) cook tangent plane P UpCan recover the image at fish eye lens top for launching window; The size of launching window be 2R wide * the 2R height, the visual angle is 90 °, no position angle, the elevation angle are 90 °; Try to achieve relation between unfolded image and the fish eye images according to the 4th step (15) formula;
6.2nd, fish-eye the place ahead: get sphere and y cThe intersection point (0, R, 0) of axle is cooked tangent plane P FrontFor the expansion window in the place ahead can recover the image in fish eye lens the place ahead, the size on plane is 2R * R, and the visual angle is 90 °, and the position angle is 90 °, and the elevation angle is 0 °; Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (16);
6.3rd, fish-eye rear: get sphere y cThe intersection point (0 ,-R, 0) of axle is cooked tangent plane P BackFor the expansion window at rear can recover the image at fish eye lens rear, the size on plane is 2R * R, and the visual angle is 90 °, and the position angle is 270 °, and the elevation angle is 0 °; Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (15) formula;
6.4th, the fish-eye left side: get sphere and x cThe intersection point (R, 0,0) of axle is cooked tangent plane P LeftFor the expansion window on the left side can recover the image on the fish eye lens left side, the size on plane is R * 2R, and the visual angle is 90 °, and the position angle is 180 °, and the elevation angle is 0 °; Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (15) formula;
6.5th, the fish-eye right side: get sphere and x cThe intersection point (R, 0,0) of axle is cooked tangent plane P RightFor the expansion window on the right side can recover the image on the fish eye lens right side, the size on plane is R * 2R, and the visual angle is 90 °, and the position angle is 0 °, and the elevation angle is 0 °; Try to achieve relation between unfolded image and the fish eye images according to the method pointwise of the 4th step described in (15) formula.
3. method according to claim 1 is characterized in that can obtaining specifying the unfolded image of any direction
With any one direction in the space, fish eye lens place is the center, and fish eye images is launched, and the expansion of any direction needs 5 parameters, the elevation angle, and the position angle, the wide and high of back image recovered at the visual angle; Given these 5 parameters can be launched according to the 1st step to the 4th step.
CN2010101255230A 2010-03-17 2010-03-17 Unfolding method for restoration of fisheye image Expired - Fee Related CN101814181B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2010101255230A CN101814181B (en) 2010-03-17 2010-03-17 Unfolding method for restoration of fisheye image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010101255230A CN101814181B (en) 2010-03-17 2010-03-17 Unfolding method for restoration of fisheye image

Publications (2)

Publication Number Publication Date
CN101814181A true CN101814181A (en) 2010-08-25
CN101814181B CN101814181B (en) 2012-05-23

Family

ID=42621429

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010101255230A Expired - Fee Related CN101814181B (en) 2010-03-17 2010-03-17 Unfolding method for restoration of fisheye image

Country Status (1)

Country Link
CN (1) CN101814181B (en)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102072702A (en) * 2010-12-31 2011-05-25 天津市亚安科技电子有限公司 Method for calculating corresponding positions of images of two cameras
CN103247030A (en) * 2013-04-15 2013-08-14 丹阳科美汽车部件有限公司 Fisheye image correction method of vehicle panoramic display system based on spherical projection model and inverse transformation model
CN103325109A (en) * 2013-05-29 2013-09-25 山西绿色光电产业科学技术研究院(有限公司) Method used for correcting distortion of fish-eye image and applicable to wall-mounted panoramic video camera
CN104135608A (en) * 2014-05-21 2014-11-05 张家港恩达通讯科技有限公司 Flattening method and system for fish-eye lens image
CN104363421A (en) * 2014-11-04 2015-02-18 福建星网视易信息***有限公司 Method and device for realizing multi-angle camera monitoring effect
CN104539893A (en) * 2014-12-23 2015-04-22 福建星网视易信息***有限公司 Method, device and system for achieving virtual PTZ monitoring effect
CN105007411A (en) * 2015-06-30 2015-10-28 广东欧珀移动通信有限公司 Distortion correction method and mobile terminal
CN105516569A (en) * 2016-01-20 2016-04-20 北京疯景科技有限公司 Method and device for obtaining omni-directional image
CN105898344A (en) * 2016-04-12 2016-08-24 乐视控股(北京)有限公司 Method and device for playing panoramic video
CN105898339A (en) * 2016-04-12 2016-08-24 乐视控股(北京)有限公司 Method and device for transcoding panoramic video
CN106101588A (en) * 2016-07-08 2016-11-09 成都易瞳科技有限公司 The compensation method of panoramic picture gradual halation phenomena
CN104200454B (en) * 2014-05-26 2017-02-01 深圳市中瀛鑫科技股份有限公司 Fisheye image distortion correction method and device
CN106600546A (en) * 2016-11-14 2017-04-26 Tcl集团股份有限公司 Ultra-wide angle camera distortion correction method and system
CN106791578A (en) * 2016-12-26 2017-05-31 西安电子科技大学 Video meeting implementing method based on flake panoramic camera
CN106780389A (en) * 2016-12-23 2017-05-31 浙江宇视科技有限公司 A kind of fisheye image correcting method and device based on Coordinate Conversion
CN107133918A (en) * 2017-05-05 2017-09-05 陈健 A kind of method that optional position in three-dimensional scenic generates panorama sketch
CN107154027A (en) * 2017-04-17 2017-09-12 深圳大学 Compensation method and device that a kind of fault image restores
CN107346530A (en) * 2016-05-06 2017-11-14 完美幻境(北京)科技有限公司 A kind of projecting method and system for correcting fish eye images
CN107767422A (en) * 2017-09-18 2018-03-06 深圳开阳电子股份有限公司 A kind of fish-eye bearing calibration, device and portable terminal
CN107993276A (en) * 2016-10-25 2018-05-04 杭州海康威视数字技术股份有限公司 The generation method and device of a kind of panoramic picture
WO2018102990A1 (en) * 2016-12-06 2018-06-14 SZ DJI Technology Co., Ltd. System and method for rectifying a wide-angle image
CN108805801A (en) * 2018-05-24 2018-11-13 北京华捷艾米科技有限公司 A kind of panoramic picture bearing calibration and system
CN109308686A (en) * 2018-08-16 2019-02-05 北京市商汤科技开发有限公司 A kind of fish eye images processing method and processing device, equipment and storage medium
CN110021044A (en) * 2018-01-10 2019-07-16 华晶科技股份有限公司 The method and image acquiring device of taken the photograph object coordinates are calculated using double fish eye images
CN111091117A (en) * 2019-12-31 2020-05-01 北京城市网邻信息技术有限公司 Target detection method, device, equipment and medium for two-dimensional panoramic image
CN112530173A (en) * 2020-12-03 2021-03-19 北京百度网讯科技有限公司 Roadside sensing method and device, electronic equipment, storage medium and roadside equipment
CN112541861A (en) * 2019-09-23 2021-03-23 华为技术有限公司 Image processing method, device, equipment and computer storage medium
CN112565730A (en) * 2020-12-03 2021-03-26 北京百度网讯科技有限公司 Roadside sensing method and device, electronic equipment, storage medium and roadside equipment
CN113344788A (en) * 2021-06-17 2021-09-03 天津理工大学 Panoramic camera based on image stitching parameter extraction and real-time parameter optimization method thereof
CN115514944A (en) * 2022-09-21 2022-12-23 南京创斐信息技术有限公司 Intelligent household projection angle correction system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006295626A (en) * 2005-04-12 2006-10-26 Canon Inc Fish-eye image processing apparatus, method thereof and fish-eye imaging apparatus
CN101072288A (en) * 2007-06-15 2007-11-14 林进灯 Method for obtaining fish-eye image correction relationship and fish-eye correction
US20090141148A1 (en) * 2007-12-04 2009-06-04 Kosuke Imamura Distortion-corrected image generation unit and distortion-corrected image generation method
CN101577002A (en) * 2009-06-16 2009-11-11 天津理工大学 Calibration method of fish-eye lens imaging system applied to target detection

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006295626A (en) * 2005-04-12 2006-10-26 Canon Inc Fish-eye image processing apparatus, method thereof and fish-eye imaging apparatus
CN101072288A (en) * 2007-06-15 2007-11-14 林进灯 Method for obtaining fish-eye image correction relationship and fish-eye correction
US20090141148A1 (en) * 2007-12-04 2009-06-04 Kosuke Imamura Distortion-corrected image generation unit and distortion-corrected image generation method
CN101577002A (en) * 2009-06-16 2009-11-11 天津理工大学 Calibration method of fish-eye lens imaging system applied to target detection

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《云南民族大学学报(自然科学版)》 20040731 陈明伟 等 球面坐标定位校正鱼眼图片并合成全景图的方法 214-217 1-3 第13卷, 第3期 2 *
《电子测量技术》 20080930 秦洁 等 基于DSP的SVM算法实现鱼眼镜头畸变校正 154-184 1-3 第31卷, 第9期 2 *

Cited By (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102072702A (en) * 2010-12-31 2011-05-25 天津市亚安科技电子有限公司 Method for calculating corresponding positions of images of two cameras
CN102072702B (en) * 2010-12-31 2015-11-18 天津市亚安科技股份有限公司 A kind of computational methods of twin camera image correspondence position
CN103247030A (en) * 2013-04-15 2013-08-14 丹阳科美汽车部件有限公司 Fisheye image correction method of vehicle panoramic display system based on spherical projection model and inverse transformation model
CN103325109A (en) * 2013-05-29 2013-09-25 山西绿色光电产业科学技术研究院(有限公司) Method used for correcting distortion of fish-eye image and applicable to wall-mounted panoramic video camera
CN103325109B (en) * 2013-05-29 2016-02-17 湖南源信光电科技有限公司 Be applicable to the distortion correction method of the fish eye images of wall-mounted type panoramic camera
CN104135608A (en) * 2014-05-21 2014-11-05 张家港恩达通讯科技有限公司 Flattening method and system for fish-eye lens image
CN104135608B (en) * 2014-05-21 2017-07-14 张家港恩达通讯科技有限公司 The flattening method and system of a kind of fisheye image
CN104200454B (en) * 2014-05-26 2017-02-01 深圳市中瀛鑫科技股份有限公司 Fisheye image distortion correction method and device
CN104363421A (en) * 2014-11-04 2015-02-18 福建星网视易信息***有限公司 Method and device for realizing multi-angle camera monitoring effect
CN104363421B (en) * 2014-11-04 2017-11-14 福建星网视易信息***有限公司 The method and apparatus for realizing Multi-angle camera monitoring effect
CN104539893A (en) * 2014-12-23 2015-04-22 福建星网视易信息***有限公司 Method, device and system for achieving virtual PTZ monitoring effect
CN104539893B (en) * 2014-12-23 2017-12-01 福建星网视易信息***有限公司 Realize the method, apparatus and system of virtual head monitoring effect
CN105007411A (en) * 2015-06-30 2015-10-28 广东欧珀移动通信有限公司 Distortion correction method and mobile terminal
CN105007411B (en) * 2015-06-30 2018-01-23 广东欧珀移动通信有限公司 The method and mobile terminal of a kind of distortion correction
CN105516569A (en) * 2016-01-20 2016-04-20 北京疯景科技有限公司 Method and device for obtaining omni-directional image
CN105898344A (en) * 2016-04-12 2016-08-24 乐视控股(北京)有限公司 Method and device for playing panoramic video
CN105898339A (en) * 2016-04-12 2016-08-24 乐视控股(北京)有限公司 Method and device for transcoding panoramic video
CN107346530B (en) * 2016-05-06 2020-09-04 完美幻境(北京)科技有限公司 Projection method and system for correcting fisheye image
CN107346530A (en) * 2016-05-06 2017-11-14 完美幻境(北京)科技有限公司 A kind of projecting method and system for correcting fish eye images
CN106101588B (en) * 2016-07-08 2019-05-14 成都易瞳科技有限公司 The compensation method of panoramic picture gradual halation phenomena
CN106101588A (en) * 2016-07-08 2016-11-09 成都易瞳科技有限公司 The compensation method of panoramic picture gradual halation phenomena
CN107993276A (en) * 2016-10-25 2018-05-04 杭州海康威视数字技术股份有限公司 The generation method and device of a kind of panoramic picture
US11330172B2 (en) 2016-10-25 2022-05-10 Hangzhou Hikvision Digital Technology Co., Ltd. Panoramic image generating method and apparatus
CN107993276B (en) * 2016-10-25 2021-11-23 杭州海康威视数字技术股份有限公司 Panoramic image generation method and device
CN106600546A (en) * 2016-11-14 2017-04-26 Tcl集团股份有限公司 Ultra-wide angle camera distortion correction method and system
CN106600546B (en) * 2016-11-14 2020-12-22 深圳市Tcl高新技术开发有限公司 Distortion correction method and system for ultra-wide-angle camera
US11195252B2 (en) 2016-12-06 2021-12-07 SZ DJI Technology Co., Ltd. System and method for rectifying a wide-angle image
WO2018102990A1 (en) * 2016-12-06 2018-06-14 SZ DJI Technology Co., Ltd. System and method for rectifying a wide-angle image
CN106780389A (en) * 2016-12-23 2017-05-31 浙江宇视科技有限公司 A kind of fisheye image correcting method and device based on Coordinate Conversion
CN106780389B (en) * 2016-12-23 2021-06-15 浙江宇视科技有限公司 Fisheye image correction method and device based on coordinate transformation
CN106791578A (en) * 2016-12-26 2017-05-31 西安电子科技大学 Video meeting implementing method based on flake panoramic camera
CN107154027A (en) * 2017-04-17 2017-09-12 深圳大学 Compensation method and device that a kind of fault image restores
CN107154027B (en) * 2017-04-17 2020-07-07 深圳大学 Compensation method and device for restoration of distorted image
CN107133918B (en) * 2017-05-05 2020-06-12 陈健 Method for generating panorama at any position in three-dimensional scene
CN107133918A (en) * 2017-05-05 2017-09-05 陈健 A kind of method that optional position in three-dimensional scenic generates panorama sketch
CN107767422A (en) * 2017-09-18 2018-03-06 深圳开阳电子股份有限公司 A kind of fish-eye bearing calibration, device and portable terminal
CN110021044B (en) * 2018-01-10 2022-12-20 华晶科技股份有限公司 Method for calculating coordinates of shot object by using double-fisheye image and image acquisition device
CN110021044A (en) * 2018-01-10 2019-07-16 华晶科技股份有限公司 The method and image acquiring device of taken the photograph object coordinates are calculated using double fish eye images
CN108805801A (en) * 2018-05-24 2018-11-13 北京华捷艾米科技有限公司 A kind of panoramic picture bearing calibration and system
CN109308686B (en) * 2018-08-16 2022-06-24 北京市商汤科技开发有限公司 Fisheye image processing method, device, equipment and storage medium
CN109308686A (en) * 2018-08-16 2019-02-05 北京市商汤科技开发有限公司 A kind of fish eye images processing method and processing device, equipment and storage medium
WO2021057626A1 (en) * 2019-09-23 2021-04-01 华为技术有限公司 Image processing method, apparatus, device, and computer storage medium
CN112541861A (en) * 2019-09-23 2021-03-23 华为技术有限公司 Image processing method, device, equipment and computer storage medium
CN112541861B (en) * 2019-09-23 2024-05-24 华为技术有限公司 Image processing method, device, equipment and computer storage medium
CN111091117A (en) * 2019-12-31 2020-05-01 北京城市网邻信息技术有限公司 Target detection method, device, equipment and medium for two-dimensional panoramic image
CN112565730A (en) * 2020-12-03 2021-03-26 北京百度网讯科技有限公司 Roadside sensing method and device, electronic equipment, storage medium and roadside equipment
CN112530173A (en) * 2020-12-03 2021-03-19 北京百度网讯科技有限公司 Roadside sensing method and device, electronic equipment, storage medium and roadside equipment
CN112565730B (en) * 2020-12-03 2023-07-25 阿波罗智联(北京)科技有限公司 Road side sensing method and device, electronic equipment, storage medium and road side equipment
CN113344788A (en) * 2021-06-17 2021-09-03 天津理工大学 Panoramic camera based on image stitching parameter extraction and real-time parameter optimization method thereof
CN115514944A (en) * 2022-09-21 2022-12-23 南京创斐信息技术有限公司 Intelligent household projection angle correction system

Also Published As

Publication number Publication date
CN101814181B (en) 2012-05-23

Similar Documents

Publication Publication Date Title
CN101814181B (en) Unfolding method for restoration of fisheye image
CN101809993B (en) Methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof
US9225942B2 (en) Imaging surface modeling for camera modeling and virtual view synthesis
Nayar Catadioptric omnidirectional camera
Nayar Omnidirectional video camera
CN101004538B (en) Omnibearing vision sensor with no dead angle
US8798451B1 (en) Methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof
CN107660337A (en) For producing the system and method for assembled view from fish eye camera
CN106534670B (en) It is a kind of based on the panoramic video generation method for connecting firmly fish eye lens video camera group
CN107705252B (en) Method and system suitable for splicing, unfolding and correcting binocular fisheye image
JP4825971B2 (en) Distance calculation device, distance calculation method, structure analysis device, and structure analysis method.
CN104835117A (en) Spherical panorama generating method based on overlapping way
CN101271187A (en) Non-dead angle binocular solid all-directional vision sensing equipment
Gao et al. Dual-fisheye omnidirectional stereo
JP2016180866A (en) Aerial shoot device
CN108769569B (en) 360-degree three-dimensional panoramic observation system and method for unmanned aerial vehicle
RU2698479C2 (en) Shooting system, method and applications
WO2009017332A1 (en) Methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof
CN108737743B (en) Video splicing device and video splicing method based on image splicing
CN102915526A (en) Spherical panorama missed information restoring method based on complementary scenario model
CN108765292A (en) Image split-joint method based on the fitting of space triangular dough sheet
JP2011087319A (en) In-vehicle panorama camera system
Tan et al. Multiview panoramic cameras using a mirror pyramid
Kweon et al. Image-processing based panoramic camera employing single fisheye lens
Kweon Panoramic image composed of multiple rectilinear images generated from a single fisheye image

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20120523

Termination date: 20130317