CN107454326B - Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system - Google Patents

Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system Download PDF

Info

Publication number
CN107454326B
CN107454326B CN201710730399.2A CN201710730399A CN107454326B CN 107454326 B CN107454326 B CN 107454326B CN 201710730399 A CN201710730399 A CN 201710730399A CN 107454326 B CN107454326 B CN 107454326B
Authority
CN
China
Prior art keywords
image
panoramic
pixel
corrected
fisheye lens
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710730399.2A
Other languages
Chinese (zh)
Other versions
CN107454326A (en
Inventor
李鸿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Jiu'an Intelligent Technology Co ltd
Original Assignee
Guangzhou Jiuan Wulian Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Jiuan Wulian Technology Co ltd filed Critical Guangzhou Jiuan Wulian Technology Co ltd
Priority to CN201710730399.2A priority Critical patent/CN107454326B/en
Publication of CN107454326A publication Critical patent/CN107454326A/en
Application granted granted Critical
Publication of CN107454326B publication Critical patent/CN107454326B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Stereoscopic And Panoramic Photography (AREA)

Abstract

The invention belongs to the field of imaging, and aims to solve the problems of poor imaging effect precision and large deformation rate of a panoramic camera in the prior art, and discloses a method for panoramic shooting by using a fisheye lens, which comprises the following steps: step 1: acquiring a panoramic image through a fisheye lens and an image sensor; wherein, the distortion parameter of the fisheye lens is linear distortion; step 2: correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; and step 3: and (3) mapping the panoramic image corrected in the step (2) to a plane or a curved surface to obtain images under different application scenes. The invention also discloses a camera and a camera system, which have good imaging effect and low deformation rate and are suitable for various different display interfaces of fixed and mobile terminals.

Description

Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system
Technical Field
The invention relates to the field of imaging, in particular to a method for carrying out panoramic shooting by using a fisheye lens, a camera and a panoramic shooting system.
Background
At present, mobile video is rapidly developed, mobile devices (mobile phones, tablets and other forms of mobile devices) become a part of life of people, and meanwhile, the safety problem also becomes another great concern of people.
A fisheye camera is a device for taking a picture at a large angle, and a fisheye lens is a lens having a focal length of 16mm or less and a viewing angle close to or equal to 180 °. It is an extreme wide-angle lens, and the "fish-eye lens" is its common name. In order to maximize the angle of view of the lens, the front lens of the lens is short in diameter and is parabolic and convex toward the front of the lens, much like the fish eye, so called "fish-eye lens". The fisheye lens belongs to a special lens in an ultra-wide angle lens, and the visual angle of the fisheye lens is required to reach or exceed the range which can be seen by human eyes. Therefore, the fisheye lens is very different from the real world scene in human eyes, because the scene seen in real life is in a regular fixed form, and the picture effect generated by the fisheye lens is beyond the scope.
Just because of the advantage of wide-angle shooting of the fisheye camera, only one camera can monitor the whole field in a fixed environment, but the picture effect generated by the fisheye camera has the following problems: the resolution ratio is low, the deformation ratio is large, and the visual angle which can be accepted by human eyes cannot be accurately converted.
Disclosure of Invention
The invention aims to provide a method for panoramic shooting by using a fisheye lens, which has high resolution, small deformation rate and accurate display of various visual angles, and also discloses a camera and a panoramic shooting system which are suitable for the method.
In order to achieve the purpose, the invention provides the following technical scheme: a method for panoramic shooting by using a fisheye lens comprises the following steps:
step 1: acquiring a panoramic image through a fisheye lens and an image sensor; wherein, the distortion parameter of the fisheye lens is linear distortion;
step 2: correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; wherein, the correction parameters comprise optical center, magnification factor and mapping model parameters appointed by a user;
and step 3: and (3) mapping the panoramic image corrected in the step (2) to a plane or a curved surface to obtain images under different application scenes.
In the above method for performing panoramic photography using a fisheye lens, a specific method for correcting a panoramic image by using correction parameters in step 2 is as follows:
s31, determining the range of the image needing to be corrected according to the optical center and the magnification;
s32: determining the magnification factor of each pixel area in the range of the image needing to be corrected according to the mapping model parameters;
s33: grabbing pixels in each pixel area and filling the pixels into the pixel area to obtain an enlarged local image;
s34: and repeating S32-S33 to obtain a plurality of local images, and combining the plurality of local images to obtain the corrected panoramic image.
The invention also discloses a camera adopting the fisheye lens, which comprises the fisheye lens and an image sensor arranged in the fisheye lens, wherein the fisheye lens is arranged on the camera body
The distortion parameter of the fisheye lens is linear distortion;
the focal length of the lens is 0.82mm +/-3%; the lens angle range is 220 °.
The invention also discloses a panoramic shooting system adopting the camera with the fisheye lens, which comprises the camera adopting the fisheye lens and a controller, wherein the camera adopting the fisheye lens is connected with the controller, and the controller comprises:
a panoramic image acquisition unit: the system is used for controlling the fisheye lens and the image sensor to acquire a panoramic image;
panoramic image correction unit: the panoramic image correction device is used for correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; wherein, the correction parameters comprise optical center, magnification factor and mapping model parameters appointed by a user;
an image mapping unit: and the panoramic image correction unit is used for mapping the panoramic image corrected by the panoramic image correction unit to a plane or a curved surface to obtain images of different application scenes.
In the panoramic camera system of the camera, the image mapping unit comprises the following sub-units:
the range determining subunit is used for determining the range of the image to be corrected according to the optical center and the magnification;
magnification determination subunit: the magnification factor of each pixel area in the range of the image needing to be corrected is determined according to the mapping model parameters;
pixel fill sub-unit: and capturing the pixels in each pixel area and filling the pixels into the pixel area to obtain an enlarged partial image.
In the panoramic imaging system of the camera, the system further includes a display, and the display is used for displaying the images of different application scenes obtained through the processing of the image mapping unit.
The controller acquires the control signal sent by the mobile terminal through the communication unit and sends images of different application scenes to the mobile terminal for display.
Compared with the prior art, the invention has the beneficial effects that: the invention can display the panoramic image on different display interfaces by adopting a specific fisheye lens and combining a corresponding image correction method, and magnifies the local image by adopting a pixel capturing and filling method to make the local image become an image acceptable to human eyes, and the image can be displayed on a spherical surface, a cylindrical curved surface and a plane and can display the locally magnified image.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
A method for panoramic shooting by using a fisheye lens comprises the following steps:
step 1: acquiring a panoramic image through a fisheye lens and an image sensor; wherein, the distortion parameter of the fisheye lens is linear distortion;
step 2: correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; wherein, the correction parameters comprise optical center, magnification factor and mapping model parameters appointed by a user;
and step 3: and (3) mapping the panoramic image corrected in the step (2) to a plane or a curved surface to obtain images under different application scenes.
Specifically, the correction method of the present invention specifically comprises: s31, determining the range of the image to be corrected according to the optical center and the preset magnification; the image obtained by the fisheye lens is displayed in the image sensor, at this time, a user can determine the optical center only by clicking with a mouse or determining the central point of an area needing to be magnified in other modes, the magnification factor is set according to the requirements of the customer, if the user needs to zoom in, the preset magnification factor is larger, and if the user does not need to pull too close, the preset magnification factor is not required to be too large;
s32: determining the magnification factor of each pixel area in the range of the image needing to be corrected according to the mapping model parameters; the mapping model can be various, such as a continuous strip-shaped curved surface, a hemispherical surface and the like, and according to the difference of the mapping model, on different mapping models, all pixel points of the image sensor are distributed on the mapping model according to a preset rule; for example, in a hemispherical mapping model, the magnification of the image area at the edge is greater than that of the central area; multiplying the preset magnification factor in the S31 by the magnification factor in the S32 to obtain the magnification factor of the pixel area;
s33: grabbing pixels in each pixel area and filling the pixels into the pixel area to obtain an enlarged local image;
the filling method in S33 specifically includes: and obtaining the pixel value of the local image needing to be amplified according to the multiplication result of the preset amplification factor in the S31 and the amplification factor in the S32, comparing the pixel value of the pixel area in the image sensor with the pixel value of the local image needing to be amplified, and determining the same number of pixels distributed around each pixel according to the ratio, thereby realizing the pixel filling in one pixel area. And the filling operation of a plurality of pixel areas is sufficient for obtaining the local image needing to be corrected by the above method.
S34: and repeating S32-S33 to obtain a plurality of local images, and combining the plurality of local images to obtain the corrected panoramic image.
The invention can display the panoramic image on different display interfaces by adopting a specific fisheye lens and combining a corresponding image correction method, and magnifies the local image by adopting a pixel capturing and filling method to make the local image become an image acceptable to human eyes, and the image can be displayed on a spherical surface, a cylindrical curved surface and a plane and can display the locally magnified image.
Example 2
The embodiment discloses a panoramic shooting system, wherein the method in the embodiment 1 is implemented by the panoramic shooting system, specifically, the implementation of the panoramic shooting system comprises a camera adopting a fisheye lens, a controller, a display, a communication unit and a mobile terminal, wherein the camera adopting the fisheye lens is connected with the controller;
the camera comprises a fisheye lens and an image sensor arranged in the fisheye lens, wherein
The distortion parameter of the fisheye lens is linear distortion;
the focal length of the lens is 0.82mm +/-3%; the angular range of the lens is 220 °
The controller includes:
a panoramic image acquisition unit: the system is used for controlling the fisheye lens and the image sensor to acquire a panoramic image;
panoramic image correction unit: the panoramic image correction device is used for correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; wherein, the correction parameters comprise optical center, magnification factor and mapping model parameters appointed by a user;
an image mapping unit: and the panoramic image correction unit is used for mapping the panoramic image corrected by the panoramic image correction unit to a plane or a curved surface to obtain images of different application scenes. The image mapping unit comprises the following subunits:
the range determining subunit is used for determining the range of the image to be corrected according to the optical center and the magnification;
magnification determination subunit: the magnification factor of each pixel area in the range of the image needing to be corrected is determined according to the mapping model parameters;
pixel fill sub-unit: and capturing the pixels in each pixel area and filling the pixels into the pixel area to obtain an enlarged partial image.
The display is used for displaying the images of different application scenes obtained by the processing of the image mapping unit;
the controller acquires the control signal sent by the mobile terminal through the communication unit and sends the images of different application scenes to the mobile terminal for display.
In practical application, an operator can implement the method through a display with a controller or a mobile terminal, when the operator implements the display with the controller, an image collected by a camera is transmitted to the controller, the operator selects a mapping type, an optical center and a magnification on the display, so that the panoramic image correction unit processes the panoramic image obtained by the panoramic image acquisition unit, the image is mapped onto a specific mapping model through the range determination subunit, the image mapping unit and the magnification determination subunit, and the image filling is performed through the pixel filling subunit, so that a corrected local or panoramic image on the specific mapping model is obtained.
In the actual operation process, an operator can also perform the operation through the mobile terminal, which is similar to the operation performed on the display with the controller, but the control instruction is transmitted to the controller through the communication unit, so that the image correction processing is realized, and the corrected image is transmitted to the mobile terminal for displaying.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims (4)

1. A method for panoramic shooting by using a fisheye lens is characterized by comprising the following steps: the method comprises the following steps:
step 1: acquiring a panoramic image through a fisheye lens and an image sensor; wherein, the distortion parameter of the fisheye lens is linear distortion;
step 2: correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; wherein, the correction parameters comprise optical center, magnification factor and mapping model parameters appointed by a user;
and step 3: mapping the panoramic image corrected in the step (2) onto a plane or a curved surface to obtain images under different application scenes;
the specific method for correcting the panoramic image through the correction parameters in the step 2 comprises the following steps:
s31: determining the range of the image needing to be corrected according to the optical center and the magnification;
s32: determining the magnification factor of each pixel area in the range of the image needing to be corrected according to the mapping model parameters;
s33: grabbing pixels in each pixel area and filling the pixels into the pixel area to obtain an enlarged local image;
the filling method in S33 specifically includes: obtaining the pixel value of the local image needing to be amplified according to the multiplication result of the preset amplification factor in the S31 and the amplification factor in the S32, comparing the pixel value of the pixel area in the image sensor with the pixel value of the local image needing to be amplified, and determining the same number of pixels distributed around each pixel according to the ratio, thereby realizing the pixel filling in one pixel area; filling operation of a plurality of pixel areas is repeated to obtain a local image needing to be corrected;
s34: and repeating S32-S33 to obtain a plurality of local images, and combining the plurality of local images to obtain the corrected panoramic image.
2. A panoramic photographing system for implementing the method of claim 1 for panoramic photographing using a fisheye lens, comprising a camera using a fisheye lens and a controller, wherein the camera using a fisheye lens is connected to the controller, and the controller comprises:
a panoramic image acquisition unit: the system is used for controlling the fisheye lens and the image sensor to acquire a panoramic image;
panoramic image correction unit: the panoramic image correction device is used for correcting the panoramic image on the image sensor according to the correction parameters to obtain a corrected panoramic image; wherein, the correction parameters comprise optical center, magnification factor and mapping model parameters appointed by a user;
an image mapping unit: the panoramic image correction unit is used for correcting the panoramic image and mapping the panoramic image to a plane or a curved surface to obtain images of different application scenes, and the image mapping unit comprises the following subunits: a range determination subunit: the device is used for determining the range of the image needing to be corrected according to the optical center and the magnification; magnification determination subunit: the magnification factor of each pixel area in the range of the image needing to be corrected is determined according to the mapping model parameters; pixel fill sub-unit: grabbing pixels in each pixel area and filling the pixels into the pixel area to obtain an enlarged local image;
the filling method of the pixel filling subunit comprises the following specific steps: obtaining the pixel value of the local image to be amplified according to the multiplication result of the preset amplification factor in the range determining subunit and the amplification factor in the amplification factor determining subunit, then comparing the pixel value in the pixel area in the image sensor with the pixel value of the local image to be amplified obtained through calculation, and determining the same number of pixels distributed around each pixel according to the ratio, thereby realizing the pixel filling in one pixel area; and the filling operation of a plurality of pixel areas repeats the method to obtain a local image needing to be corrected.
3. The panoramic camera system of claim 2, further comprising a display for displaying images of different application scenes processed by the image mapping unit.
4. The panoramic camera system of claim 3, further comprising a communication unit and a mobile terminal, wherein the controller obtains the control signal sent by the mobile terminal through the communication unit and sends the images of different application scenes to the mobile terminal for display.
CN201710730399.2A 2017-08-23 2017-08-23 Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system Active CN107454326B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710730399.2A CN107454326B (en) 2017-08-23 2017-08-23 Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710730399.2A CN107454326B (en) 2017-08-23 2017-08-23 Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system

Publications (2)

Publication Number Publication Date
CN107454326A CN107454326A (en) 2017-12-08
CN107454326B true CN107454326B (en) 2021-01-26

Family

ID=60493806

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710730399.2A Active CN107454326B (en) 2017-08-23 2017-08-23 Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system

Country Status (1)

Country Link
CN (1) CN107454326B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109246416A (en) * 2018-09-21 2019-01-18 福州大学 The panorama mosaic method of vehicle-mounted six road camera
CN109391802A (en) * 2018-12-20 2019-02-26 北京伊神华虹***工程技术有限公司 A kind of method and apparatus for realizing real-time VR function

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663734A (en) * 2012-03-15 2012-09-12 天津理工大学 Fish eye lens calibration and fish eye image distortion correction method
CN106780389A (en) * 2016-12-23 2017-05-31 浙江宇视科技有限公司 A kind of fisheye image correcting method and device based on Coordinate Conversion
CN106803874A (en) * 2017-03-13 2017-06-06 杭州昇星网络科技有限公司 A kind of full-view camera

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663734A (en) * 2012-03-15 2012-09-12 天津理工大学 Fish eye lens calibration and fish eye image distortion correction method
CN106780389A (en) * 2016-12-23 2017-05-31 浙江宇视科技有限公司 A kind of fisheye image correcting method and device based on Coordinate Conversion
CN106803874A (en) * 2017-03-13 2017-06-06 杭州昇星网络科技有限公司 A kind of full-view camera

Also Published As

Publication number Publication date
CN107454326A (en) 2017-12-08

Similar Documents

Publication Publication Date Title
US10306165B2 (en) Image generating method and dual-lens device
CN106934772B (en) Horizontal calibration method and system for panoramic image or video and portable terminal
US20230132407A1 (en) Method and device of video virtual background image processing and computer apparatus
CN101720027B (en) Method for cooperative acquisition of multi-target videos under different resolutions by variable-focus array camera
CN106780389B (en) Fisheye image correction method and device based on coordinate transformation
CN106651808B (en) Fisheye diagram conversion method and device
AU2017370476B2 (en) Virtual reality-based viewing method, device, and system
KR101521008B1 (en) Correction method of distortion image obtained by using fisheye lens and image display system implementing thereof
CN106357976A (en) Omni-directional panoramic image generating method and device
CN107113376A (en) A kind of image processing method, device and video camera
CN105741233B (en) Video image spherical surface splicing method and system
CN106550192A (en) A kind of virtual reality shooting and the method for showing, system
CN105120247A (en) White-balance adjusting method and electronic device
CN103247024A (en) 180-degree fisheye image spread method based on concentric algorithm and device
CN106454098A (en) Virtual reality shooting and displaying method and system
CN107454326B (en) Method for panoramic shooting by using fisheye lens, camera and panoramic shooting system
KR100948872B1 (en) Camera image correction method and apparatus
KR101233948B1 (en) Method and imaging system for obtaining digital pan·tilt images using rotationally symmetric wide-angle lens
CN105472263B (en) Image acquisition method and the image capture equipment for using the method
CN104539893B (en) Realize the method, apparatus and system of virtual head monitoring effect
CN103139457A (en) Method for obtaining and controlling images and electronic device
CN109218602A (en) Image capture unit, image treatment method and electronic device
KR101407119B1 (en) Camera system using super wide angle camera
WO2018157350A1 (en) Horizontal calibration method and system for panoramic image or video, and portable terminal
CN107222689B (en) Real scene switching method and device based on VR (virtual reality) lens

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 511430 1st and 2nd floors of Building 2, west side of Shanxi village, Dashi street, Panyu District, Guangzhou City, Guangdong Province

Applicant after: GUANGZHOU JUAN INTELLIGENT TECH JOINT STOCK Co.,Ltd.

Address before: 510000 1st floor, No.9, Third Road, HENGLING Industrial Zone, Tianhe District, Guangzhou City, Guangdong Province

Applicant before: GUANGZHOU JUAN OPTICAL & ELECTRONICAL TECH JOINT STOCK Co.,Ltd.

CB02 Change of applicant information
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20201230

Address after: 1 / F and 2 / F, building 1, No. 1, Shanxi Gangbian street, Dashi street, Panyu District, Guangzhou City, Guangdong Province

Applicant after: Guangzhou Jiuan Wulian Technology Co.,Ltd.

Address before: 511430 1st and 2nd floors of Building 2, west side of Shanxi village, Dashi street, Panyu District, Guangzhou City, Guangdong Province

Applicant before: GUANGZHOU JUAN INTELLIGENT TECH JOINT STOCK Co.,Ltd.

GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20220628

Address after: 510000 first and second floors of Building 2, west side of Shanxi village, Dashi street, Guangzhou, Guangdong Province (plant 2)

Patentee after: Guangzhou Jiuan Intelligent Technology Co.,Ltd.

Address before: 1 / F and 2 / F, building 1, No. 1, Shanxi Gangbian street, Dashi street, Panyu District, Guangzhou City, Guangdong Province

Patentee before: Guangzhou Jiuan Wulian Technology Co.,Ltd.

TR01 Transfer of patent right
CP03 Change of name, title or address

Address after: 510000 Floor 1 and 2, Building 2, West Side of Shanxi Village, Dashi Street, Panyu District, Guangzhou City, Guangdong Province

Patentee after: Guangdong Jiu'an Intelligent Technology Co.,Ltd.

Address before: 510000 first and second floors of Building 2, west side of Shanxi village, Dashi street, Guangzhou, Guangdong Province (plant 2)

Patentee before: Guangzhou Jiuan Intelligent Technology Co.,Ltd.

CP03 Change of name, title or address