CN109087251B - Vehicle-mounted panoramic image display method and system - Google Patents
Vehicle-mounted panoramic image display method and system Download PDFInfo
- Publication number
- CN109087251B CN109087251B CN201810999718.4A CN201810999718A CN109087251B CN 109087251 B CN109087251 B CN 109087251B CN 201810999718 A CN201810999718 A CN 201810999718A CN 109087251 B CN109087251 B CN 109087251B
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- driving mode
- mode
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000012937 correction Methods 0.000 claims abstract description 31
- 230000004927 fusion Effects 0.000 claims description 48
- 230000000007 visual effect Effects 0.000 claims description 15
- 230000009466 transformation Effects 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 230000000694 effects Effects 0.000 claims description 12
- 230000003287 optical effect Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 238000007500 overflow downdraw method Methods 0.000 claims description 4
- 230000007613 environmental effect Effects 0.000 claims description 3
- 238000003702 image correction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Closed-Circuit Television Systems (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a vehicle-mounted panoramic image display method and a vehicle-mounted panoramic image display system, wherein the display method comprises the following steps: acquiring a driving mode of an automobile, wherein the driving mode comprises a forward driving mode and a low-speed parking mode; adjusting the position of a camera according to the running mode; the number of the cameras is 4, and the cameras are respectively positioned in the front, the back, the left and the right directions of the automobile; acquiring images shot by the cameras; correcting each image by adopting a checkerboard correction method to obtain a corrected image; projecting each corrected image to the same plane according to the driving mode to obtain a projected image; fusing the projection images to obtain a panoramic image; and displaying the panoramic image. The system outputs the panoramic view image when the automobile is in a forward driving mode, and outputs the overlooking panoramic view image when the automobile is in a low-speed parking mode. The system has the advantages that the auxiliary driving function is more complete, the auxiliary driving mode can be switched, the system is suitable for complex roads, and the driving safety performance is higher.
Description
Technical Field
The invention relates to the field of image processing, in particular to a vehicle-mounted panoramic image display method and system.
Background
A panoramic display system based on a fisheye lens is an important aspect in modern auxiliary driving technology, environmental information in a large visual angle range is acquired through the fisheye lens, and an acquired image containing fisheye lens distortion is corrected and provided for a driver, so that the driver can acquire surrounding road conditions and vehicle conditions in real time during driving, and traffic accidents caused by visual field blind areas are avoided. The theoretical basis of the panoramic display driving assisting system is an image fusion and fish eye correction technology, namely an image fusion theory based on feature point matching and an image correction theory based on a carroll algorithm.
The concept of panoramic visualization in a driver-assist system was first proposed in 2006 by k.kato, m.suzuki, y.fujita, y.hirama et al. The panoramic display system is the latest content in the driving assistance system, is also the most practical part, has a solid and complete theoretical basis, and is in the continuous development stage at present. The panoramic all-round looking system is mainly used for an automobile parking auxiliary system. The cameras are arranged on the periphery of the car body, so that an all-around visual image surrounding the car body is provided, the information of driving visual dead angles and blind areas is reflected to a driver in real time, and the safety and efficiency of parking are greatly improved.
The auxiliary driving system based on the panoramic display function has wide application in practical engineering, such as the fields of driving anti-collision early warning, lane line detection, intelligent navigation and the like. However, the vehicle-mounted panoramic display system in the prior art has a single auxiliary mode, so that the vehicle-mounted panoramic display system cannot adapt to complex road conditions.
Disclosure of Invention
The invention aims to provide a vehicle-mounted panoramic image display method and a vehicle-mounted panoramic image display system, which can provide effective driving environment information for a driver under a complex road well, eliminate a visual field blind area and greatly improve the driving safety.
In order to achieve the purpose, the invention provides the following scheme:
a vehicle-mounted panoramic image display method comprises the following steps:
acquiring a driving mode of an automobile, wherein the driving mode comprises a forward driving mode and a low-speed parking mode;
adjusting the position of a camera according to the running mode; the number of the cameras is 4, and the cameras are respectively positioned in the front, the back, the left and the right directions of the automobile;
acquiring images shot by the cameras;
correcting each image by adopting a checkerboard correction method to obtain a corrected image;
projecting each corrected image to the same plane according to the driving mode to obtain a projected image;
fusing the projection images to obtain a panoramic image;
and displaying the panoramic image.
Optionally, adjusting the position of the camera according to the driving mode specifically includes:
when the driving mode is a forward driving mode, adjusting the optical axis of the camera to be parallel to the ground;
and when the running mode is a low-speed parking mode, adjusting the optical axis of the camera to be 45 degrees downward from the ground.
Optionally, projecting each of the corrected images onto the same plane according to the driving mode specifically includes:
when the driving mode is a low-speed parking mode, performing perspective transformation on each corrected image to switch the visual angle to a top view visual angle, and projecting the top view visual angle to a top view plane;
and when the running mode is a forward running mode, projecting each corrected image onto the same cylindrical surface.
Optionally, the camera is a fisheye camera.
Optionally, the display method further includes:
calibrating each fisheye camera by adopting a camera calibration principle to obtain internal and external parameters of the camera, and specifically comprises the following steps:
shooting a black and white checkerboard by a fisheye camera, and calculating the position of a corresponding angular point in an image coordinate system; establishing a model according to a transformation process of mapping the corner points from the world coordinate system to the corresponding positions of the image coordinate system; collecting chessboard angular point information and inputting the chessboard angular point information into the model to calculate model parameters to obtain a camera parameter matrix.
Optionally, fusing the projection images, specifically including:
calculating a boundary straight line of the overlapping area;
calculating the distance between the pixels in the image overlapping area and the two boundaries according to the boundary straight line to distribute the weight of the spliced image;
and performing image fusion by adopting a weighted average fusion method according to the weight.
An in-vehicle panoramic image display system, the display system comprising:
the image acquisition and correction module is used for acquiring images in real time and correcting the images to obtain four corrected images;
the mode conversion module is used for adjusting the position of the camera according to the driving mode, acquiring the environment images of the vehicle body in different directions and outputting the environment images to the image projection module;
the image projection module is used for projecting each corrected image to the same plane according to a driving mode to obtain a projected image;
the image fusion module is used for fusing the projection images to obtain a panoramic image;
and the display module is used for displaying the panoramic image.
Optionally, the image projection module includes:
the cylindrical surface projection unit is used for projecting each corrected image onto the same cylindrical surface when the running mode is a forward running mode;
and an overhead projection unit configured to perform perspective transformation on each of the corrected images to switch an angle of view to an overhead angle of view and project the converted images onto an overhead plane when the driving mode is a low-speed parking mode.
Optionally, the image fusion module:
the cylindrical image fusion unit is used for carrying out image fusion on the four-path cylindrical projection effect image when the automobile is in a forward driving mode;
and the overlook image fusion unit is used for carrying out image fusion on the four overlook projection effect graphs when the automobile is in the low-speed parking mode.
Optionally, the image obtaining and correcting module includes:
the camera calibration unit is used for calibrating the camera by utilizing a camera calibration principle to obtain internal and external parameters of the camera;
the image acquisition unit is used for acquiring images in real time;
and the distortion correction unit is used for correcting the acquired image by using the internal and external parameters of the camera by adopting a checkerboard correction method based on a carroll algorithm.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention provides a multi-mode vehicle-mounted panoramic image display method and a multi-mode vehicle-mounted panoramic image display system. The system has the advantages that the auxiliary driving function is more complete, the auxiliary driving mode can be switched, the system is suitable for complex roads, and the driving safety performance is higher.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flow chart of a vehicle panoramic image display method of the present invention;
fig. 2 is a structural connection diagram of the vehicle-mounted panoramic image display system of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a vehicle-mounted panoramic image display method and a vehicle-mounted panoramic image display system, which can be used for switching auxiliary driving modes by using a mode conversion module when different driving modes are adopted. The video is obtained by fish-eye cameras arranged in four directions of the front, the back, the left and the right of the vehicle body, and the obtained video is decomposed into video frames, namely the video frames are stored in an image form. And carrying out fisheye correction on the acquired images with barrel-shaped distortion, carrying out projection transformation on the corrected images according to a driving mode, and finally carrying out image fusion on the four projected images and outputting the images through a vehicle-mounted display.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a flowchart of a vehicle-mounted panoramic image display method according to the present invention, as shown in fig. 1, the display method includes:
step 11: the method comprises the steps of obtaining driving modes of an automobile, wherein the driving modes comprise a forward driving mode and a low-speed parking mode.
Step 12: adjusting the position of a camera according to the running mode; the number of the cameras is 4, and the cameras are respectively positioned in the front, the back, the left and the right directions of the automobile.
The position of the camera is different according to different driving modes: when the driving mode is a forward driving mode, adjusting the optical axis of the camera to be parallel to the ground; and when the running mode is a low-speed parking mode, adjusting the optical axis of the camera to be 45 degrees downward from the ground. The steering engine is used for adjusting the fisheye lens to obtain the vehicle body environment information at different distances, so that the steering engine can adapt to complex road conditions and has flexibility.
Wherein, the camera is the fisheye camera. Before the fisheye camera calibration method is used, a camera calibration principle is needed to calibrate each fisheye camera to obtain internal and external parameters of the camera, and the specific method comprises the following steps: shooting a black and white checkerboard by a fisheye camera, and calculating the position of a corresponding angular point in an image coordinate system; establishing a model according to a transformation process of mapping the corner points from the world coordinate system to the corresponding positions of the image coordinate system; collecting chessboard angular point information and inputting the chessboard angular point information into the model to calculate model parameters to obtain a camera parameter matrix.
Step 13: acquiring images shot by the cameras;
step 14: correcting each image by adopting a checkerboard correction method to obtain a corrected image; and carrying out image correction on the internal and external parameters of the camera obtained by calibration by using a checkerboard correction method based on a carroll algorithm, and finally obtaining four corrected images.
Step 15: and projecting each corrected image to the same plane according to the driving mode to obtain a projected image.
Because the angles of the cameras in different driving modes are different, the planes selected during projection are different: when the driving mode is a low-speed parking mode, performing perspective transformation on each corrected image to switch the visual angle to a top view visual angle, and projecting the top view visual angle to a top view plane; and when the running mode is a forward running mode, projecting each corrected image onto the same cylindrical surface. The invention carries out cylindrical projection and fusion display on the image, and meets the requirement of a driver on the remote vehicle body environment information in the vehicle body when the driver drives at a high speed.
Step 16: fusing the projection images to obtain a panoramic image, and specifically comprising the following steps:
step 161: calculating a boundary straight line of the overlapping area;
step 162: calculating the distance between the pixels in the image overlapping area and the two boundaries according to the boundary straight line to distribute the weight of the spliced image;
step 163: and performing image fusion by adopting a weighted average fusion method according to the weight.
The invention adopts the method of calculating the image fusion boundary and directly splicing, thereby saving the trouble of detecting and matching the characteristic points and improving the splicing speed.
And step 17: and displaying the panoramic image.
Fig. 2 is a structural connection diagram of a vehicle-mounted panoramic image display system of the present invention, as shown in fig. 2, the display system includes: an image acquisition and correction module 21, a mode conversion module 22, an image projection module 23, an image fusion module 24, and a display module 25.
And the image acquisition and correction module 21 is configured to acquire an image in real time, correct the image, and obtain four corrected images.
The image acquisition and correction module 21 includes:
the camera calibration unit 211 is configured to calibrate the camera according to a camera calibration principle to obtain internal and external parameters of the camera;
an image acquisition unit 212 for acquiring an image in real time;
and an aberration correcting unit 213, configured to perform image correction on the acquired image by using the internal and external parameters of the camera through a checkerboard correction method based on a carroll algorithm.
And the mode conversion module 22 is used for adjusting the position of the camera according to the driving mode, acquiring the environment images of the vehicle body in different directions and outputting the environment images to the image projection module.
And the image projection module 23 is configured to project each of the corrected images onto the same plane according to a driving mode to obtain a projected image.
The image projection module 23 includes:
a cylindrical surface projection unit 231 for projecting each of the corrected images onto the same cylindrical surface when the driving mode is a forward driving mode;
and an overhead projection unit 232 configured to, when the driving mode is the low-speed parking mode, perform perspective transformation on each of the corrected images to switch an angle of view to an overhead angle of view, and project the converted image onto an overhead plane.
And an image fusion module 24, configured to fuse the projection images to obtain a panoramic image.
The image fusion module 24 includes:
the cylindrical image fusion unit 241 is used for performing image fusion on the four-path cylindrical projection effect image when the automobile is in a forward driving mode;
and an overhead view image fusion unit 242 for performing image fusion on the four overhead view projection effect maps when the automobile is in the low-speed parking mode.
And a display module 25 for displaying the panoramic image.
The invention provides a vehicle-mounted panoramic image display system which mainly comprises an image acquisition and correction module, a mode conversion module, an image projection module, an image fusion module and a display module. The vehicle body environment image is obtained by fish-eye cameras arranged in four directions of the front, the back, the left and the right of the vehicle body, and the obtained video is decomposed into video frames, namely the video frames are stored in an image form. And carrying out fisheye correction on the acquired image with barrel-shaped distortion, taking the corrected image as the input of an image projection module, and carrying out projection transformation by a mode conversion module according to a driving mode. And finally, carrying out image fusion on the projected four images by using an image fusion module, and outputting the images through a vehicle-mounted display.
The image acquiring and correcting module 21 mainly comprises a camera calibration unit 211, an image acquiring unit 212 and a distortion correcting unit 213. And the calibrated fisheye cameras are arranged in four directions, namely front, back, left and right directions of the vehicle body, so that the real-time acquisition of images is realized. And then calibrating the camera by utilizing a camera calibration principle to obtain the internal and external parameters of the camera. And carrying out image correction on the acquired image and the calibrated internal and external parameters of the fisheye lens by using a checkerboard correction method based on a carroll algorithm, and finally obtaining four corrected images. The vehicle body environment image is acquired through four fisheye cameras arranged on the front, the back, the left and the right of a vehicle body, the cameras are connected with the vehicle body through steering engines, image information is transmitted to an image acquisition and correction module through a USB serial port, and the module receives the video information and then decomposes the video into frame images. Firstly, the fisheye camera is calibrated, and because the black and white checkerboard has the advantages that the angular points are easy to detect, and the template parameters are beneficial to calculation, the black and white checkerboard is utilized for calibration. The center of the chessboard grid template is assumed to be the origin of a world coordinate system, the right side of the origin is a positive X axis of the world coordinate system, a positive Y axis is a direction far away from the chessboard grid template, a positive Z axis direction is an upper direction, and the positive Z axis direction is superposed with an optical axis of a camera to establish the world coordinate system. And shooting the black and white checkerboard by the fisheye camera, and calculating the position of the corresponding corner point in the image coordinate system. And establishing a model by utilizing a transformation process of mapping the corner points from the world coordinate system to the corresponding positions of the image coordinate system. Collecting chessboard angular point information, storing the angular point information, inputting the angular point information into a model, and calculating model parameters to obtain a camera parameter matrix. Performing fish-eye correction on the original image by using the acquired camera parameter matrix pair as the input of the fish-eye correction function in the image acquisition and correction module
And a mode conversion module 22, which is located between the image acquisition and correction module 21 and the image projection module 23, acquires the body environment images in different directions by adjusting the position of the camera, and outputs the output images of the image acquisition and correction module to the corresponding units of the image projection module according to the different driving conditions of the automobile, thereby realizing the mode switching function.
And an image projection module 23 which acquires an image input through the image acquisition and correction module and is divided into a cylindrical projection unit and an overhead projection unit according to a driving mode. The cylindrical projection unit is suitable for a forward driving mode, projects the input four images to the same cylindrical surface, and finally outputs a cylindrical projection effect image. The overlooking projection unit is suitable for a low-speed parking mode, perspective transformation is carried out on the input four-way image to enable the view angle to be switched to an overlooking view angle, and finally an overlooking projection effect picture is output.
And an image fusion module 24, which takes the output of the image projection module as the image input and mainly consists of a cylindrical image fusion unit and a top-view image fusion unit. And when the automobile is in a forward driving mode, calling a cylindrical image fusion unit, carrying out image fusion on the four paths of cylindrical projection effect images, and finally outputting a cylindrical panoramic image. And when the automobile is in a low-speed parking mode, calling an overlook image fusion unit, carrying out image fusion on the four overlook projection effect images, and finally outputting an overlook panoramic image. The specific fusion process is as follows: storing angular point information of four images, calculating boundary straight slope in a splicing direction, comparing the boundary straight slopes of splicing parts of the two spliced images, calculating a splicing coincidence region by combining the angular point information, calculating horizontal distance and vertical distance from a pixel point to the two straight lines in the coincidence region, distributing pixel weight values of the two spliced images in the region, finally performing image fusion on the overlapping region by using a weighted average fusion algorithm, namely performing summation operation by taking the proportion of the vertical distance and the proportion of the horizontal distance accounting for the sum of the vertical distance and the horizontal distance as pixel value coefficients of the two spliced images, and taking the final result as the pixel value of the point.
The image acquisition and correction module 21, the image projection module 23 and the image fusion module 24 are structurally independent from each other and are selectively connected through the mode conversion module 22, the driving mode is selected through the mode conversion module after the images are acquired and corrected through the fisheye lens, the images are output to corresponding units of the image projection module and the image fusion module according to different driving modes, and finally the panoramic display images are output.
The selected driving mode is a forward driving mode, the fisheye camera is located at a default position at the moment, the optical axis of the fisheye camera is parallel to the ground, and the acquired video frame image is selectively input into the cylindrical surface projection unit in the projection module after passing through the mode conversion module. A cylindrical model with the radius of the focal length f is established in the unit, the original image is tangent to the cylindrical surface, and the Z-axis coordinates of all pixels on the image are-f. And projecting the image on a cylindrical surface to establish a cylindrical surface projection model, applying the cylindrical surface projection model to the input four paths of images, and obtaining the four paths of images after cylindrical surface projection. And then taking the image after cylindrical projection as the input of a cylindrical image fusion unit in the image fusion module, storing the angular point information of the four images in the unit, and calculating the slope of a boundary straight line in the splicing direction. Comparing the slope of the border straight line of the spliced part of the two spliced images, calculating a spliced overlapping area by combining the angular point information, calculating the horizontal distance and the vertical distance from a pixel point to the two straight lines in the overlapping area, distributing the pixel weight of the two spliced images in the area, finally performing image fusion on the overlapping area by using a weighted average fusion algorithm, namely performing summation operation by taking the proportion of the vertical distance and the horizontal distance which respectively account for the sum of the vertical distance and the horizontal distance as the pixel value coefficients of the two spliced images, taking the final result as the pixel value of the point, and finally outputting the cylindrical panoramic image.
The driving mode is selected to be a low-speed parking mode, a square wave signal is input to the steering engine, the steering engine rotates downwards by 45 degrees, and the optical axis of the fisheye camera forms an inclined downward 45-degree included angle with the ground to shoot the close-range environment information of the vehicle body. The acquired video image is output to an image after fisheye correction through the image acquisition and correction module, then an overlook projection unit input into the image projection module is selected through the mode conversion module, and an overlook projection transformation matrix is applied to the input image to acquire an image after overlook projection. Inputting the overlooked and projected image into an overlooked image fusion unit in an image fusion module, calculating and storing the boundary straight line of the overlapping area, calculating the weight of the distance distribution spliced image of the pixel and the two boundaries in the image overlapping area, performing image fusion by using a weighted average fusion method, and finally outputting the overlooked panoramic image.
The multi-mode vehicle-mounted panoramic image display system provided by the invention is suitable for scenes with complicated road conditions and variable driving conditions. The system utilizes the steering engine to adjust the angle of the fisheye camera, obtains the near, medium and long-distance environment information of the vehicle body, automatically outputs cylindrical panoramic images and overlooking panoramic images according to the driving mode conversion, has clear display effect, has no obvious splicing seams, effectively eliminates visual field blind areas, and meets the requirements of drivers on the environment information of the vehicle body under different driving conditions.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (5)
1. A vehicle-mounted panoramic image display method is characterized by comprising the following steps:
acquiring a driving mode of an automobile, wherein the driving mode comprises a forward driving mode and a low-speed parking mode;
adjusting the position of a camera according to the running mode; the number of the cameras is 4, and the cameras are respectively positioned in the front, the back, the left and the right directions of the automobile; the camera is connected with the vehicle body through a steering engine; adjusting the angle of a fisheye camera by using a steering engine to acquire the near, medium and long-distance environmental information of the vehicle body;
adjusting the position of the camera according to the driving mode specifically comprises:
when the driving mode is a forward driving mode, the fisheye camera is at a default position, and the optical axis of the camera is adjusted to be parallel to the ground;
when the driving mode is a low-speed parking mode, inputting a square wave signal to the steering engine to enable the steering engine to rotate downwards by 45 degrees, and adjusting the optical axis of the camera to be inclined downwards by 45 degrees with the ground;
acquiring images shot by the cameras;
correcting each image by adopting a checkerboard correction method to obtain a corrected image;
projecting each corrected image to the same plane according to the driving mode to obtain a projected image;
projecting each of the corrected images onto the same plane according to the driving mode, specifically comprising:
when the driving mode is a low-speed parking mode, performing perspective transformation on each corrected image to switch the visual angle to a top view visual angle, and projecting the top view visual angle to a top view plane;
when the running mode is a forward running mode, projecting each corrected image onto the same cylindrical surface;
fusing the projection images to obtain a panoramic image;
fusing each projection image, which specifically comprises the following steps:
calculating a boundary straight line of the overlapping area;
calculating the distance between the pixels in the image overlapping area and the two boundaries according to the boundary straight line to distribute the weight of the spliced image;
performing image fusion according to the weight by adopting a weighted average fusion method;
and displaying the panoramic image.
2. The display method according to claim 1, wherein the camera is a fisheye camera.
3. The display method according to claim 2, further comprising:
calibrating each fisheye camera by adopting a camera calibration principle to obtain internal and external parameters of the camera, and specifically comprises the following steps:
shooting a black and white checkerboard by a fisheye camera, and calculating the position of a corresponding angular point in an image coordinate system; establishing a model according to a transformation process of mapping the corner points from the world coordinate system to the corresponding positions of the image coordinate system; collecting chessboard angular point information and inputting the chessboard angular point information into the model to calculate model parameters to obtain a camera parameter matrix.
4. An in-vehicle panoramic image display system, characterized in that the display system comprises:
the image acquisition and correction module is used for acquiring images in real time and correcting the images to obtain four corrected images;
the mode conversion module is used for adjusting the position of the camera according to the driving mode, acquiring the environment images of the vehicle body in different directions and outputting the environment images to the image projection module; the camera is connected with the vehicle body through a steering engine; adjusting the angle of a fisheye camera by using a steering engine to acquire the near, medium and long-distance environmental information of the vehicle body;
the image projection module is used for projecting each corrected image to the same plane according to a driving mode to obtain a projected image;
the image projection module includes:
the cylindrical surface projection unit is used for projecting each corrected image onto the same cylindrical surface when the running mode is a forward running mode;
a downward projection unit, configured to perform perspective transformation on each corrected image to switch an angle of view to a downward angle of view and project the image onto a downward plane when the driving mode is a low-speed parking mode;
the image fusion module is used for fusing the projection images to obtain a panoramic image;
the image fusion module:
the cylindrical image fusion unit is used for carrying out image fusion on the four-path cylindrical projection effect image when the automobile is in a forward driving mode;
the overlook image fusion unit is used for carrying out image fusion on the four overlook projection effect graphs when the automobile is in a low-speed parking mode;
and the display module is used for displaying the panoramic image.
5. The display system of claim 4, wherein the image acquisition and correction module comprises:
the camera calibration unit is used for calibrating the camera by utilizing a camera calibration principle to obtain internal and external parameters of the camera;
the image acquisition unit is used for acquiring images in real time;
and the distortion correction unit is used for correcting the acquired image by using the internal and external parameters of the camera by adopting a checkerboard correction method based on a carroll algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810999718.4A CN109087251B (en) | 2018-08-30 | 2018-08-30 | Vehicle-mounted panoramic image display method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810999718.4A CN109087251B (en) | 2018-08-30 | 2018-08-30 | Vehicle-mounted panoramic image display method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109087251A CN109087251A (en) | 2018-12-25 |
| CN109087251B true CN109087251B (en) | 2020-10-30 |
Family
ID=64795218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810999718.4A Active CN109087251B (en) | 2018-08-30 | 2018-08-30 | Vehicle-mounted panoramic image display method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109087251B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110239439A (en) * | 2019-06-21 | 2019-09-17 | 上海大学 | Vehicle drive panoramic looking-around auxiliary system |
| CN110738599B (en) * | 2019-10-14 | 2023-04-25 | 北京百度网讯科技有限公司 | Image stitching method and device, electronic equipment and storage medium |
| CN110930457B (en) * | 2019-10-15 | 2023-02-28 | 中国第一汽车股份有限公司 | Camera calibration method, device, vehicle and storage medium |
| CN111243034A (en) * | 2020-01-17 | 2020-06-05 | 广州市晶华精密光学股份有限公司 | Panoramic auxiliary parking calibration method, device, equipment and storage medium |
| CN114972494A (en) * | 2021-02-26 | 2022-08-30 | 魔门塔(苏州)科技有限公司 | Map construction method and device for memorizing parking scene |
| CN113191974B (en) * | 2021-04-29 | 2023-02-03 | 青岛科技大学 | Method and system for obtaining ship panoramic image based on machine vision |
| CN113592949B (en) * | 2021-07-01 | 2024-03-29 | 广东工业大学 | Control system and method for wireless parking image of vehicle |
| CN113516722B (en) * | 2021-09-14 | 2021-12-07 | 巨视慧行(武汉)科技有限公司 | Vehicle camera calibration method and device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104851076B (en) * | 2015-05-27 | 2018-10-09 | 武汉理工大学 | Panoramic looking-around parking assisting system and camera installation method for commercial car |
-
2018
- 2018-08-30 CN CN201810999718.4A patent/CN109087251B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109087251A (en) | 2018-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109087251B (en) | Vehicle-mounted panoramic image display method and system | |
| CN108263283B (en) | Method for calibrating and splicing panoramic all-round looking system of multi-marshalling variable-angle vehicle | |
| CN109741455B (en) | Vehicle-mounted stereoscopic panoramic display method, computer readable storage medium and system | |
| US9858639B2 (en) | Imaging surface modeling for camera modeling and virtual view synthesis | |
| CN110381255B (en) | Vehicle-mounted video monitoring system and method applying 360-degree panoramic looking-around technology | |
| CN103177439B (en) | A kind of automatic calibration method based on black and white lattice corners Matching | |
| JP5455124B2 (en) | Camera posture parameter estimation device | |
| CN112965503A (en) | Multi-path camera fusion splicing method, device, equipment and storage medium | |
| CN112224132B (en) | Vehicle panoramic all-around obstacle early warning method | |
| CN102298771A (en) | Fish-eye image rapid correction method of panoramic parking auxiliary system | |
| CN104851076A (en) | Panoramic 360-degree-view parking auxiliary system for commercial vehicle and pick-up head installation method | |
| CN103617606A (en) | Vehicle multi-angle panorama generating method for aided driving | |
| CN106994936A (en) | A kind of 3D panoramic parking assist systems | |
| CN110363085B (en) | Method for realizing looking around of heavy articulated vehicle based on articulation angle compensation | |
| CN107284455B (en) | A kind of ADAS system based on image procossing | |
| CN112233188B (en) | Calibration method of data fusion system of laser radar and panoramic camera | |
| CN104506764A (en) | An automobile traveling recording system based on a spliced video image | |
| CN113362228A (en) | Method and system for splicing panoramic images based on improved distortion correction and mark splicing | |
| CN103204104B (en) | Monitored control system and method are driven in a kind of full visual angle of vehicle | |
| CN112348741A (en) | Panoramic image splicing method, panoramic image splicing equipment, storage medium, display method and display system | |
| WO2024017003A1 (en) | Vehicle merging detection method and apparatus based on combined algorithms, and device | |
| CN114640801A (en) | Vehicle-end panoramic view angle auxiliary driving system based on image fusion | |
| CN115239922A (en) | AR-HUD three-dimensional coordinate reconstruction method based on binocular camera | |
| CN112802109B (en) | Method for generating aerial view panorama of automobile | |
| CN105774657B (en) | Single-camera panoramic reverse image system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |