CN107403404A - The three-dimensional panorama system and method for vehicle - Google Patents
The three-dimensional panorama system and method for vehicle Download PDFInfo
- Publication number
- CN107403404A CN107403404A CN201610329549.4A CN201610329549A CN107403404A CN 107403404 A CN107403404 A CN 107403404A CN 201610329549 A CN201610329549 A CN 201610329549A CN 107403404 A CN107403404 A CN 107403404A
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- image data
- flake
- correcting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 17
- 230000005855 radiation Effects 0.000 claims abstract description 63
- 238000012937 correction Methods 0.000 claims abstract description 38
- 230000003068 static effect Effects 0.000 claims abstract description 38
- 238000013519 translation Methods 0.000 claims abstract description 34
- 241000251468 Actinopterygii Species 0.000 claims description 60
- 241001269238 Data Species 0.000 claims description 25
- 238000000605 extraction Methods 0.000 claims description 15
- 238000009434 installation Methods 0.000 claims description 10
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 5
- 239000011521 glass Substances 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 claims 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 49
- 238000006243 chemical reaction Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 241000406668 Loxodonta cyclotis Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of three-dimensional panorama system and method for vehicle, the system includes：Flake adjustment of image setting module, to receive flake image data caused by shooting correcting plate；The correction of flake image data is turned into correcting image data and produces flake correction look-up table；The rotation of radiation vertical reference line and translation of the correcting image data of neighbouring relations are produced into rotation translation look-up table as the overlapping of identical angle of radiation；And static three-dimensional splicing image generation, to produce splicing image look-up table according to rotation translation look-up table and flake correction look-up table, the pixel value of each pixel in the calculated for pixel values correcting image data of each pixel in flake image data, and adjacent correcting image data are subjected to image joint, to produce static splicing image data.
Description
Technical field
The invention belongs to vehicular field, especially with regard to a kind of three-dimensional panorama system and method for vehicle.
Background technology
Traditional " panorama system (Around View Monitoring (AVM) System) ", passes through the side of image joint
Formula, splicing result is presented into closely (3~5 meters) image of car front, rear, left and right four direction in a 2D way, assists to drive
The situation of car surrounding is known, to avoid injuring the object of car surrounding.Such a presentation mode (2D-AVM) causes panorama system
It is only limitted to use during the operation of vehicle low speed.
In order to make up this shortcoming, in recent years manufacturer towards three-dimensional panorama image (3D-AVM) System Development.But
Traditional 3D-AVM also has some shortcomings, such as：The dead angle of some splicings can be produced when splicing image, in other words, is actually gone out
The object of present vehicle's surroundings, can vanish from sight in the region of image joint, or, traditional 3D-AVM is when splicing image, meeting
The situation that generation body repeats, and can be more obvious apart from the situation that more remote place generation image repeats.
The content of the invention
In view of above mentioned problem, it is an object of the invention to provide a kind of three-dimensional panorama system and method for vehicle, its
The generation at splicing dead angle is reduced when splicing image, makes the object for actually appearing in vehicle's surroundings will not in the region of image joint
Vanish from sight, also will not generation body situation about repeating.
The first aspect of the present invention provides a kind of three-dimensional panorama method of vehicle, and it comprises the following steps：
Multiple flake image datas caused by one correcting plate of shooting are received, there are linearly multiple on the correcting plate
Horizontal reference line and multiple radiation vertical reference lines, the horizontal reference line and radiation vertical reference line are in the flake image number
It is in circular arc lines in；
By the multiple flake image data school with the horizontal reference line in circular arc lines and radiation vertical reference line
Just turn into linear horizontal reference line and radiate multiple correcting image data of vertical reference line and produce a flake school
Positive look-up table, each pixel obtained according to flake correction look-up table in the multiple correcting image data correspond to described more
The coordinate position of each pixel in individual flake image data；
The linear radiation vertical reference line rotation and translation of the correcting image data of neighbouring relations are turned into phase
With angle of radiation overlapping and produce a rotation translation look-up table, look-up table is translated according to the rotation and the flake corrects
Each pixel that look-up table is obtained in rotated and translation the correcting image data is corresponded in the flake image data
The coordinate position of each pixel；And
Look-up table is translated according to the rotation and flake correction look-up table produces a splicing image look-up table, according to institute
State each pixel for obtaining in the correcting image data of splicing image look-up table correspond to it is each in the flake image data
The coordinate position of pixel, in correcting image data described in the calculated for pixel values of each pixel in the flake image data
Each pixel pixel value, and adjacent correcting image data are subjected to image joint, to produce a static splicing shadow
As data.
The second aspect of the present invention provides a kind of three-dimensional panorama system of vehicle, and it includes：
One flake adjustment of image setting module, to receive multiple flake image datas caused by one correcting plate of shooting,
By the multiple flake image data school with multiple horizontal reference lines in circular arc lines and multiple radiation vertical reference lines
Just turn into linear horizontal reference line and radiate multiple correcting image data of vertical reference line and produce a flake school
Positive look-up table, the linear radiation vertical reference line for the correcting image data of neighbouring relations is rotated and translation turns into
Identical angle of radiation overlapping and produce a rotation translation look-up table, wherein, have on the correcting plate linear described
Multiple horizontal reference lines and the multiple radiation vertical reference line, the horizontal reference line and institute in the flake image data
It is in circular arc lines to state radiation vertical reference line, is obtained according to flake correction look-up table each in the correcting image data
Pixel corresponds to the coordinate position of each pixel in the flake image data, and look-up table and the fish are translated according to the rotation
Each pixel that eye correction look-up table is obtained in rotated and translation the correcting image data corresponds to the flake image number
The coordinate position of each pixel in；And
One static three-dimensional splices image generation, to be looked into according to the rotation translation look-up table and flake correction
Look for table to produce a splicing image look-up table, corrected described in the calculated for pixel values of each pixel in the flake image data
The pixel value of each pixel in image data, and adjacent correcting image data are subjected to image joint, it is static to produce one
Splice image data, wherein, it is corresponding according to each pixel that the splicing image look-up table is obtained in the correcting image data
The coordinate position of each pixel in the flake image data.
Brief description of the drawings
Fig. 1 is the block diagram of the three-dimensional panorama system of vehicle of the present invention；
Fig. 2A is the schematic diagram of fish eye lens shooting angle of the present invention；
Fig. 2 B are the icon of fish eye lens filmed image of the present invention；
Fig. 3 A and Fig. 3 B are the schematic diagram of correcting plate of the present invention；
Fig. 4 A are the schematic diagram of the flake image of horizontal reference line of the present invention；
Fig. 4 B are the schematic diagram of a horizontal reference line in a plurality of horizontal reference lines of Fig. 4 A；
Fig. 5 is the schematic diagram of the flake image of present invention radiation vertical reference line；
Fig. 6 A be the present invention shooting correcting plate obtained by flake image schematic diagram；
Fig. 6 B are the schematic diagram that Fig. 6 A flake image is corrected to rear resulting correcting image；
Fig. 6 C are the schematic diagram of the pixel coordinate position in Fig. 6 A flake image；
Fig. 6 D are the schematic diagram of the pixel coordinate position in the pixel correction image in corresponding diagram 6C flake image；
The flake of establishing that Fig. 7 A and Fig. 7 B are the present invention corrects look-up table schematic diagram；
Fig. 8 A and Fig. 8 B are the foundation rotation translation look-up table schematic diagram of the present invention；
Fig. 9 is for the present invention according to radiation vertical reference line come the schematic diagram of overlapping adjacent correction image；
Figure 10 A are that the schematic diagram of the pixel value in the non-overlapping region of image in image is spliced in the calculating of the present invention；
Figure 10 B are that the schematic diagram of the pixel value in the non-overlapping region of image in image is not spliced in the calculating of the present invention；
Figure 11 A are that the schematic diagram of the pixel value of image overlap area in image is spliced in the calculating of the present invention；
Figure 11 B are that the schematic diagram of the pixel value of image overlap area in image is not spliced in the calculating of the present invention；
Figure 12 is the schematic diagram of the width of the image overlap area of fixing really of the invention；
The image that Figure 13 is the present invention is not compressed and mobile schema；
The image that Figure 14 is the present invention is compressed and mobile schema；
Figure 15 A are the schematic diagram of the pixel value of the calculating splicing image of the present invention；
Figure 15 B are the schematic diagram that the pixel value of image is not spliced in the calculating of the present invention；
Figure 16 is the icon for the 3-dimensional image that splicing multiple images of the present invention form；And
Figure 17 is the flow chart of the three-dimensional panorama method of the vehicle of the present invention.
Reference numerals list：
100 three-dimensional panorama systems
102 fish eye lenses
104 buffer storage
106 data presentation devices
108 data processing equipments
110 images input and output module
112 video cameras install correction module
114 flake adjustment of image setting modules
116 static three-dimensionals splice image generation
118 feature extraction modules
120 Dynamic parameter adjustment modules
122 dynamic 3 Ds splice image generation
130 vehicles
132 correcting plates
134 correcting plates
136 horizontal reference lines
138 radiation vertical reference lines
140 horizontal reference lines
142 radiation vertical reference lines
144 horizontal reference lines
146 radiation vertical reference lines
148 horizontal reference lines
150 radiation vertical reference lines
152 correcting images
The non-overlapping region of 154 images
156 image overlap areas
158 splicing images
160 dynamic splicing images
The non-overlapping region of 162 images
164 image overlap areas
Embodiment
To enable those skilled in the art to be further understood that the present invention, hereafter spy enumerates the present invention
Preferred embodiment, and coordinate institute's accompanying drawings, describe in detail the present invention constitution content and it is to be reached the effect of.
Fig. 1 is the block diagram of the three-dimensional panorama system of vehicle of the present invention.In Fig. 1, three-dimensional panorama system 100 includes multiple
Fish eye lens 102, a buffer storage 104, a data presentation device 106 and a data processing equipment 108.Data processing equipment
108 include image input and output module 110, video camera installation correction module 112, a flake adjustment of image setting module
114th, static three-dimensional splicing image generation 116, a feature extraction module 118, a Dynamic parameter adjustment module 120 and one
Dynamic 3 D splices image generation 122.
In the present embodiment, fish eye lens 102 is all around separately installed with to shoot the shadow of vehicle's surroundings in vehicle
Picture.Captured image is respectively converted into flake image data by multiple fish eye lenses 102, and is stored in buffer storage 104
In.Image data is shown as image, such as the flake image data that fish eye lens 102 is changed by data presentation device 106
It is shown as the image captured by fish eye lens 102.
Correction is positioned at the installation site of each fish eye lens 102 in four orientation of the front, rear, left and right of vehicle, such as Fig. 2A
For the present invention fish eye lens shooting angle schematic diagram and Fig. 2 B be the present invention fish eye lens filmed image icon shown in,
The image for showing Fig. 2 B by data presentation device 106 makes each flake mirror to correct the installation site of each fish eye lens 102
Image captured by first 102 is the image of underbody part at 1/8th of the bottom of whole image.
When carrying out static three-dimensional image joint program, as shown in the schematic diagram that Fig. 3 A and Fig. 3 B are correcting plate of the present invention,
Each fish eye lens 102 shoots the correcting plate 132,134 around vehicle 130, by taking Fig. 3 A correcting plate 132 as an example, each flake
Camera lens 102 photographs the scenery of a linear horizontal reference line 136 and a linear radiation vertical reference line 138,
By taking Fig. 3 B correcting plate 134 as an example, each fish eye lens 102 photographs a plurality of linear horizontal reference line 140 and a plurality of is in
The scenery of the radiation vertical reference line 142 of straight line.The explanation of following examples of the present invention is by taking Fig. 3 B correcting plate 134 as an example.
The flake image that each shooting figure 3B of fish eye lens 102 correcting plate 134 is obtained is a plurality of in circular arc lines
Horizontal reference line 144 (as Fig. 4 A be the present invention horizontal reference line flake image schematic diagram shown in) and it is a plurality of be in circular arc line
The radiation vertical reference line 146 (as shown in the schematic diagram for the flake image for radiating vertical reference line that Fig. 5 is the present invention) of bar
Image, and by a plurality of horizontal reference line 144 in circular arc lines and radiate vertical reference line 146 video conversion adult fish eye shadow picture
Data.Multiple flake image datas are stored in buffer storage 104 by each fish eye lens 102.
Video camera installs correction module 112 and receives multiple 102 turns of fish eye lenses with output module 110 via image input
Multiple flake image datas (a plurality of horizontal reference line 144 in circular arc lines and the flake shadow for radiating vertical reference line 146 changed
As data), or multiple flake image datas are read in buffer storage 104.
Fig. 4 B are the schematic diagram of a horizontal reference line in a plurality of horizontal reference lines of Fig. 4 A.Video camera installs correction module
112, according to the height h1 and h2 of two end points of Fig. 4 B horizontal reference line 144 in circular arc lines, make horizontal reference line
It is horizontal during the height h1 and h2 identical of 144 two end points with adjusting the horizontal level of each fish eye lens 102.
Video camera installs correction module 112 according to the radiation vertical reference line 146 in circular arc lines, to adjust each flake
The left and right angle position of camera lens 102, enable each fish eye lens 102 left and right ends photograph correcting plate 134 radiation it is vertical
Reference line 142.
Fig. 6 A are the schematic diagram of the flake image obtained by the shooting correcting plate of the present invention, and Fig. 6 B are by Fig. 6 A flake shadow
Schematic diagram as being corrected rear resulting correcting image, Fig. 6 C are the coordinate position of the pixel in Fig. 6 A flake image
Schematic diagram, Fig. 6 D are the coordinate position signal of the pixel in the correcting image of the pixel in corresponding diagram 6C flake image
Figure.
Flake image obtained from each shooting figure 3B of fish eye lens 102 correcting plate 134 as shown in Figure 6A, and flake shadow
Horizontal reference line 144 and radiation vertical reference line 146 as in is because of the relation of fish eye lens 102 and in a manner of circular arc lines
Present.Fish eye lens 102 stores flake image data flake video conversion adult fish eye image data as shown in Figure 6A
In buffer storage 104.
Flake adjustment of image setting module 114 receives each institute of fish eye lens 102 via image input with output module 110
The flake image data of conversion or the flake image data of read buffer memory 104.The profit of flake adjustment of image setting module 114
With known adjustment of image technology by Fig. 6 A in the horizontal reference line 144 of circular arc lines and radiation vertical reference line 146 school
Just into Fig. 6 B linear horizontal reference line 148 and radiation vertical reference line 150.
The horizontal reference line 140 and radiation vertical reference line 142 of correcting plate 134 are reduced into the presentation of straight line from circular arc lines
Mode, then represent the correction program for completing flake image.In brief, the program of flake adjustment of image is mainly in flake image
X and Y-axis each determine one group of optimal elliptic arc length parameter, then according to the parameter will be in flake image data it is all
Pixel is compressed toward image center point, as shown in Fig. 6 C and Fig. 6 D.Assuming that Fig. 6 C are flake image, by flake adjustment of image program
Fig. 6 D correcting image is obtained afterwards, it is therefore, right in a pixel coordinate (x1, y1) for the correcting image data of Fig. 6 D correcting image
Answer the coordinate (x2, y2) of the flake image data of Fig. 6 C flake image.
Flake adjustment of image setting module 114 can produce flake correction look-up table when carrying out flake adjustment of image
(Lookup Table), flake correction look-up table are mainly in the correcting image data after flake adjustment of image is found out in help
The coordinate (x2, y2) of each pixel in the corresponding flake image data of coordinate (x1, y1) of each pixel.Therefore, substantially up to
" X-coordinate " and " Y-coordinate " two flake correction look-up tables are had less.
Fig. 7 A and Fig. 7 B are the schematic diagram established flake and correct look-up table of the present invention.In Fig. 7 A and Fig. 7 B in heavy black line
The coordinate position of pixel that is expressed as in correcting image data of coordinate (x1, y1) and coordinate (x2, y2) be expressed as flake image
The coordinate position of pixel in data, wherein, x2 and y2 is unknown-value.
H1 and h2 in Fig. 7 A represents Y-coordinate point y1 and y2 to the height value of image center line respectively.According to etc. than principle,
Formula (2) can be obtained by derivation in the hope of formula (1), in other words, y2 is right in " Y-coordinate flake corrects look-up table "
Answer coordinate (x1, y1) value.
Using same method (such as Fig. 7 B), according to formula (3) x2 can be derived in " X-coordinate flake corrects look-up table "
The value of respective coordinates (x1, y1), wherein w1 and w2 represent X-coordinate point x1 and x2 to the width value of image center line respectively.
Flake adjustment of image is being flake adjustment of image setting after correcting image by flake adjustment of image setting module 114
The angle of radiation of radiation vertical reference line 150 in adjacent correcting image is adjusted to equal angular and can weighed by module 114
It is folded, rotation translation look-up table can be produced during the rotation and translation of image is corrected.
Fig. 8 A and Fig. 8 B are the schematic diagram of the foundation rotation translation look-up table of the present invention.The origin of correcting image in Fig. 8 A
(0,0) moves to correcting image new coordinate (xs, ys) position, and done for origin with (xs, ys) in the lower right-hand corner of image
Rotate counterclockwise θ angles, as shown in Figure 8 B.It is intended to the coordinate (x2, y2) for trying to achieve rotation and the pixel in translation post-equalization image data
With the relation of the coordinate (x1, y1) of the pixel in former correcting image data, can be tried to achieve by formula (4), (5), meanwhile, also have " X
Coordinate " and " Y-coordinate " two different rotation translation look-up tables.In other words, according to " X-coordinate " and " Y-coordinate " two
Rotation translation look-up table can learn it is rotated and translation post-equalization image data in each pixel coordinate position with it is original
The position relationship of the coordinate position of each pixel in correcting image data.
X2=(x1-xs) cos θ+(y1-ys) sin θ (4)
Y2=- (x1-xs) sin θ+(y1-ys) cos θ (5)
Flake adjustment of image setting module 114 according to adjacent two-by-two correcting image (such as：Left side correcting image with below
Correcting image and right side correcting image and correcting image below) in " radiation vertical reference line ", based on rotating as above and put down
Move the left and right sides correcting image correcting image data in each pixel coordinate position, make it is overlapping after school adjacent two-by-two
" radiation vertical reference line " in positive image can overlap as far as possible, and (such as Fig. 9 is the present invention according to radiation vertical reference line
Shown in the schematic diagram for carrying out overlapping adjacent correction image).In the present embodiment, the anglec of rotation θ of the correcting image of the left and right sides compared with
Good is this anglec of rotation and the known significantly anglec of rotation described in the present embodiment within the scope of 5 °≤θ≤45 °
(being more than 65 degree) has obvious difference.
It is above-mentioned adjacent correcting image two-by-two is subjected to image overlap when, static three-dimensional splicing image generation 116
Look-up table and rotation translation look-up table are corrected according to flake to produce a splicing image look-up table.When static three-dimensional splicing image production
When correcting image shown in Fig. 9 is carried out image joint by raw module 116, static three-dimensional splices image generation 116 according to this
Splicing image look-up table learns that each pixel in each correcting image data of image joint to be carried out corresponds to each flake shadow
As the coordinate position of each pixel in data.
Figure 10 A are that the schematic diagram of the pixel value in the non-overlapping region of image in image is spliced in the calculating of the present invention, and Figure 10 B are this
The schematic diagram of the pixel value in the non-overlapping region of image in image is not spliced in the calculating of invention, and Figure 11 A are the calculating splicing of the present invention
Image overlap area in image is not spliced in the schematic diagram of the pixel value of image overlap area in image, the calculating that Figure 11 B are the present invention
The schematic diagram of the pixel value in domain.Each acquiescence of correcting image 152 has the non-overlapping image overlap area of region 154 and one of an image
156, as shown in Figure 10 B and Figure 11 B.
The mode that static three-dimensional splicing image generation 116 produces static three-dimensional splicing image substantially will be according to figure
Pixel position in 10A and Figure 11 A splicing image 158, the image of each correcting image data is calculated respectively without weight
Folded region 154 and pixel value (i.e. r (red), g (green), b (blueness) picture of each pixel in image overlap area 156
Element value).
Static three-dimensional splicing image generation 116 calculates correction of the splicing image 158 in the non-overlapping region 154 of image
The pixel value of the pixel of image data, the calculating only needs to refer to a correcting image, therefore is looked for according to splicing image look-up table
The coordinate position corresponding with the pixel in flake image to pixel of the correcting image 152 in the non-overlapping region 154 of image, then
The pixel value of the coordinate position corresponding according to this can try to achieve splicing shadow of the splicing image 158 in the non-overlapping region 154 of image
As the pixel value (as shown in Figure 10 A) of the pixel of data.
Assuming that to try to achieve Figure 10 A coordinate (x, y) for splicing the pixel in image 158 in the non-overlapping region 154 of image
Pixel value (r, g, b), the flake of the coordinate (x, y) of the pixel in corresponding splicing image 158 is found according to splicing image look-up table
The coordinate position of pixel in image is (x1+ α 1, y1+ β 1), and as shown in Figure 10 B, wherein weighted value α 1, β 1 numerical value are 0≤α
The 1 ＜ 1 and ＜ 1 of 0≤β 1.Static three-dimensional splicing image generation 116 calculates the picture of the coordinate (x, y) of the pixel of splicing image
Plain value will use coordinate (x1, y1), (x1+1, y1), (x1, y1+1) and (x1+ of four pixels in flake image
1, y1+1) pixel value, and can be tried to achieve by formula (6), (7), (8).
The β 1 (6) of r=r1 (1- α 1) (1- β 1)+r2 α 1 (1- β 1)+r3 (1- α 1) β 1+r4 α 1
The β 1 (7) of g=g1 (1- α 1) (1- β 1)+g2 α 1 (1- β 1)+g3 (1- α 1) β 1+g4 α 1
The β 1 (8) of b=b1 (1- α 1) (1- β 1)+b2 α 1 (1- β 1)+b3 (1- α 1) β 1+b4 α 1
Static three-dimensional splicing image generation 116 calculates pixel of the splicing image 158 in image overlap area 156
Pixel value, the calculating needs to refer to two correcting images (as shown in Figure 11 B), therefore is found according to splicing image look-up table and be intended to spell
Pixel of two correcting images 152 connect in image overlap area 156 coordinate bit corresponding with the pixel in flake image
Put, then the pixel value of the coordinate position corresponding according to this can try to achieve splicing of the splicing image 158 in image overlap area 156
The pixel value (as shown in Figure 11 A) of the pixel of image data.
Assuming that to try to achieve the coordinate (x, y) of the pixel in Figure 11 A image overlap area 156 spliced in image 158
Pixel value (r, g, b), the splicing image data with Figure 11 A is found in image overlap area 156 according to splicing image look-up table
Pixel corresponding to flake image data in pixel coordinate position, find the coordinate bit of the pixel in flake image data
Put and can learn to should coordinate position pixel pixel value.Need to refer to the pixel value of the pixel of two flake image datas
(as shown in Figure 11 B) it is, and corresponding with pixel to be respectively obtained in two flake image datas using formula (6)~(8)
Pixel value (ra, ga, ba) and (rb, gb, bb).Image is carried out in pixel of the splicing image 158 in image overlap area 156
The pixel value of pixel during splicing can be tried to achieve by formula (9)~(11).
R=0.5*ra+0.5*rb (9)
G=0.5*ga+0.5*gb (10)
B=0.5*ba+0.5*bb (11)
Static three-dimensional splice image generation 116 calculate obtain splicing image 158 in the non-overlapping region 154 of image and
The pixel value of pixel in image overlap area 156, just complete by multiple flake images it is calibrated, rotation and translation form it is quiet
State three-dimensional splicing image, and use of the splicing image look-up table for following progress dynamic 3 D image joints can be produced.
When carrying out dynamic 3 D image joint program, each fish eye lens 102 shoots the shadow thing in vehicle traveling
And flake image is obtained, fish eye lens 102 is by captured flake video conversion adult fish eye image data, and by flake image number
According to being stored in buffer storage 104.
Feature extraction module 118 inputs the fish for receiving each fish eye lens 102 with output module 110 and being changed via image
Flake image data in eye image data or read buffer memory 104.Feature extraction module 118 refers to multiple flake images
To carry out dynamic 3 D image joint, and learn that a dynamic splices a dynamic splicing of image according to above-mentioned splicing image look-up table
Each pixel in image data corresponds to the coordinate position of each pixel in multiple flake image datas.
For the image overlap area of flake image adjacent two-by-two, (such as Figure 12 is the image weight of present invention fixing really
The region of L and R in the schematic diagram of the width in folded region) by analysis " edge direction (Edge Direction) " and " ladder
Degree intensity (Gradient Intensity) " is to obtain " edge type category map (Edge Style Classification
Map,ESCM)”.In the present embodiment, if the width Wd2 and flake of image overlap area (region that L and R are denoted as in Figure 12)
During the relation of the width Wd1 coincidence formulas 12 of image, it can reach using minimum computing and try to achieve optimal result.
The horizontal gradient that feature extraction module 118 calculates each pixel of the image overlap area of flake image data is exhausted
To value, a vertical gradient absolute value and an edge direction values, and by each pixel of the image overlap area of flake image data
Horizontal gradient absolute value, vertical gradient absolute value and edge direction values compared with multiple threshold values, and by flake image
Each pixel classifications of the image overlap area of data are one kind in multiple edge types.
That is, feature extraction module 118 carries out feature extraction to flake image data and mainly captures flake image data
Image overlap area each pixel gradient (Gradient) feature.In more complicated flake image (such as：Contain car
, shade, artificiality ... etc.) excessive edge pixel (Edge Pixel) can be detected, and increase the tired of subsequent characteristics comparison
Difficulty.In view of this, the present embodiment utilizes " edge type category map " can be efficiently by the image of flake image data
Its characteristic of the pixel basis of overlapping region is divided into " edge type (the Edge Style) " of at least four classes.It is ESCM classification below
Formula is as follows：
Wherein, ES (x, y) is " edge of pixel coordinate position (x, the y) point of the image overlap area of flake image data
Type " classification results；Δ AHG (x, y) and Δ AVG (x, y) is respectively " the horizontal gradient absolute value of pixel (x, y)
(Horizontal Gradient Absolute Value) " and " vertical gradient absolute value (Vertical Gradient
Absolute Value) ", its value scope is 0≤Δ AHG (x, y)≤255 and 0≤Δ AVG (x, y)≤255；HdV(x,y)
ForValue, in the range of 0≤HdV (x, y)≤∞；ThE1、ThE2、ThE3Respectively three kinds of threshold values
(threshold).Found through many experiments, following threshold settings can provide optimal result in 3D-AVM applications.
5≤ThE1≤10 (14)
0.3≤ThE2≤0.7 (15)
1.5≤ThE3≤2.5 (16)
The image that Figure 13 is the present invention is not compressed and mobile schema.Tried to achieve above-mentioned by feature extraction module 118
Two-by-two adjacent image (such as：" left side image with below image ", " right side image and image below ") image overlap area
In Gradient Features after, the object (such as lane line or car) in image overlap area is first found out, further according to object at two
Size in adjacent image, determine two important parameters of dynamic adjustment：It is " ratio of compressing image size " and " mobile
The degree of image ".
When static image splices, it is found that inconsistent phenomenon is had at image joint, such as in Figure 13 (a), it is " left
Lane line on the stitching portion road of side image and image below " and " right side image and image below " has apparent differ
Phenomenon is caused, in addition to lane line does not link, lane line also has thickness significantly different.
The adjustment of dynamic image is carried out in the present embodiment, to solve the above problems.First, Dynamic parameter adjustment module
120 will find out corresponding object in the image overlap area of adjacent image, judge that the image in adjacent image is pressed
The ratio of contracting.Such as：Figure 13 (b) and left side image and image below that (c) is respectively Figure 13 (a), and " image is shallow white " part
For " image overlap area ".By the classification of above-mentioned Gradient Features, it can find out in the image overlap area of adjacent image and compare
Representational object (straight-line segment of such as lane line), by analyzing the mean breadth (fineness) of straight line, to determine shadow
Parameter as needing implementation size compression.Such as：Figure 13 (d) and the figure that (e) is respectively Figure 13 (b) and the track line width of (c)
Show.
Assuming that Figure 13 (d) and (e) line segment mean breadth (thickness) are respectively 5 and 3 pixel wides.Dynamic parameter adjustment
Module 120 may decide that the parameter of compressing image would is that 3/5=0.6, in other words, it is necessary to image size Figure 13 (b)
0.6 times of size of script is compressed to, such image joint result just can be more perfect.
The image that Figure 14 is the present invention is compressed and mobile schema.Adjusted by the dynamic of " compressing image size "
After whole, and the schematic diagram of movable image generation splicing image is as shown in figure 14.Assuming that the length of original left side image is 500 pictures
Primitive unit cell (shown in such as Figure 14 (a)), compression factor is set to 0.6, and the image length after compression is changed into 300 pixel unit (such as Figure 14
(b) shown in).
Figure 14 (c) is the schematic diagram of " left side image " and " rear image " before image movement, after movable image
As a result as shown in Figure 14 (d).In the present embodiment, Dynamic parameter adjustment module 120 determines believable in image overlap area
Object when, if finding more than one group of object simultaneously, Dynamic parameter adjustment module 120 will according to these objects institute it is in place
Put, and determine its importance, substantially, can be than pair of up (Up) position in the feature of the object of lower section (Bottom) position
The feature of elephant is important.
Each pixel in multiple flake image datas is carried out rim detection and obtains one by Dynamic parameter adjustment module 120
Average gradient absolute value intensity.For example, table 1 (a), (b) are the partial image (7 × 7 size) in two flake images
The Gradient Features of the pixel of image data, and table 1 (a), the middle runic of (b) draw the pixel of bottom line then to want computing at present
The position of pixel.Table 1 (c), (d) are then the pixels of two image datas by horizontal Sobel edge detection (Sobel Edge
Detection the result after) detecting.Assuming that the default zone of the reference of the pixel of wanted computing is 5 × 5 sizes, dynamic is joined
Several adjusting modules 120 try to achieve " the average gradient absolute value intensity " of the pixel that computing is correspondingly wanted in two flake images respectively
(Ma=3600/25=144) and (Mb=360/25=14.4), as shown in table 1 (c), (d).Wherein, table 1 (e) is Sobel
3 × 3 templates (Mark) numerical value of rim detection.
Table 1
When carrying out dynamic 3 D splicing image, dynamic 3 D splices image generation 122 to be searched according to splicing image
Table learns that each pixel in dynamic splicing image data corresponds to the coordinate position of each pixel in each flake image data.
Figure 15 A are the schematic diagram of the pixel value of the calculating splicing image of the present invention, and the calculating that Figure 15 B are the present invention is not spliced
The schematic diagram of the pixel value of image.In Figure 15 A, dynamic 3 D splicing image generation 122 calculates dynamic splicing image 160
The mode of the pixel value of the pixel of image data in the non-overlapping region 162 of image is such as the splicing image production of above-mentioned static three-dimensional
Raw module 116 calculates the mode of the pixel value of the pixel of image data of the splicing image 158 in the non-overlapping region 162 of image,
The calculating only needs to refer to a flake image, therefore finds dynamic splicing image 160 in image according to splicing image look-up table
Pixel in non-overlapping region 162 coordinate position corresponding with the pixel in flake image, then the coordinate corresponding according to this
The pixel value of position can try to achieve the picture of dynamic splicing image data of the dynamic splicing image 160 in the non-overlapping region 162 of image
The pixel value of element.
Assuming that to try to achieve Figure 15 A dynamic splicing image 160 in image overlap area 164 in pixel coordinate (x,
Y) pixel value (r, g, b), found according to splicing image look-up table and splice image data in image overlap area with Figure 15 A dynamic
The coordinate position of the pixel in flake image data corresponding to pixel in domain 164, finds the pixel in flake image data
Coordinate position and can learn to should coordinate position pixel pixel value.Need to refer to the pixel of two flake image datas
Pixel value, and using formula (6)~(8) with respectively obtained in two flake image datas with pixel (xa, ya) and (xb,
Yb) corresponding pixel value (ra, ga, ba) and (rb, gb, bb) (as shown in fig. 15b).
Dynamic 3 D splices image generation 122 to be distinguished according to splicing image look-up table in two flake image datas
Obtain with pixel (xa, ya) and (xb, yb) (as shown in fig. 15b) corresponding pixel value (ra, ga, ba) and (rb, gb,
Bb), the pixel value of the pixel of image data of the dynamic splicing image 160 in image overlap area 164 and is thus calculated, it can
Tried to achieve by formula (17)~(19).Wherein：
MaWith MbFor " the average ladder in flake image data around the corresponding pixel to be calculated (xa, ya) and (xb, yb)
The numerical value of degree absolute value intensity ".
R=ra*RatioMa+rb*RatioMb (17)
G=ga*RatioMa+gb*RatioMb (18)
B=ba*RatioMa+bb*RatioMb (19)
Figure 16 is the icon for the 3-dimensional image that splicing multiple images of the present invention form.Dynamic 3 D splicing image produces mould
Block 122 will calculate the dynamic splicing image data obtained (comprising the non-overlapping region of image and image overlap area for dynamically splicing shadow
The image data in domain) via image input data presentation device 106 is sent to output module 110, and by data presentation device
The dynamic 3 D that 106 displays are formed by multiple flake images (such as Figure 16 (a), (b), (c) are shown) through dynamic compression and movement
Splice image (shown in such as Figure 16 (d)).
Block diagram and the schematic diagram related to the three-dimensional panorama system of vehicle with reference to the three-dimensional panorama system of above-mentioned vehicle,
With the operation of the three-dimensional panorama method of the vehicle of the explanation present invention.
Figure 17 is the flow chart of the three-dimensional panorama method of the vehicle of the present invention.In the present embodiment, a left side before and after vehicle
The right side is separately installed with fish eye lens 102 to shoot the image (step S20) of vehicle's surroundings.It will be clapped by multiple fish eye lenses 102
The image taken the photograph is respectively converted into flake image data, and is stored in buffer storage 104.
Correction is positioned at the installation site of each fish eye lens 102 in four orientation of the front, rear, left and right of vehicle, such as Fig. 2A
And shown in Fig. 2 B, the installation site of each fish eye lens 102 is corrected according to the display Fig. 2 of data presentation device 106 B image,
Make image captured by each fish eye lens 102 for underbody part image at 1/8th of the bottom of whole image.
When carrying out static three-dimensional image joint program, as shown in Fig. 3 A and Fig. 3 B, car is shot by each fish eye lens 102
Correcting plate 132,134 around 130, as shown in Fig. 3 B correcting plate 134, each fish eye lens 102 photographs a plurality of in straight
The scenery of the horizontal reference line 140 of line and a plurality of linear radiation vertical reference line 142.
It is a plurality of level in circular arc lines to shoot the flake image that correcting plate 134 obtained by each fish eye lens 102
The image of reference line 144 (as shown in Figure 4 A) and a plurality of radiation vertical reference line 146 (as shown in Figure 5) in circular arc lines, and
By a plurality of horizontal reference line 144 in circular arc lines and the video conversion adult fish eye image data of radiation vertical reference line 146.Often
Multiple flake image datas are stored in buffer storage 104 by one fish eye lens 102.
By video camera installation correction module 112 multiple institutes of fish eye lens 102 are received via image input with output module 110
Multiple flake image datas (flake of a plurality of horizontal reference line 144 and radiation vertical reference line 146 in circular arc lines of conversion
Image data), or multiple flake image datas are read in buffer storage 104.
Two end points by video camera installation correction module 112 according to Fig. 4 B horizontal reference line 144 in circular arc lines
Height h1 and h2, when making the height h1 and h2 identical of two end points of horizontal reference line 144 to adjust each fish eye lens
102 horizontal level is horizontal.
Correction module 112 is installed according to the radiation vertical reference line 146 in circular arc lines by video camera, to adjust each fish
The left and right angle position of glasses first 102, the left and right ends of each fish eye lens 102 is photographed the radiation of correcting plate 134 and hang down
Straight reference line 142.
Flake image is obtained as shown in Figure 6A by each shooting figure 3B of fish eye lens 102 correcting plate 134, and flake shadow
Horizontal reference line 144 and radiation vertical reference line 146 as in is because of the relation of fish eye lens 102 and in a manner of circular arc lines
Present.Fish eye lens 102 stores flake image data flake video conversion adult fish eye image data as shown in Figure 6A
In buffer storage 104.
By flake adjustment of image setting module 114 each fish eye lens 102 is received via image input with output module 110
The flake image data or the flake image data of read buffer memory 104 changed.Flake adjustment of image setting module 114
Using known adjustment of image technology by Fig. 6 A in the horizontal reference line 144 of circular arc lines and radiation vertical reference line 146
It is corrected into Fig. 6 B linear horizontal reference line 148 and radiation vertical reference line 150 (step S22).
The horizontal reference line 140 and radiation vertical reference line 142 of correcting plate 134 are reduced into the presentation of straight line from circular arc lines
Mode, then represent the correction program for completing flake image.In brief, the program of flake adjustment of image is mainly in flake image
X and Y-axis each determine one group of optimal elliptic arc length parameter, then according to the parameter will be in flake image data it is all
Pixel is compressed toward image center point, as shown in Fig. 6 C and Fig. 6 D.Assuming that Fig. 6 C are flake image, by flake adjustment of image program
Fig. 6 D correcting image is obtained afterwards, therefore, in the coordinate (x1, y1) of a pixel of the correcting image data of Fig. 6 D correcting image
The coordinate (x2, y2) of the flake image data of corresponding diagram 6C flake image.
Flake correction look-up table can be produced when carrying out flake adjustment of image by flake adjustment of image setting module 114
(step S24), flake correction look-up table are mainly each in the correcting image data after flake adjustment of image is found out in help
The coordinate (x2, y2) of each pixel in the corresponding flake image data of the coordinate (x1, y1) of pixel.Therefore, substantially at least can
There are " X-coordinate " and " Y-coordinate " two flake correction look-up tables.
H1 and h2 in Fig. 7 A represents Y-coordinate point y1 and y2 to the height value of image center line respectively.According to etc. than principle,
Formula (2) can be obtained by derivation in the hope of formula (1), using respective coordinates in obtaining y2 as " Y-coordinate flake corrects look-up table "
The value of (x1, y1).Using same method (such as Fig. 7 B), according to formula (3) x2 can be derived for " correction of X-coordinate flake is searched
The value of respective coordinates (x1, y1) in table ".
Flake adjustment of image setting module 114 is set after by flake adjustment of image for correcting image by flake adjustment of image
The angle of radiation of radiation vertical reference line 150 in adjacent correcting image is adjusted to equal angular and can be with by cover half block 114
It is overlapping, rotation translation look-up table (step S26) can be produced during the rotation and translation of image is corrected.
Correcting image is moved to new coordinate in the lower right-hand corner of image by the origin (0,0) of the correcting image in Fig. 8 A
The position of (xs, ys), and be that origin does rotate counterclockwise θ angles with (xs, ys), as shown in Figure 8 B.It is intended to after trying to achieve rotation and translation
The relation of the coordinate (x2, y2) of pixel in correcting image data and the coordinate (x1, y1) of the pixel in former correcting image data,
It can be tried to achieve by formula (4), (5), meanwhile, also have " X-coordinate " and " Y-coordinate " two different rotation translation look-up tables.Change
Sentence is talked about, and rotated and translation post-equalization image can be learnt according to " X-coordinate " and " Y-coordinate " two rotation translation look-up tables
The position relationship of the coordinate position of each pixel in data and the coordinate position of each pixel in original calibration image data.
By flake adjustment of image setting module 114 according to correcting image adjacent two-by-two (such as left side correcting image and below
Correcting image and right side correcting image and correcting image below) in " radiation vertical reference line ", according to rotating as above and put down
Move the left and right sides correcting image correcting image data in each pixel coordinate position, make it is overlapping after school adjacent two-by-two
" radiation vertical reference line " in positive image can overlap (as shown in Figure 9) as far as possible.In the present embodiment, the left and right sides
The anglec of rotation θ of correcting image is preferably within the scope of 5 °≤θ≤45 °.
It is above-mentioned adjacent correcting image two-by-two is subjected to image overlap when, image generation is spliced by static three-dimensional
116 correct look-up table and rotation translation look-up table to produce a splicing image look-up table according to flake.When static three-dimensional splices shadow
When correcting image as shown in generation module 116 by Fig. 9 carries out image joint, static three-dimensional splicing image generation 116
Learn that each pixel in each correcting image data of image joint to be carried out corresponds to each fish according to the splicing image look-up table
The coordinate position of each pixel in eye image data.
According to the pixel position in Figure 10 A and Figure 11 A splicing image 158, produced by static three-dimensional splicing image
Module 116 calculate respectively each correcting image data the non-overlapping region 154 of image and image overlap area 156 in it is each
The pixel value (i.e. r, g, b pixel value) of pixel.
School of the splicing image 158 in the non-overlapping region 154 of image is calculated by static three-dimensional splicing image generation 116
The pixel value of the pixel of positive image data, the calculating only need to refer to a correcting image, therefore according to splicing image look-up table
Pixel of the correcting image 152 in the non-overlapping region 154 of the image coordinate position corresponding with the pixel in flake image is found,
The pixel value of the coordinate position corresponding according to this can try to achieve splicing of the splicing image 158 in the non-overlapping region 154 of image again
The pixel value (as shown in Figure 10 A) of the pixel of image data.
Assuming that to try to achieve Figure 10 A coordinate (x, y) for splicing the pixel in image 158 in the non-overlapping region 154 of image
Pixel value (r, g, b), the flake of the coordinate (x, y) of the pixel in corresponding splicing image 158 is found according to splicing image look-up table
The coordinate position of pixel in image is (x1+ α 1, y1+ β 1), and wherein weighted value α 1, β 1 numerical value are the ＜ 1 and 0≤β of 0≤α 1
1 ＜ 1.The pixel value of the coordinate (x, y) of the pixel of splicing image is calculated by static three-dimensional splicing image generation 116 to be made
Use the picture of the coordinate (x1, y1), (x1+1, y1), (x1, y1+1) and (x1+1, y1+1) of four pixels in flake image
Element value, as shown in Figure 10 B, and can be tried to achieve by formula (6), (7), (8).
Pixel of the splicing image 158 in image overlap area 156 is calculated by static three-dimensional splicing image generation 116
Pixel value, the calculating needs to refer to two correcting images (as shown in Figure 11 B), therefore finds desire according to splicing image look-up table
Pixel of two correcting images 152 of splicing in image overlap area 156 coordinate corresponding with the pixel in flake image
Position, then the pixel value of the coordinate position corresponding according to this can try to achieve spelling of the splicing image 158 in image overlap area 156
Connect the pixel value (as shown in Figure 11 A) of the pixel of image data.
Assuming that to try to achieve the coordinate (x, y) of the pixel in Figure 11 A image overlap area 156 spliced in image 158
Pixel value (r, g, b), the splicing image data with Figure 11 A is found in image overlap area 156 according to splicing image look-up table
Pixel corresponding to flake image data in pixel coordinate position, find the coordinate bit of the pixel in flake image data
Put and can learn to should coordinate position pixel pixel value.Need to refer to the pixel value of the pixel of two flake image datas
(as shown in Figure 11 B) it is, and corresponding with pixel to be respectively obtained in two flake image datas using formula (6)~(8)
Pixel value (ra, ga, ba) and (rb, gb, bb).Image is carried out in pixel of the splicing image 158 in image overlap area 156
The pixel value of pixel during splicing can be tried to achieve by formula (9)~(11).
Static three-dimensional splice image generation 116 calculate obtain splicing image 158 in the non-overlapping region 154 of image and
The pixel value of pixel in image overlap area 156, just complete by multiple flake images it is calibrated, rotation and translation form it is quiet
State three-dimensional splicing image, and use (step of the splicing image look-up table for following progress dynamic 3 D image joints can be produced
S28)。
When carrying out dynamic 3 D image joint program, the shadow in vehicle traveling is shot by each fish eye lens 102
Thing and obtain flake image, fish eye lens 102 is by captured flake video conversion adult fish eye image data, and by flake image
Data storage is in buffer storage 104 (step S30).
Receive what each fish eye lens 102 was changed with output module 110 via image input by feature extraction module 118
Flake image data in flake image data or read buffer memory 104.Feature extraction module 118 refers to multiple flake shadows
Picture learns that a dynamic splices a dynamic spelling of image to carry out dynamic 3 D image joint according to above-mentioned splicing image look-up table
Connect the coordinate position that each pixel in image data corresponds to each pixel in multiple flake image datas.
Pass through analysis for the image overlap area (region of L and R in such as Figure 12) of flake image adjacent two-by-two
" edge direction " and " gradient intensity " is to obtain " edge type category map ".
A horizontal gradient of each pixel of the image overlap area of flake image data is calculated by feature extraction module 118
Absolute value, a vertical gradient absolute value and an edge direction values, and by each picture of the image overlap area of flake image data
Horizontal gradient absolute value, vertical gradient absolute value and the edge direction values of element are compared with multiple threshold values, and by flake shadow
As each pixel classifications of the image overlap area of data are one kind (step S32) in multiple edge types.
That is, feature extraction is carried out to flake image data by feature extraction module 118 and mainly captures flake image number
According to image overlap area each pixel Gradient Features, the present embodiment can have using above-mentioned " edge type category map "
Efficient the pixel basis of the image overlap area of flake image data its characteristic is divided into " edge type " of at least four classes.
It is above-mentioned by feature extraction module 118 try to achieve image adjacent two-by-two (such as：Left side image and image, the right side below
Side image and image below) image overlap area in Gradient Features after, first find out object (such as car in image overlap area
Diatom or car), further according to size of the object in two adjacent images, to determine two important ginsengs of dynamic adjustment
Number：" ratio of compressing image size " and " degree of movable image " (step S34).
Corresponding object will be found out in the image overlap area of adjacent image by Dynamic parameter adjustment module 120, judged
The ratio that the image gone out in adjacent image is compressed.Such as：Figure 13 (b) and (c) be respectively Figure 13 (a) left side image with
Image below, and " image is shallow white " is partly " image overlap area ".By the classification of above-mentioned Gradient Features, can find out adjacent
More representational object (straight-line segment of such as lane line) in the image overlap area of image, by analyzing being averaged for straight line
Width (fineness), to determine that image needs to implement the parameter of size compression, as Figure 13 (d) and (e) be respectively Figure 13 (b) with
(c) explanation of the diagram of track line width and above-mentioned line segment mean breadth example.
After the dynamic adjustment of " compressing image size ", and movable image produces the schematic diagram of splicing image such as
Shown in Figure 14.Assuming that the length of original left side image is 500 pixel units (shown in such as Figure 14 (a)), compression factor is set to 0.6,
Image length after compression is changed into 300 pixel units (shown in such as Figure 14 (b)).
Figure 14 (c) is the schematic diagram of " left side image " and " rear image " before image movement, after movable image
As a result as shown in Figure 14 (d).In the present embodiment, determined by Dynamic parameter adjustment module 120 in image overlap area credible
During bad object, if finding more than one group of object simultaneously, Dynamic parameter adjustment module 120 will according to these objects institute it is in place
Put, and determine its importance, substantially, can be than the feature weight of the object of up position in the feature of the object of lower position
Will.
, can be by Dynamic parameter adjustment module 120 by each pixel in multiple flake image datas in the explanation of above-mentioned table 1
Carry out rim detection and obtain an average gradient absolute value intensity.
When carrying out dynamic 3 D splicing image, image generation 122 is spliced by dynamic 3 D and looked into according to splicing image
Table is looked for learn that each pixel in dynamic splicing image data corresponds to the coordinate bit of each pixel in each flake image data
Put.
In Figure 15 A, dynamic splicing image 160 is calculated in image without weight by dynamic 3 D splicing image generation 122
The mode of the pixel value of the pixel of image data in folded region 162 splices image generation 116 such as above-mentioned static three-dimensional
The mode of the pixel value of the pixel of image data of the splicing image 158 in the non-overlapping region 162 of image is calculated, the calculating only needs
A flake image is referred to, therefore dynamic splicing image 160 is found in the non-overlapping region of image according to splicing image look-up table
Pixel in 162 coordinate position corresponding with the pixel in flake image, then the pixel of the coordinate position corresponding according to this
Value can try to achieve the pixel value of the pixel of dynamic splicing image data of the dynamic splicing image 160 in the non-overlapping region 162 of image.
Assuming that to try to achieve Figure 15 A dynamic splicing image 160 in image overlap area 164 in pixel coordinate (x,
Y) pixel value (r, g, b), found according to splicing image look-up table and splice image data in image overlap area with Figure 15 A dynamic
The coordinate position of the pixel in flake image data corresponding to pixel in domain 164, finds the pixel in flake image data
Coordinate position and can learn to should coordinate position pixel pixel value.Need to refer to the pixel of two flake image datas
Pixel value, and using formula (6)~(8) with respectively obtained in two flake image datas with pixel (xa, ya) and (xb,
Yb) corresponding pixel value (ra, ga, ba) and (rb, gb, bb) (as shown in fig. 15b).
Divided by dynamic 3 D splicing image generation 122 according to splicing image look-up table in two flake image datas
Do not obtain with pixel (xa, ya) and (xb, yb) (as shown in fig. 15b) corresponding pixel value (ra, ga, ba) and (rb, gb,
Bb), the pixel value of the pixel of image data of the dynamic splicing image 160 in image overlap area 164 and is thus calculated, it can
Tried to achieve (step S36) by formula (17)~(19).
The dynamic obtained splicing image data will be calculated by dynamic 3 D splicing image generation 122 (to spell comprising dynamic
Connect the non-overlapping region of image and the image data of image overlap area of shadow) it is sent to via image input with output module 110
Data presentation device 106, and shown by data presentation device 106 by multiple flake images (such as Figure 16 (a), (b), (c) are shown)
Dynamic 3 D splicing image (shown in such as Figure 16 (d)) (step S38) through dynamically compressing and movement forms.
The present invention provides a kind of three-dimensional panorama system and method for vehicle, and it is dead that its feature reduces splicing when splicing image
The generation at angle, the object for actually appearing in vehicle's surroundings is set not vanish from sight in the region of image joint, also will not generation
The situation that body repeats.
Although the present invention describes as above with reference to preferred embodiment and illustrative accompanying drawing, but should not be as limit.
Those skilled in the art make various modifications to the content of its form and specific embodiment, omit and change, all without departing from the present invention
The scope that claim is advocated.
Claims (10)
1. a kind of three-dimensional panorama method of vehicle, it is characterised in that comprise the following steps：
Multiple flake image datas caused by one correcting plate of shooting are received, there are linear multiple levels on the correcting plate
Reference line and multiple radiation vertical reference lines, the horizontal reference line and radiation vertical reference line are in the flake image data
In circular arc lines；
The multiple flake image data with the horizontal reference line in circular arc lines and radiation vertical reference line is corrected into
Flake correction is produced for multiple correcting image data with linear horizontal reference line and radiation vertical reference line to look into
Table is looked for, obtaining each pixel in the multiple correcting image data according to flake correction look-up table corresponds to the multiple fish
The coordinate position of each pixel in eye image data；
The linear radiation vertical reference line rotation and translation of the correcting image data of neighbouring relations are turned into identical
Angle of radiation overlapping and produce a rotation translation look-up table, look-up table is translated according to the rotation and flake correction is searched
Each pixel that table is obtained in rotated and translation the correcting image data corresponds to each in the flake image data
The coordinate position of pixel；And
Look-up table is translated according to the rotation and flake correction look-up table produces a splicing image look-up table, according to the spelling
Connect each pixel that image look-up table is obtained in the correcting image data and correspond to each pixel in the flake image data
Coordinate position, it is every in correcting image data described in the calculated for pixel values of each pixel in the flake image data
The pixel value of one pixel, and the adjacent correcting image data are subjected to image joint, to produce a static splicing image number
According to.
2. the method as described in claim 1, it is characterised in that include in the step of image of the acquisition correcting plate following
Step：
Multiple fish-eye installation sites are adjusted, make the flake image data caused by the fish eye lens filmed image
Image bottom 1/8th at for a vehicle bottom image；
The height of two end points of one in the horizontal reference line in circular arc lines in the flake image data
Degree, adjust the fish-eye horizontal level；
The radiation vertical reference line in circular arc lines in the flake image data adjusts each fish eye lens, makes
The fish-eye left and right ends photograph the radiation vertical reference line of the correcting plate；And
The image of the correcting plate shot according to the fish eye lens produces the flake image data.
3. the method as described in claim 1, it is characterised in that rotate the linear radiation of the correcting image data
The anglec of rotation of vertical reference line is between 5 ° to 45 °.
4. the method as described in claim 1, it is characterised in that the step of carrying out image joint comprises the following steps：
The default image of each in the multiple correcting image data is obtained according to the splicing image look-up table
Each pixel in non-overlapping region and an image overlap area corresponds to each pixel in the multiple flake image data
Coordinate position；
The image of correcting image data described in the calculated for pixel values of each pixel in the flake image data without
The pixel value of each pixel of overlapping region, it is each in the static non-overlapping region of image for splicing image data to obtain
The pixel value of pixel；And
The image weight of correcting image data described in the calculated for pixel values of each pixel in the flake image data
The pixel value of each pixel in folded region, and image weight is carried out in the correcting image data according to the calculated for pixel values calculated
The pixel value of each pixel in the folded image overlap area, to obtain the static image overlap for splicing image data
The pixel value of each pixel in region.
5. the method as described in claim 1, it is characterised in that after the step of producing a static splicing image data, enter
One step comprises the following steps：
Receive captured flake image data in vehicle traveling；
Each pixel in one dynamic splicing image data is obtained according to the splicing image look-up table and corresponds to the flake image
The coordinate position of each pixel in data；
Calculate a horizontal gradient absolute value, a vertical ladder for each pixel of an image overlap area of the flake image data
Absolute value and an edge direction values are spent, by the water of each pixel of the image overlap area of the flake image data
Flat ladder degree absolute value, vertical gradient absolute value and edge direction values are compared with multiple threshold values, by the flake image
Each pixel classifications of the image overlap area of data are one in multiple edge types；
According to the edge type of each pixel of the image overlap area of the flake image data, obtain in adjacent image
The image overlap area in an object size, to determine that the dynamic splices a compressing image size of image data
Scale parameter and a movable image extent index, and to being the shadow of one in adjacent image in dynamic splicing image data
As data carry out image compression and movement；
Shadow is dynamically spliced according to the calculated for pixel values one of each pixel in the non-overlapping region of an image of the flake image data
As the pixel value of each pixel in the non-overlapping region of the image of data；And
The adjacent shadow that the pixel value of each pixel of the image overlap area according to the flake image data is calculated
The pixel value of each pixel of the image overlap area of the image data of picture is multiplied by an average gradient absolute value intensity, to obtain
The pixel value of each pixel of the image overlap area of the dynamic splicing image data is obtained, wherein to the flake image
Each pixel of the image overlap area of data carries out rim detection and obtains the average gradient absolute value intensity.
A kind of 6. three-dimensional panorama system of vehicle, it is characterised in that including：
One flake adjustment of image setting module, to receive shooting one correcting plate caused by multiple flake image datas, will have
Have and be corrected into multiple horizontal reference lines of circular arc lines and the multiple flake image data of multiple radiation vertical reference lines
Flake correction is produced for multiple correcting image data with linear horizontal reference line and radiation vertical reference line to look into
Table is looked for, the linear radiation vertical reference line for the correcting image data of neighbouring relations is rotated and translation turns into identical
Angle of radiation overlapping and produce a rotation translation look-up table, wherein, have on the correcting plate linear the multiple
Horizontal reference line and the multiple radiation vertical reference line, the horizontal reference line and the spoke in the flake image data
It is in circular arc lines to penetrate vertical reference line, and correcting look-up table according to the flake obtains each pixel in the correcting image data
The coordinate position of each pixel in the corresponding flake image data, look-up table and the flake school are translated according to the rotation
Each pixel that positive look-up table is obtained in rotated and translation the correcting image data is corresponded in the flake image data
Each pixel coordinate position；And
One static three-dimensional splices image generation, to translate look-up table and flake correction look-up table according to the rotation
Produce a splicing image look-up table, correcting image described in the calculated for pixel values of each pixel in the flake image data
The pixel value of each pixel in data, and adjacent correcting image data are subjected to image joint, to produce a static splicing
Image data, wherein, according to corresponding to each pixel that the splicing image look-up table is obtained in the correcting image data
The coordinate position of each pixel in flake image data.
7. system as claimed in claim 6, it is characterised in that also include：
One video camera installs correction module, to adjust multiple fish-eye installation sites, makes the fish eye lens shooting shadow
As caused by 1/8th of the image bottom of the flake image data for a vehicle bottom image, according to described
The height of two end points of one in the horizontal reference line in circular arc lines in flake image data, adjusts the fish
The horizontal level of glasses head, and the radiation vertical reference line in circular arc lines in the flake image data adjusts
Each fish eye lens makes the fish-eye left and right ends photograph the radiation vertical reference line of the correcting plate；And
One image inputs and output module, and the video camera installation correction module and flake adjustment of image setting module are via described
Image inputs the flake image data that the fish eye lens transmission is received with output module, and static three-dimensional splicing image produces
Module is via image input and the output module transmission static splicing image data a to data presentation device.
8. system as claimed in claim 7, it is characterised in that also include：
One feature extraction module, to receive the captured flake image data in vehicle traveling, calculate the flake
A horizontal gradient absolute value, a vertical gradient absolute value and an edge for each pixel of one image overlap area of image data
Direction value, by the horizontal gradient absolute value of each pixel of the image overlap area of the flake image data, hang down
Vertical ladder degree absolute value and edge direction values are compared with multiple threshold values, by the image weight of the flake image data
Each pixel classifications in folded region are one in multiple edge types；
One Dynamic parameter adjustment module, to each pixel of the image overlap area according to the flake image data
Edge type, the size of the object in the image overlap area of adjacent image is obtained, to determine a dynamic splicing shadow
As the compressing image size parameter and a movable image extent index of data, by the shadow of the flake image data
As each pixel of overlapping region carries out rim detection and obtain an average gradient absolute value intensity；And
One dynamic 3 D splices image generation, to according to the compressing image size parameter and the movable image
Extent index and image compression and movement are carried out to one in adjacent image, according to an image of the flake image data without
Dynamically splice each pixel in the non-overlapping region of image of image data described in the calculated for pixel values of each pixel of overlapping region
Pixel value, the pixel value of each pixel of the image overlap area according to the flake image data is calculated
It is strong that the pixel value of each pixel of the image overlap area of the image data of adjacent image is multiplied by an average gradient absolute value
Degree, to obtain the pixel value of each pixel of the image overlap area of the dynamic splicing image data.
9. system as claimed in claim 6, it is characterised in that the flake adjustment of image setting module rotates the correction shadow
As data the linear radiation vertical reference line the anglec of rotation between 5 ° to 45 °.
10. system as claimed in claim 6, it is characterised in that the static three-dimensional splicing image generation performs following
Function：
The default image of each in the multiple correcting image data is obtained according to the splicing image look-up table
Each pixel in non-overlapping region and an image overlap area corresponds to each pixel in the multiple flake image data
Coordinate position；
Calculate the flake image number corresponding to each pixel in the non-overlapping region of the image of the correcting image data
The pixel value of each pixel in, to obtain the picture of each pixel in the non-overlapping region of image of the splicing image data
Element value；And
Calculate the flake image data corresponding to each pixel of the image overlap area of the correcting image data
In each pixel pixel value, and according to carrying out image overlap in the calculated for pixel values shadow correcting image data calculated
The pixel value of each pixel in the image overlap area, to obtain the image overlap area of the splicing image data
In each pixel pixel value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610329549.4A CN107403404A (en) | 2016-05-18 | 2016-05-18 | The three-dimensional panorama system and method for vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610329549.4A CN107403404A (en) | 2016-05-18 | 2016-05-18 | The three-dimensional panorama system and method for vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107403404A true CN107403404A (en) | 2017-11-28 |
Family
ID=60394080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610329549.4A Pending CN107403404A (en) | 2016-05-18 | 2016-05-18 | The three-dimensional panorama system and method for vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107403404A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993686A (en) * | 2017-12-29 | 2019-07-09 | 爱唯秀股份有限公司 | A kind of six segmentation panorama systems that auxiliary drives |
| US20220371510A1 (en) * | 2021-05-24 | 2022-11-24 | Aeon Motor Co., Ltd. | Vehicle rearview warning system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101072288A (en) * | 2007-06-15 | 2007-11-14 | 林进灯 | Method for obtaining fish-eye image correction relationship and fish-eye correction |
| JP2008048443A (en) * | 2007-09-21 | 2008-02-28 | Fujitsu Ltd | Fisheye lens camera apparatus and image extraction method thereof |
| TW201019715A (en) * | 2008-11-12 | 2010-05-16 | Avisonic Technology Corp | Fisheye correction with perspective distortion reduction method and related image processor |
| CN102143302A (en) * | 2010-01-29 | 2011-08-03 | 财团法人工业技术研究院 | All-round aerial view image seamless junction method and system |
| CN103295231A (en) * | 2013-05-14 | 2013-09-11 | 杭州海康希牧智能科技有限公司 | Method for geometrically correcting vertically mapped images of fisheye lenses in fisheye image mosaic |
-
2016
- 2016-05-18 CN CN201610329549.4A patent/CN107403404A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101072288A (en) * | 2007-06-15 | 2007-11-14 | 林进灯 | Method for obtaining fish-eye image correction relationship and fish-eye correction |
| JP2008048443A (en) * | 2007-09-21 | 2008-02-28 | Fujitsu Ltd | Fisheye lens camera apparatus and image extraction method thereof |
| TW201019715A (en) * | 2008-11-12 | 2010-05-16 | Avisonic Technology Corp | Fisheye correction with perspective distortion reduction method and related image processor |
| CN102143302A (en) * | 2010-01-29 | 2011-08-03 | 财团法人工业技术研究院 | All-round aerial view image seamless junction method and system |
| CN103295231A (en) * | 2013-05-14 | 2013-09-11 | 杭州海康希牧智能科技有限公司 | Method for geometrically correcting vertically mapped images of fisheye lenses in fisheye image mosaic |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993686A (en) * | 2017-12-29 | 2019-07-09 | 爱唯秀股份有限公司 | A kind of six segmentation panorama systems that auxiliary drives |
| US20220371510A1 (en) * | 2021-05-24 | 2022-11-24 | Aeon Motor Co., Ltd. | Vehicle rearview warning system |
| US11618379B2 (en) * | 2021-05-24 | 2023-04-04 | Aeon Motor Co., Ltd. | Vehicle rearview warning system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11743416B2 (en) | Apparatus and methods for the storage of overlapping regions of imaging data for the generation of optimized stitched images | |
| CN108600576B (en) | Image processing apparatus, method and system, and computer-readable recording medium | |
| CN106875339B (en) | Fisheye image splicing method based on strip-shaped calibration plate | |
| CN106101635B (en) | Vehicle surrounding image processing method and device | |
| US8134608B2 (en) | Imaging apparatus | |
| KR101592740B1 (en) | Apparatus and method for correcting image distortion of wide angle camera for vehicle | |
| CN103839227B (en) | Fisheye image correcting method and device | |
| US20170295354A1 (en) | Efficient determination of optical flow between images | |
| US10397524B1 (en) | Three-dimensional around view monitoring system of vehicle and method thereof | |
| CN103685917A (en) | Image processor, image processing method and program, and imaging system | |
| JP2006100965A (en) | Monitoring system around vehicle | |
| CN103295231A (en) | Method for geometrically correcting vertically mapped images of fisheye lenses in fisheye image mosaic | |
| DE102017118714A1 (en) | Multi-level camera carrier system for stereoscopic image acquisition | |
| KR101714213B1 (en) | Apparatus for revising image distortion of lens | |
| CN116245722A (en) | Panoramic image stitching system and method applied to heavy high-speed vehicle | |
| CN107403404A (en) | The three-dimensional panorama system and method for vehicle | |
| US10388001B2 (en) | Apparatus and method for compensating image distortion | |
| CN113114975A (en) | Image splicing method and device, electronic equipment and storage medium | |
| CN107071391B (en) | A method of enhancing display 3D naked eye figure | |
| Zhu et al. | Expanding a fish-eye panoramic image through perspective transformation | |
| CN116777755A (en) | Distortion correction method and device, vehicle-mounted equipment and vehicle | |
| CN204652529U (en) | Vehicle Surround Video treatment system | |
| CN115908511A (en) | Method and apparatus for image registration | |
| CN118279203A (en) | Image processing method and device and vehicle | |
| CN117413511A (en) | Electronic device and control method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210105 Address after: 9 / F, 97 Jingye 1st Road, Zhongshan District, Taipei, Taiwan, China Applicant after: Aiweixiu Co.,Ltd. Address before: Tortola, British Virgin Islands Applicant before: Aiweixiu Co.,Ltd. |
|
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20171128 |