CN110555797B - Panoramic aerial view image illumination homogenization processing method based on least square method - Google Patents
Panoramic aerial view image illumination homogenization processing method based on least square method Download PDFInfo
- Publication number
- CN110555797B CN110555797B CN201910706393.0A CN201910706393A CN110555797B CN 110555797 B CN110555797 B CN 110555797B CN 201910706393 A CN201910706393 A CN 201910706393A CN 110555797 B CN110555797 B CN 110555797B
- Authority
- CN
- China
- Prior art keywords
- aerial view
- coefficient
- view image
- panoramic
- channels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000005286 illumination Methods 0.000 title claims abstract description 26
- 238000000265 homogenisation Methods 0.000 title claims abstract description 11
- 238000003672 processing method Methods 0.000 title claims abstract description 7
- 240000004050 Pentaglottis sempervirens Species 0.000 claims description 11
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims description 11
- 238000010586 diagram Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 241000951471 Citrus junos Species 0.000 description 1
- 101100544613 Neurospora crassa (strain ATCC 24698 / 74-OR23-1A / CBS 708.71 / DSM 1257 / FGSC 987) msp-45 gene Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 101150106994 yme2 gene Proteins 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention relates to a panoramic aerial view image illumination homogenization processing method based on a least square method, which comprises the following steps: selecting N fisheye cameras for panoramic view stitching to obtain panoramic aerial view images of the N cameras; dividing the boundary between cameras in the panoramic aerial view, and selecting N pixels of the splice seams as adjacent boundary regions; and multiplying the average pixel value of the boundary areas at two sides of each splice joint by a coefficient by using a least square method to match the pixel values: taking the first distortion diagram as a reference to obtain coefficient matrixes Yme, yme1 and Yme2 of three R, G, B channels; and multiplying the coefficients by global pixel values of R, G, B three channels of the corresponding aerial view image to obtain the panoramic aerial view with uniform illumination.
Description
Technical Field
The invention belongs to the field of computer vision and image processing, and particularly relates to an image illumination homogenization processing method based on a least square method.
Background
In the trend of unmanned gradual realization, various automobile auxiliary driving systems are generated, and the automobile auxiliary driving systems provide technical guarantees for safe driving. The panoramic aerial view is a technology of an automobile auxiliary driving system, a plurality of fish-eye cameras can be used for acquiring scenes in a 360-degree range around a vehicle, and the distorted images at different viewpoints can be spliced into one image at the same aerial view point by utilizing the technologies of fish-eye de-distortion, aerial view transformation, image splicing and the like. However, in actual operation, since the plurality of cameras are different in mounting position and mounting posture, under the condition of large illumination difference, obvious splicing seams easily appear in the spliced images, and each part has obvious brightness difference.
Many high-grade cars begin to popularize the panoramic aerial view function at present, but solutions to the illumination difference are different, and the processing effect is not satisfactory. Therefore, the development of the method which can perform good illumination uniformity treatment on the spliced image and has the speed reaching real time has great market value.
The patent applicant has searched that the present invention patent of illumination uniformization processing after image stitching mainly focuses on an illumination uniformization filter [4] and a uniformization aspect [5] according to compensation coefficients of the overlapping area of the stitched images. Panhua in the patent "a color consistency adjustment method" (application number CN 201811171735.5) [6] a method for color adjustment of a fisheye camera by the least square method is proposed. The patent mainly aims at adjusting the color of a camera according to the adjusting coefficient of the camera, and does not relate to the illumination homogenizing process of color image stitching.
Reference is made to:
[1] wu Jinjie A class of techniques for stitching images with non-uniform illuminance [ D ]. University of electronic science and technology, 2012.
[2] Xiapu, wu Huizhong, xiaoliang, etc. an illumination robust image stitching fusion algorithm [ J ]. Chinese image graphics theory report, 2007,12 (9): 1671-1675.
[3] Cheng Bing, zheng Naning. Panorama stitching for robustness to ambient light [ J ]. Chinese image graphics theory, 2003,8 (2): 135-139.
[4] Sun Xiaoliang, liu Xiaolin, shangyang, zhang Xiaohu, zhang Yuejiang, patent name: a method for obtaining illumination uniformization images, application number: CN201210243811.5
[5] To span, li Bing, zhang Zhiwei, hu Jinmin, chen Xiangcheng, rojie, patent name: method and device for homogenizing surrounding illumination for surrounding-aided parking, application number: CN201811526448.1
[6] Panhua, patent name: a color consistency adjusting method, application number: CN201811171735.5
Disclosure of Invention
The invention aims to provide an effective illumination homogenization algorithm, which is suitable for panoramic aerial views, achieves the fusion of spliced image splicing seams, has the advantages of no longer obvious brightness difference of all parts, high speed, wide application range and the like. The technical scheme of the invention is as follows:
a panoramic aerial view image illumination homogenization processing method based on a least square method comprises the following steps:
1) Selecting N fisheye cameras for panoramic view stitching to obtain panoramic aerial view images of the N cameras;
2) Dividing the boundary between cameras in the panoramic aerial view, and selecting N pixels of the splice seams as adjacent boundary regions for subsequent operation processing;
3) The pixel mean of R, G, B three channels in each adjacent boundary region was calculated. Variables are defined in the following table:
4) And multiplying the average pixel value of the boundary areas at two sides of each splice joint by a coefficient by using a least square method to match the pixel values: for the B channel, the coefficient to be multiplied by each distortion graph is a, B, c, d, e …, N pieces of the coefficients are taken as a total, and a matching formula based on a least square method is as follows:
(aB 12 -bB 21 ) 2 +(bB 23 -cB 32 ) 2 +…+(dB (n-1)n -eB n(n-1) ) 2 +(eB n1 -aB 1n ) 2 ＝0
G. the R channels are matched according to the same method;
six matrices are defined as follows:
fourth, taking the first distortion chart as a reference, a=1, and performing the following operation to obtain coefficient matrixes Yme, yme1 and Yme2 of R, G, B three channels:
Yme＝(Xme T ·Xme) -1 ·Xme T ·Zme
Yme1＝(Xme1 T ·Xme1) -1 ·Xme1 T ·Zme1
Yme2＝(Xme2 T ·Xme2) -1 ·Xme2 T ·Zme2
the three matrixes are all the matrixes with the size of (N-1) 1; yme the first term is the coefficient B to be multiplied by the second bird's-eye view image B value, the second term is the coefficient c to be multiplied by the third bird's-eye view image B value, the third term is the coefficient d to be multiplied by the fourth bird's-eye view image B value, and so on; yme1 and Yme are obtained by the same way;
5) And multiplying the coefficients by global pixel values of R, G, B three channels of the corresponding aerial view image to obtain the panoramic aerial view with uniform illumination.
The invention has the beneficial effects that: 1) Because all pixels are not used for processing, the speed is high, and real-time operation can be achieved; 2) The invention is still applicable when a large vehicle needs to use a plurality of fisheye cameras for looking around and splicing, regardless of the number of the fisheye cameras involved.
Drawings
In the panoramic aerial view spliced in fig. 1, red lines are spliced seams, and six distortion removal diagrams are segmented.
FIG. 2 partial experimental results
(a) Original panoramic aerial view
(b) Panoramic aerial view subjected to illumination homogenization treatment
Detailed Description
The invention discloses a least square method-based panoramic aerial view image illumination homogenization processing method, which comprises the steps of utilizing a plurality of fisheye cameras to carry out 360-degree looking-around splicing, carrying out least square method-based illumination homogenization processing on a spliced panoramic aerial view image, and comprising an adjacent boundary region dividing module, a processing coefficient calculating module and an illumination homogenization processing module, wherein the adjacent boundary region dividing module is an operation region for defining two sides of each splicing seam; the processing coefficient obtaining module obtains R, G, B processing coefficients needed by each part by utilizing RGB data in the pixel block, and in the process, a least square method is applied, and average pixel values of operation areas at two sides of each splicing seam are multiplied by the processing coefficients according to the least square method to match; the illumination uniformity processing module is used for processing R, G, B values of all the pixel blocks of the spliced panoramic aerial view by using the obtained coefficients, and finally, the illumination uniformity effect is achieved. The invention is described below with reference to the drawings and examples.
Firstly, dividing the boundary between cameras in a panoramic aerial view, and selecting 200 rows and 50 columns of pixels on each side of a splice joint as adjacent boundary regions for subsequent operation processing;
second, the pixel mean value of the R, G, B three channels in each adjacent boundary area is calculated, and the variables are defined as follows:
thirdly, applying a least square method idea, multiplying average pixel values of boundary areas at two sides of each splice joint by coefficients to match the pixel values, for example, for a B channel, setting the coefficient to be multiplied by each distortion map as a, B, c, d, e, f, and adopting a matching formula based on the least square method idea as follows:
(aA1-bB1) 2 +(bB4-cC1) 2 +(cC4-dD1) 2 +(dD4-eE1) 2 +(eE4-fF1) 2 +(fF4-aA4) 2 ＝0
for ease of computation, six matrices are defined as follows:
fourth, we use the first distortion chart as a reference, so a=1, and the following operations are performed to obtain coefficient matrices Yme, yme1, yme2 of R, G, B three channels:
Yme＝(Xme T ·Xme) -1 ·Xme T ·Zme
Yme1＝(Xme1 T ·Xme1) -1 ·Xme1 T ·Zme1
Yme2＝(Xme2 T ·Xme2) -1 ·Xme2 T ·Zme2
the three matrixes are five-row and one-column matrixes; yme the first term is a coefficient B to be multiplied by a second bird's-eye view image B value, the second term is a coefficient c to be multiplied by a third bird's-eye view image B value, the third term is a coefficient d to be multiplied by a fourth bird's-eye view image B value, the fourth term is a coefficient e to be multiplied by a fifth bird's-eye view image B value, and the fifth term is a coefficient f to be multiplied by a sixth bird's-eye view image B value; yme1 and Yme are the same.
And fifthly, multiplying the coefficients by global pixel values of R, G, B three channels of the corresponding aerial view image to obtain the panoramic aerial view with uniform illumination.
Claims (1)
1. A panoramic aerial view image illumination homogenization processing method based on a least square method comprises the following steps:
1) Selecting N fisheye cameras for panoramic view stitching to obtain panoramic aerial view images of the N cameras;
2) Dividing the boundary between cameras in the panoramic aerial view, and selecting N pixels of the splice seams as adjacent boundary regions for subsequent operation processing;
3) The pixel mean value of R, G, B three channels in each adjacent boundary area is calculated, and the variables are defined as follows:
4) And multiplying the average pixel value of the boundary areas at two sides of each splice joint by a coefficient by using a least square method to match the pixel values: for the B channel, the coefficient to be multiplied by each distortion graph is a, B, c, d, e …, N pieces of the coefficients are taken as a total, and a matching formula based on a least square method is as follows:
(aB 12 -bB 21 ) 2 +(bB 23 -cB 32 ) 2 +…+(dB (n-1)n -eB n(n-1) ) 2 +(eB n1 -aB 1n ) 2 ＝0
G. the R channels are matched according to the same method;
six matrices are defined as follows:
5) Taking the first distortion chart as a reference, a=1, and performing the following operation to obtain coefficient matrixes Yme, yme1 and Yme2 of R, G, B three channels:
Yme＝(Xme T ·Xme) -1 ·Xme T ·Zme
Yme1＝(Xme1 T ·Xme1) -1 ·Xme1 T ·Zme1
Yme2＝(Xme2 T ·Xme2) -1 ·Xme2 T ·Zme2
the three matrixes are all the matrixes with the size of (N-1) 1; yme the first term is the coefficient B to be multiplied by the second bird's-eye view image B value, the second term is the coefficient c to be multiplied by the third bird's-eye view image B value, the third term is the coefficient d to be multiplied by the fourth bird's-eye view image B value, and so on; yme1 and Yme are obtained by the same way;
6) And multiplying the coefficients by global pixel values of R, G, B three channels of the corresponding aerial view image to obtain the panoramic aerial view with uniform illumination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910706393.0A CN110555797B (en) | 2019-08-01 | 2019-08-01 | Panoramic aerial view image illumination homogenization processing method based on least square method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910706393.0A CN110555797B (en) | 2019-08-01 | 2019-08-01 | Panoramic aerial view image illumination homogenization processing method based on least square method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110555797A CN110555797A (en) | 2019-12-10 |
| CN110555797B true CN110555797B (en) | 2023-04-25 |
Family
ID=68737090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910706393.0A Active CN110555797B (en) | 2019-08-01 | 2019-08-01 | Panoramic aerial view image illumination homogenization processing method based on least square method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110555797B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111369423A (en) * | 2020-02-05 | 2020-07-03 | 天津大学 | YUV domain panoramic aerial view illumination homogenization processing method based on least square method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101425181A (en) * | 2008-12-15 | 2009-05-06 | 浙江大学 | Panoramic view vision auxiliary parking system demarcating method |
| CN109523491A (en) * | 2018-12-13 | 2019-03-26 | 深圳市路畅智能科技有限公司 | Method and apparatus are uniformed for looking around the illumination of looking around that auxiliary is parked |
| CN109948398A (en) * | 2017-12-20 | 2019-06-28 | 深圳开阳电子股份有限公司 | The image processing method and panorama parking apparatus that panorama is parked |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI578268B (en) * | 2012-02-22 | 2017-04-11 | 能晶科技股份有限公司 | Bird view system and compensating method thereof |
| US9801539B2 (en) * | 2013-05-23 | 2017-10-31 | Stiftung Caesar—Center Of Advanced European Studies And Research | Ocular Videography System |
-
2019
- 2019-08-01 CN CN201910706393.0A patent/CN110555797B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101425181A (en) * | 2008-12-15 | 2009-05-06 | 浙江大学 | Panoramic view vision auxiliary parking system demarcating method |
| CN109948398A (en) * | 2017-12-20 | 2019-06-28 | 深圳开阳电子股份有限公司 | The image processing method and panorama parking apparatus that panorama is parked |
| CN109523491A (en) * | 2018-12-13 | 2019-03-26 | 深圳市路畅智能科技有限公司 | Method and apparatus are uniformed for looking around the illumination of looking around that auxiliary is parked |
Non-Patent Citations (2)
| Title |
|---|
| yang jiache.etc.."Blind assessment for stereo images considering binocular characteristics and deep perception map based on deep belief network".《INFORMATION SCIENCES》.2018,全文. * |
| 王晨光 ; .城市道路交通***承载的十大典型应用.通信世界.2012,(第18期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110555797A (en) | 2019-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107133988B (en) | Calibration method and calibration system for camera in vehicle-mounted panoramic looking-around system | |
| CN108769578B (en) | Real-time panoramic imaging system and method based on multiple cameras | |
| US9533618B2 (en) | Method, apparatus and system for processing a display from a surround view camera solution | |
| CN111986129B (en) | HDR image generation method, equipment and storage medium based on multi-shot image fusion | |
| CN110443771B (en) | Method for adjusting consistency of brightness and color of annular view in vehicle-mounted annular view camera system | |
| CN108650495B (en) | Panoramic looking-around system for vehicle and self-adaptive light supplementing method thereof | |
| CN104299185A (en) | Image magnification method, image magnification device and display device | |
| CN102665031A (en) | Video signal processing method and photographic equipment | |
| CN105578021A (en) | Imaging method of binocular camera and apparatus thereof | |
| CN110555797B (en) | Panoramic aerial view image illumination homogenization processing method based on least square method | |
| US9214034B2 (en) | System, device and method for displaying a harmonized combined image | |
| CN110689506A (en) | Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof | |
| CN113362228A (en) | Method and system for splicing panoramic images based on improved distortion correction and mark splicing | |
| US11694308B2 (en) | Images for perception modules of autonomous vehicles | |
| Zhou et al. | Adapting semantic segmentation models for changes in illumination and camera perspective | |
| DE102021124986A1 (en) | IMAGE COLORING FOR VEHICLE CAMERA IMAGES | |
| Liu et al. | Photometric alignment for surround view camera system | |
| CN115936995A (en) | Panoramic splicing method for four-way fisheye cameras of vehicle | |
| CN111491103B (en) | Image brightness adjusting method, monitoring equipment and storage medium | |
| CN104933671A (en) | Image color fusion method | |
| CN114663521A (en) | All-round-view splicing processing method for assisting parking | |
| US10666919B1 (en) | Color and brightness calibration for video stitching | |
| CN110072054B (en) | Terminal equipment and zooming processing method and device for image of terminal equipment | |
| CN114022562A (en) | Panoramic video stitching method and device capable of keeping integrity of pedestrians | |
| CN113808022A (en) | Mobile phone panoramic shooting and synthesizing method based on end-side deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |