CN102831586A - Method for enhancing image/video in real time under poor lighting condition - Google Patents

Abstract

The invention discloses a method for enhancing image/video in real time under a poor lighting condition, and belongs to the technical field of image and video enhancement. The method comprises the following steps of: detecting an input digital image or video; judging the types of lighting conditions; preprocessing the digital image or the video by using a dark channel method to acquire a transmission parameter of an image; performing gamma correction on the digital image or the video and preprocessing the digital image or the video by a light channel method to acquire a transmission parameter of the image or the video; and acquiring a final enhancement result by using the improved image degradation reduction model according to the original image or the video and the transmission parameters which are acquired by the preprocessing. By the method, the image or the video which is acquired under poor lighting conditions of low light, greasy weather, rain snow, high dynamic ranges and the like can be enhanced in real time.

Description

A kind of to the real-time Enhancement Method of the image/video under the abominable illumination condition

Technical field

The invention belongs to figure image intensifying and video enhancement techniques field, particularly be applied to figure image intensifying and video enhancement techniques under the abominable illumination condition.

Background technology

Along with the popularity rate of smart mobile phone is increasingly high, performance of the cam device of configuration is also become better and better on it.These high performance cameras will satisfy the demand that people take pictures or catch video.And when taking pictures or catch video, often be easy to run into abominable illumination condition, like low light, greasy weather, sleet, HDR or the like.These abominable illumination conditions can make the photo of camera shooting and the difficult quality of video be guaranteed.

Simultaneously, the function that does not also have photo and video under the abominable illumination condition of reduction in the design of most portable digital camera.In the field of image and video enhancement, some methods, such as histogram equalization, faster but results are poor, while the other part way, how Kai Ming, who proposed to fog on the dark path algorithm and Dong Xuan et al. the others made the dark passage-based low-light enhancement algorithm, the effect is satisfactory, but the hard real-time processing.These methods all be based on a kind of image degradation also master mould handle.Image degradation also master mould specifically can be represented with following formula:

R(x)=J(x)t(x)+A[1-t(x)] (1)

Wherein: the image after R (x) representative is degenerated, J (x) represents original image, and on behalf of transmission coefficient, A, t (x) represent atmosphere intensity.

Therefore according to above-mentioned model, want to recover original image J (x), then only need obtain transmission coefficient t (x) and atmosphere intensity A and get final product.

The atmosphere intensity A can be through peaked preceding 15% obtaining in the statistical picture pixel.Therefore model hereto, the key of dealing with problems is how to obtain transmission coefficient.

At last, after the calculating transmission coefficient finishes, just can go back original image according to following formula:

$J\left(x\right)=\frac{R\left(x\right)-A}{t\left(x\right)}+A---\left(2\right)$

People such as He Kaiming calculate this transmission coefficient t (x) with regard to the method that has been to use dark channel.

On the other hand, for the enhancement apparatus of abominable illumination condition, some devices and solution have been arranged at present, but all there are some problems in they.With the low light enhancement apparatus is example, and at present the infrared photography machine equipment in monitoring field widespread use has the comparison costliness usually, is difficult to safeguard, working environment is high to temperature requirement, power consumption is big, its frequency range is much narrower as or the like many shortcomings than common camera.These shortcomings all can cause the said equipment to be inconvenient to use.Simultaneously, in the video image of a lot of smart mobile phones is used, owing to reasons such as power consumption are impossible integrated infrared system.And at present existing use the algorithm that uses usually computation complexity be big, it is many to regulate parameter, inconvenient domestic consumer uses.

Summary of the invention

The objective of the invention is for overcoming the weak point of prior art, proposed the real-time Enhancement Method of video image under a kind of abominable illumination condition.This method can strengthen in real time and comprises video and the picture that obtains under the abominable illumination condition such as low light, greasy weather, sleet, HDR.

The a kind of of the present invention's proposition shows that to the real-time Enhancement Method of the image under the abominable illumination condition it is characterized in that, this method may further comprise the steps:

1) obtains a numbered image;

2) calculating and detection obtain this digital picture illumination condition type, if testing result is greasy weather or sleet, then get into step 3); If testing result is low light or HDR; Then get into step 4), otherwise said digital picture is a normal picture, directly gets into step 6);

3) use the dark channel method to obtain the transmission parameter of image to said digital picture to carrying out pre-service;

4) said digital picture is carried out gamma correction and uses bright passage method to carry out pre-service obtaining getting into step 7) behind the transmission parameter of image;

5) use improved image degradation also master mould obtain final enhancing result according to the transmission parameter processing that original image and pre-service obtain;

As the digital picture output after strengthening.

The present invention also proposes the real-time video Enhancement Method under a kind of abominable illumination condition, it is characterized in that, this method may further comprise the steps:

1) obtains a frame of video of a group of picture in the video;

2) judge whether this frame of video is first frame of this group of picture, if be, then gets into step 3), otherwise get into step 4);

3) this frame of video is carried out illumination condition type detection and the setting of illumination condition type, get into step 6) after accomplishing;

4) judge that further its this frame of video denys a scene change,, then get into step 5), otherwise get into step 6) if be;

5) this frame of video is carried out illumination condition type decision and the setting of illumination condition type;

6) judge whether this frame of video is key frame, if be then to get into step 7) otherwise the entering step 8);

7) read the illumination condition type that said key video sequence frame configures; If the illumination condition type of this frame of video is greasy weather or sleet; Then get into step 9),, then get into step 10) if the illumination condition type of this frame of video is low light or HDR; Otherwise said frame of video is the normal video frame, directly gets into step 12);

8) use method for estimating that said frame of video is distributed transmission parameter, and get into step 11);

9) using the dark channel method to carry out pre-service to said frame of video obtains getting into step 11) behind the frame of video transmission parameter;

10) said frame of video is carried out gamma correction and uses bright passage method to carry out pre-service obtaining getting into step 11) behind the transmission parameter of frame of video;

11) use improved image degradation also master mould obtain final enhancing result according to the transmission parameter processing that original video frame and pre-service obtain;

12) final enhancing result is exported as the frame of video after strengthening.

Characteristics of the present invention and beneficial effect:

The invention discloses the real-time Enhancement Method of video image under a kind of abominable illumination condition.Abominable illumination condition comprises low light, greasy weather, sleet, HDR etc.This method can be handled image and the video that obtains under the above-mentioned abominable illumination condition in real time.This method at first detects the image or the video of input, judges its illumination condition type, and selects corresponding preprocess method to do pre-service, uses improved image degradation reduction models treated afterwards, obtains final process result at last.

This method can strengthen in real time and comprises video and the picture that obtains under the abominable illumination condition such as low light, greasy weather, sleet, HDR.

Description of drawings

Fig. 1 is the block diagram and the flow process of the overall architecture of the image enchancing method among the present invention.

Fig. 2 detects the block diagram and the flow process of illumination condition type method for the present invention.

Fig. 3 is block diagram and the flow process of the present invention to the preprocess method of greasy weather or sleet image.

Fig. 4 is block diagram and the flow process of the present invention to the preprocess method of low light or high-dynamics image.

Fig. 5 is the block diagram and the flow process of improved image degradation reduction model method of the present invention.

Fig. 6 is the block diagram and the flow process of the overall architecture of the video Enhancement Method among the present invention.

A kind of pixel sampling contrast pattern that Fig. 7 proposes for the present invention.

Algorithm block diagram and flow process that Fig. 8 utilizes method for estimating to strengthen in real time for video for the present invention.

Embodiment

The real-time Enhancement Method of the image/video under the abominable illumination condition that the present invention proposes combines accompanying drawing and embodiment to specify as follows:

Overall flow to the real-time Enhancement Method of image under the abominable illumination condition of the present invention is as shown in Figure 1, and this method may further comprise the steps:

1) obtains a numbered image;

2) calculating and detection obtain this digital picture illumination condition type, if testing result is greasy weather or sleet, then get into step 3); If testing result is low light or HDR; Then get into step 4), otherwise said digital picture is a normal picture, directly gets into step 6);

3) use the dark channel method to obtain the transmission parameter of image to said digital picture to carrying out pre-service;

4) said digital picture being carried out gamma (gamma) proofreaies and correct and uses bright passage method to carry out pre-service and obtain getting into step 7) behind the transmission parameter of image;

5) use improved image degradation also master mould obtain final enhancing result according to the transmission parameter processing that original image and pre-service obtain;

6) as the digital picture output after strengthening.

Above-mentioned steps 2) calculate and to obtain the idiographic flow of this digital picture illumination condition type as shown in Figure 2, the method includes the steps of:

2-1) digital picture of obtaining is carried out the conversion of rgb space to yuv space; And brightness Y carried out statistics with histogram; The ratio of utilizing the Y value whether to account for total pixel greater than 150 pixel after the statistics surpasses 80% condition and detects this image whether mist is arranged; If testing result thinks then that for being this digital picture is greasy weather or sleet image, otherwise get into step 2);

2-2) this digital picture is carried out image inversion, and whether have mist to detect,, otherwise think that this digital picture is a normal picture if testing result thinks then that for being this digital picture is low light or high dynamic range images to the digital picture after the counter-rotating

Above-mentioned steps 3) to obtain the idiographic flow of transmission parameter of image as shown in Figure 3 to carrying out pre-service said digital picture to be used the dark channel method, and the method includes the steps of:

3-2) calculate atmosphere intensity level A to obtaining digital picture;

3-2) to obtaining each pixel of digital picture, use formula (3) to calculate the dark channel value of this pixel:

The dark channel of a pixel defines as follows:

$J^{\mathrm{dark}}\left(x\right)=\underset{c\∈\{r,g,b\}}{\min }\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{\min }\left(J^c\left(y\right)\right)\right)---\left(3\right)$

Wherein: J ^Dark(x) the dark channel value of the pixel of expression x position, min representes to get minimum value, and c representes certain passage, and generally leading to all has r, g, three passages of b, J ^c(y) the pixel value size of expression y position c passage, the position range of Ω (x) remarked pixel;

3-3) use formula (4) to calculate the transmission parameter at this pixel place:

$t\left(x\right)=1-\underset{c\∈\{r,g,b\}}{\min }\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{\min }\left(\frac{J^c\left(y\right)}{A}\right)\right)---\left(4\right)$

Above-mentioned steps 4) to obtain the idiographic flow of transmission parameter of image as shown in Figure 4 to carrying out pre-service said digital picture to be used bright passage method, and the method includes the steps of:

4-1) to obtaining the calculating atmosphere intensity level of digital picture;

4-2) carry out gamma (gamma) correction to obtaining digital picture;

4-3) use following formula to calculate the bright channel value of this pixel to each pixel of obtaining digital picture

The bright channel definition of a pixel is following:

$J^{\mathrm{bright}}\left(x\right)=\underset{c\∈\{r,g,b\}}{\max }\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{\max }\left(J^c\left(y\right)\right)\right)---\left(5\right)$

Wherein: J ^Bright(x) the bright channel value of the pixel of expression x position, max representes to get maximal value;

4-4) use formula (6) to calculate the transmission parameter at this pixel place:

$t\left(x\right)=\frac{\underset{c\∈\{r,g,b\}}{\max }\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{\max }\left(\frac{J^c\left(y\right)}{A}\right)\right)-A}{255-A}---\left(6\right)$

Above-mentioned steps 5) the improved image degradation of described use also the transmission parameter that obtains according to original image and pre-service of master mould to handle the idiographic flow that obtains final enhancing result as shown in Figure 5, the method includes the steps of:

5-1) obtain transmission parameter and the original image that pre-service obtains;

5-2) use improved image degradation reduction model formation (7) to calculate the pixel value of this pixel position after strengthening to each pixel of the original image that obtains;

$J\left(x\right)=\frac{R\left(x\right)-A}{t\left(x\right)P\left(x\right)}+A---\left(7\right)$

Wherein

R (x) is the pixel value size of the original image x position that gets access to

$P\left(x\right)=\left\{\begin{array}{cc}K& 0<t\left(x\right)\&le;0.5\\ \mathrm{Kt}\left(x\right)+\frac{M}{t\left(x\right)}& 0.5<t\left(x\right)\&le;1\end{array}\right.---\left(8\right)$

K=0.9 wherein, M=0.5

5-3) as the digital picture output after strengthening.

The overall flow of abominable illumination condition real-time video Enhancement Method of the present invention is as shown in Figure 6, and the concrete steps of this method are:

1) obtains a frame of video of a group of picture in the video;

2) judge whether this frame of video is first frame of this group of picture, if be, then gets into step 3), otherwise get into step 4);

3) this frame of video is carried out illumination condition type detection and the setting of illumination condition type, get into step 6) after accomplishing;

4) judge that further its this frame of video denys a scene change,, then get into step 5), otherwise get into step 6) if be;

5) this frame of video is carried out illumination condition type decision and the setting of illumination condition type;

6) judge whether this frame of video is key frame, if be then to get into step 7) otherwise the entering step 8);

7) read the illumination condition type that said key video sequence frame configures; If the illumination condition type of this frame of video is greasy weather or sleet; Then get into step 9),, then get into step 10) if the illumination condition type of this frame of video is low light or HDR; Otherwise said frame of video is the normal video frame, directly gets into step 12);

8) use method for estimating that said frame of video is distributed transmission parameter, and get into step 11);

9) using the dark channel method to carry out pre-service to said frame of video obtains getting into step 11) behind the frame of video transmission parameter;

10) said frame of video being carried out gamma (gamma) proofreaies and correct and uses bright passage method to carry out pre-service and obtain getting into step 11) behind the transmission parameter of frame of video;

11) use improved image degradation also master mould obtain final enhancing result according to the transmission parameter processing that original video frame and pre-service obtain;

12) as the frame of video output after strengthening.

Above-mentioned steps 8) use method for estimating to distribute the idiographic flow of transmission parameter as shown in Figure 8 to said frame of video, its concrete steps comprise:

8-1) said frame of video is used motion estimation techniques, obtain the motion vector of each motion estimation block; (in this step, specifically adopted the pixel sampling contrast patterns of Fig. 7.This pattern is represented a motion estimation block in the frame of video, and each little lattice is represented a pixel, and wherein the part of black representes to do absolute difference and (SAD) point, and the part of white is represented the point that need not do.The motion vector that uses this pattern to take exercises and estimate and obtain each motion estimation block of whole video frame.）

8-2) to each motion estimation block, judge according to the motion vector that obtains whether this piece is the inter-frame forecast mode piece, if be then to get into step 8-3), otherwise get into step 8-4);

8-3) be the transmission parameter of this piece assigned references piece, think that promptly the transmission parameter of this piece is the same with the transmission parameter of reference block, thereby directly duplicate; Get into step 8-6 after accomplishing);

8-4) be this piece calculating transmission parameter (method of this step and above-mentioned steps 7), 9 according to abominable illumination condition type), 10) method identical);

The transmission parameter that 8-5) judges whether all motion estimation blocks all calculates and finishes, and finishes if all calculate, and then gets into step 8-6), otherwise get into step 8-2);

8-6) all motion estimation block distribution are obtained or calculate transmission parameter output.

Claims (6)

1. one kind to the real-time Enhancement Method of the image under the abominable illumination condition, it is characterized in that this method may further comprise the steps:

1) obtains a numbered image;

2) calculating and detection obtain this digital picture illumination condition type, if testing result is greasy weather or sleet, then get into step 3); If testing result is low light or HDR; Then get into step 4), otherwise said digital picture is a normal picture, directly gets into step 6);

3) use the dark channel method to obtain the transmission parameter of image to said digital picture to carrying out pre-service;

4) said digital picture is carried out gamma correction and uses bright passage method to carry out pre-service obtaining getting into step 7) behind the transmission parameter of image;

5) use improved image degradation also master mould obtain final enhancing result according to the transmission parameter processing that original image and pre-service obtain;

6) as the digital picture output after strengthening.

2. the method for claim 1 is characterized in that, said step 2) calculate and obtain specifically may further comprise the steps of this digital picture illumination condition type:

2-1) digital picture of obtaining is carried out the conversion of rgb space to yuv space; And brightness Y carried out statistics with histogram; The ratio of utilizing the Y value whether to account for total pixel greater than 150 pixel after the statistics surpasses 80% condition and detects this image whether mist is arranged; If testing result thinks then that for being this digital picture is greasy weather or sleet image, otherwise get into step 2);

2-2) this digital picture is carried out image inversion, and whether have mist to detect,, otherwise think that this digital picture is a normal picture if testing result thinks then that for being this digital picture is low light or high dynamic range images to the digital picture after the counter-rotating.

3. the method for claim 1 is characterized in that, said step 3) uses the dark channel method that the transmission parameter that carries out pre-service and obtain image specifically may further comprise the steps to said digital picture:

3-1) calculate atmosphere intensity level A to obtaining digital picture;

3-2) to obtaining each pixel of digital picture, use formula (3) to calculate the dark channel value of this pixel:

$J^{\mathrm{dark}}\left(x\right)=\underset{c\&Element;\{r,g,b\}}{\min }\left(\underset{y\&Element;\mathrm{\&Omega;}\left(x\right)}{\min }\left(J^c\left(y\right)\right)\right)---\left(3\right)$

Wherein: J ^Dark(x) the dark channel value of the pixel of expression x position, min representes to get minimum value, and c representes certain passage, and generally leading to all has r, g, three passages of b, J ^c(y) the pixel value size of expression y position c passage;

The position range of Ω (x) remarked pixel;

3-3) use formula (4) to calculate the transmission parameter at this pixel place:

$t\left(x\right)=1-\underset{c\&Element;\{r,g,b\}}{\min }\left(\underset{y\&Element;\mathrm{\&Omega;}\left(x\right)}{\min }\left(\frac{J^c\left(y\right)}{A}\right)\right)---\left(4\right).$

4. the method for claim 1 is characterized in that, said step 4) uses bright passage method that the transmission parameter that carries out pre-service and obtain image specifically may further comprise the steps to said digital picture:

4-1) to obtaining the calculating atmosphere intensity level of digital picture;

4-2) carry out gamma correction to obtaining digital picture;

4-3) use formula (5) to calculate the bright channel value of this pixel to each pixel of obtaining digital picture:

$J^{\mathrm{bright}}\left(x\right)=\underset{c\&Element;\{r,g,b\}}{\max }\left(\underset{y\&Element;\mathrm{\&Omega;}\left(x\right)}{\max }\left(J^c\left(y\right)\right)\right)---\left(5\right)$

Wherein: J ^Bright(x) the bright channel value of the pixel of expression x position, max representes to get maximal value;

4-4) use formula (6) to calculate the transmission parameter at this pixel place:

$t\left(x\right)=\frac{\underset{c\&Element;\{r,g,b\}}{\max }\left(\underset{y\&Element;\mathrm{\&Omega;}\left(x\right)}{\max }\left(\frac{J^c\left(y\right)}{A}\right)\right)-A}{255-A}---\left(6\right).$

5. the method for claim 1 is characterized in that, said step 5) use improved image degradation also the transmission parameter that obtains according to original image and pre-service of master mould handle and obtain final enhancing result and specifically may further comprise the steps:

5-1) obtain transmission parameter and the original image that pre-service obtains;

5-2) use improved image degradation reduction model formation (7) to calculate the pixel value of this pixel position after strengthening to each pixel of the original image that obtains;

$J\left(x\right)=\frac{R\left(x\right)-A}{t\left(x\right)P\left(x\right)}+A---\left(7\right)$

Wherein: R (x) is the pixel value size of the original image x position that gets access to;

$P\left(x\right)=\left\{\begin{array}{cc}K& 0<t\left(x\right)\&le;0.5\\ \mathrm{Kt}\left(x\right)+\frac{M}{t\left(x\right)}& 0.5<t\left(x\right)\&le;1\end{array}\right.---\left(8\right)$

K=0.9 wherein, M=0.5

5-3) as the digital picture output after strengthening.

6. the real-time video Enhancement Method under the abominable illumination condition is characterized in that this method may further comprise the steps:

1) obtains a frame of video of a group of picture in the video;

2) judge whether this frame of video is first frame of this group of picture, if be, then gets into step 3), otherwise get into step 4);

3) this frame of video is carried out illumination condition type detection and the setting of illumination condition type, get into step 6) after accomplishing;

4) judge that further its this frame of video denys a scene change,, then get into step 5), otherwise get into step 6) if be;

5) this frame of video is carried out illumination condition type decision and the setting of illumination condition type;

6) judge whether this frame of video is key frame, if be then to get into step 7) otherwise the entering step 8);

7) read the illumination condition type that said key video sequence frame configures; If the illumination condition type of this frame of video is greasy weather or sleet; Then get into step 9),, then get into step 10) if the illumination condition type of this frame of video is low light or HDR; Otherwise said frame of video is the normal video frame, directly gets into step 12);

8) use method for estimating that said frame of video is distributed transmission parameter, and get into step 11);

9) using the dark channel method to carry out pre-service to said frame of video obtains getting into step 11) behind the frame of video transmission parameter;

10) said frame of video is carried out gamma correction and uses bright passage method to carry out pre-service obtaining getting into step 11) behind the transmission parameter of frame of video;

11) use improved image degradation also master mould obtain final enhancing result according to the transmission parameter processing that original video frame and pre-service obtain;

12) final enhancing result is exported as the frame of video after strengthening.

Images