CN112907582B - Mine-oriented image saliency extraction defogging method and device and face detection - Google Patents
Mine-oriented image saliency extraction defogging method and device and face detection Download PDFInfo
- Publication number
- CN112907582B CN112907582B CN202110330453.0A CN202110330453A CN112907582B CN 112907582 B CN112907582 B CN 112907582B CN 202110330453 A CN202110330453 A CN 202110330453A CN 112907582 B CN112907582 B CN 112907582B
- Authority
- CN
- China
- Prior art keywords
- image
- defogging
- value
- intensity
- saliency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000001514 detection method Methods 0.000 title claims abstract description 19
- 230000005540 biological transmission Effects 0.000 claims abstract description 74
- 238000003860 storage Methods 0.000 claims abstract description 66
- 238000010586 diagram Methods 0.000 claims abstract description 55
- 238000004458 analytical method Methods 0.000 claims abstract description 14
- 230000009467 reduction Effects 0.000 claims description 19
- 230000004927 fusion Effects 0.000 claims description 10
- 238000009499 grossing Methods 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 2
- 230000015556 catabolic process Effects 0.000 abstract description 4
- 238000006731 degradation reaction Methods 0.000 abstract description 4
- 239000000428 dust Substances 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 239000003245 coal Substances 0.000 description 3
- 239000010419 fine particle Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 125000001475 halogen functional group Chemical group 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 201000003152 motion sickness Diseases 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The application relates to a mine-oriented image saliency extraction defogging method and device and face detection, belongs to the technical field of image processing, and solves the problems of defogging image distortion, supersaturation and edge vignetting artifact caused by different underground fog dust concentration, uneven light sources and different image saliency degrees. The method comprises the following steps: acquiring an original fog storage image under a mine; performing brightness degradation processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimated value according to the fog storage image after brightness degradation; obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, the preset small-scale value and the large-scale value; fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram; and (5) reversely solving the atmospheric scattering model based on the fused transmission diagram and the atmospheric light estimated value to obtain a defogging image.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a mine-oriented image saliency extraction defogging method and device and face detection.
Background
The existing underground coal mine monitoring system has been widely used, safety monitoring and linkage control in dangerous areas can be achieved, personnel can be liberated from dangerous environments, and the guaranteeing capacity of coal mine safety production and the monitoring and early warning level of natural disasters of the coal mine can be improved. However, the special environment under the mine presents serious challenges for image acquisition under the mine. In the under-mine environment, many fine particles are gathered in the air, and the fine particles can generate absorption or refraction phenomena to light, influence normal radiation of the light, and when the fine particles are gathered in a certain area too much, severe environment phenomena such as fog, haze and the like can be generated, so that the color, contrast, saturation, detail and the like of an image captured by a monitoring system are often very adversely affected. To address this particular submerged environmental impact, a number of different technological routes of defogging algorithms have been proposed, of which defogging algorithms (Dark Channel Prior, DCP) based on dark channel a priori models are outstanding. The algorithm finds an a priori rule that for a pixel point of an image without haze having no sky, the pixel intensity of at least one color channel is very low. And acquiring global atmospheric light and a transmission diagram through the prior rule, and finally calculating a defogging diagram based on the global atmospheric light model.
Currently, global atmospheric light is usually obtained by taking the average value of the real pixel intensities corresponding to the point 0.1% before the intensity in the dark channel map, and the error of the estimated atmospheric light value is reduced to a certain extent, but still has a big problem. The light source under the mine is generally a point light source, and reflection of the point light source or the mirror surface can cause an excessively bright area of the acquired image, and the pixel intensity in the area is far greater than that of other pixels, so that serious estimation errors are caused when the atmospheric light estimated value is obtained, and the defogging image is seriously color distorted. In addition, for the same image, people have different attention degrees on different objects, the significance degree of each pixel in the image is different, defogging and supersaturation can be caused by defogging of a small-scale model by adopting a DCP algorithm, and vignetting and artifact can be caused at the edge position of the object by defogging of a large-scale model by adopting the DCP algorithm.
Therefore, due to the special image acquisition environment under the mine, the problems of defogging image distortion, supersaturation and edge vignetting artifact caused by different fog dust concentration, nonuniform light source and different image significance degree under the mine are difficult to solve by adopting the existing DCP image defogging scheme.
Disclosure of Invention
In view of the above analysis, the embodiment of the application aims to provide a mine-oriented image saliency extraction defogging method and device and face detection, which are used for solving the problems of defogging image distortion, supersaturation and edge corona artifact caused by different underground fog dust concentration, uneven light source and different image saliency degrees.
On one hand, the embodiment of the application provides a mine-oriented image saliency extraction defogging method, which comprises the following steps:
acquiring an original fog storage image under a mine;
performing brightness reduction processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimated value according to the fog storage image after brightness reduction;
obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, a preset small-scale value and a large-scale value;
fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model;
constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram;
and based on the fused transmission diagram and the atmospheric light estimated value, reversely solving the atmospheric scattering model to obtain a defogging image.
Further, the histogram analysis-based brightness reduction processing is performed on the original fog storage image, and the method comprises the following steps:
dividing the original fog-storing image into three color channels of red, green and blue;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram;
calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channel c The formula is as follows:
in Max c 、Mea c And Mid c Respectively representing the intensity maximum, the intensity average and the intensity median of each color channel, c representsColor channels, and c epsilon { r, g, b }, r, g, b representing three color channels of red, green, blue in the color channels, respectively;
and comparing the brightness factor of each color channel with a brightness threshold set by an image, setting the intensity of a pixel point with the intensity larger than the maximum value of 0.9 times of the intensity in the channel as the intensity average value of the channel if the brightness factor is smaller than or equal to the brightness threshold, otherwise, not reducing the intensity.
Further, the calculating the atmospheric light estimated value according to the fog storage image after the brightness is reduced comprises the following steps:
obtaining a dark channel diagram according to the fog storage image with the brightness reduced;
finding out the pixel point with the maximum intensity of 0.1% in the dark channel diagram;
finding out the pixel point corresponding to the 0.1% pixel point in the fog storage image after the brightness is reduced;
and taking the average intensity value of the pixel points corresponding to the fog storage image after the brightness is reduced as an atmospheric light estimated value.
Further, the value of the small scale is set to 1;
the large-scale value k is set according to the resolution of the image and is expressed as follows:
where w and h represent the width and height of the image, respectively.
Further, the large-scale transmission diagram t pa (x) And a small-scale transmission diagram t pi (x) The expression is as follows:
wherein alpha representsDistant view fog adding parameter A c Representing the color channel value corresponding to the atmospheric light estimated value in the dark channel, Ω (x) represents a local area centered on the pixel point x, I c And (y) is a color channel diagram corresponding to the original fog storage image.
Further, taking a significance value in the significance map as a weight, and carrying out weighted fusion on the large-scale transmission map and the small-scale transmission map to obtain the significance extraction defogging model, wherein the significance extraction defogging model is expressed as follows:
wherein t is sed (x) Represents the saliency extraction defogging transmission diagram at the pixel point x, W sig (x) Andrespectively representing t in the saliency extraction defogging model on the pixel point x pi (x) And t pa (x) Ratio of W sig (x) At t pi (x) Saliency value of saliency map corresponding to the point and +.>
Further, the constraint on the saliency extraction defogging model by using L2 regularization is performed to obtain a fused transmission diagram, and the method comprises the following steps:
extracting defogging model constraints on the salience through L2 regularization as follows:
wherein lambda is t Represents regularization coefficient, R (t sed (x) A smoothing term for the transmission map;
r (t) sed (x) Neglecting), obtaining:
smoothing the above equation by using a guide filter to obtain smoothed t sed (x)。
Further, the inverse atmospheric scattering model yields defogging images, expressed as:
wherein J is sed (x) And (3) representing defogging images, and I (x) representing original fog storage images under the mine.
On the other hand, the embodiment of the application provides a mine-oriented image saliency extraction defogging device, which comprises:
the fog storage image acquisition module is used for acquiring an original fog storage image under the mine;
the atmosphere light estimation module is used for carrying out brightness value reduction processing on the original fog storage image based on histogram analysis and calculating an atmosphere light estimation value according to the fog storage image after brightness value reduction;
the fusion transmission diagram obtaining module is used for obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, the preset small-scale value and the large-scale value; fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram;
and the defogging image obtaining module is used for obtaining defogging images by reversely solving the atmospheric scattering model based on the fused transmission diagram and the atmospheric light estimated value.
The embodiment of the application also provides a face detection method for the mine, which is used for processing the collected images in the mine according to the image saliency extraction defogging method of any one of claims 1-8 to obtain defogging images; and detecting the face in the defogging image by using a plane rotation face detection method.
Compared with the prior art, the application has at least one of the following beneficial effects:
the application provides a mine-oriented image saliency extraction defogging method, a device and a face detection,
1. the original fog storage image collected in the mine is subjected to brightness value reduction processing through histogram analysis, and the fog storage image after brightness value reduction is used for calculating an atmospheric light estimated value, so that an excessive bright area in the image caused by a point light source and specular reflection in the mine is eliminated, the accuracy of the atmospheric light estimated value can be further improved, the quality of a defogging image is further improved, and the color distortion of the defogging image is avoided;
2. according to the method, a large-scale transmission image and a small-scale transmission image are fused according to a saliency image of an original fog storage image, so that a fused transmission image is obtained, and finally, a defogging image is obtained, the saliency division is carried out on the image, the saliency image is used as a weight fusion, the edges of objects in the image can be better extracted, the small-scale defogging is carried out on the area with high saliency, the large-scale defogging is carried out on the area with low saliency, the quality of the defogging image can be effectively improved, and the supersaturation and edge halation artifact phenomena of the defogging image are avoided;
3. the L2 regularization constraint is carried out by extracting the defogging model in the fusion process, so that the edge region in the image is closer to the small-scale transmission diagram, and the halo artifact generated by the edge of the object in the defogging image is further avoided.
In the application, the technical schemes can be mutually combined to realize more preferable combination schemes. Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the application, like reference numerals being used to refer to like parts throughout the several views.
Fig. 1 is a schematic flow chart of a method for extracting and defogging significance of an image facing a mine in embodiment 1 of the present application;
FIG. 2 is a logic block diagram of a mine-facing image saliency extraction defogging method in embodiment 1 of the present application;
fig. 3 is a flow chart of the full RIP face detection in embodiment 3 of the present application.
Detailed Description
The following detailed description of preferred embodiments of the application is made in connection with the accompanying drawings, which form a part hereof, and together with the description of the embodiments of the application, are used to explain the principles of the application and are not intended to limit the scope of the application.
Example 1
The embodiment 1 of the application discloses a mine-oriented image saliency extraction defogging method, wherein a flow chart is shown in fig. 1, a logic block diagram is shown in fig. 2, and the method comprises the following steps:
and acquiring an original fog storage image of the mine. Specifically, the original fog storage image can be obtained through video images shot by a fixed camera or an onboard camera under the mine.
And carrying out brightness degradation processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimated value according to the fog storage image after brightness degradation.
And obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, the preset small-scale value and the large-scale value.
And fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model.
And constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram.
And (5) reversely solving the atmospheric scattering model based on the fused transmission diagram and the atmospheric light estimated value to obtain a defogging image.
Compared with the prior art, the mine-oriented image saliency extraction defogging method provided by the embodiment carries out brightness reduction processing on the original fog storage image acquired in the mine through histogram analysis, fuses a large-scale transmission image and a small-scale transmission image according to the saliency image of the original fog storage image, and utilizes L2 regularization constraint to enable the obtained defogging image to avoid the problems of image distortion, supersaturation and edge corona artifact caused by different underground fog dust concentration, uneven light sources and different image saliency degrees, and can obtain a high-quality defogging image.
When the method is implemented, the brightness value reduction processing is carried out on the original fog storage image based on the histogram analysis, and the method comprises the following steps:
dividing an original fog-storing image into three color channels of red, green and blue;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram; it should be noted that the intensity, i.e. the amplitude.
Calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channel c The formula is as follows:
in Max c 、Mea c And Mid c Respectively representing the intensity maximum value, the intensity average value and the intensity median of each color channel, c represents the color channel, and c epsilon { r, g, b }, r, g, b respectively representing the three color channels of red, green and blue in the color channel.
And comparing the brightness factor of each color channel with a brightness threshold set by an image, setting the intensity of a pixel point with the intensity larger than the maximum value of 0.9 times of the intensity in the channel as the intensity average value of the channel if the brightness factor is smaller than or equal to the brightness threshold, otherwise, not reducing the intensity. It should be noted that a low luminance factor indicates the presence of an over-bright region in the image; the pixel point with the intensity larger than the maximum value of 0.9 times of the intensity in the image is selected for reducing the value, so that the over-bright pixel point in the over-bright area of the image can be better reduced, the pixel point with normal brightness cannot be reduced by mistake, and the over-bright pixel point cannot be regarded as the normal brightness pixel point by mistake, so that the brightness after the reduction is not too low or the brightness is insufficient, and the effect of eliminating the over-bright area of the image caused by point light sources and specular reflection under a mine can be better achieved by the image after the reduction.
Preferably, the brightness threshold range of the image is set to be 0,1, and the brightness threshold is set to be 0.7 according to the experimental result in the embodiment, and the setting to be 0.7 can better and more accurately identify the over-bright area in the image, and the calculated atmospheric light estimated value is more accurate by reducing the brightness of the over-bright area.
When the method is implemented, the atmospheric light estimated value is calculated according to the fog storage image with the brightness reduced value, and the method comprises the following steps:
and obtaining a dark channel diagram according to the fog storage image after the brightness is reduced.
It should be noted that the dark channel map is based on the fact that for an image without a sky, the partial pixel intensities of at least one color channel are very low, close to 0, and then the minimum pixel intensity in each color channel is taken, and the image that it constitutes is the dark channel map. The dark channel map can be described by the following formula:
wherein d Ω (x) Represents the dark primary value of a pixel x in an image, Ω (x) represents a local region centered on the pixel x, I c (y) is a diagram representing a corresponding color channel.
The pixel point with the maximum intensity of 0.1% is found in the dark channel map.
And finding out the pixel point corresponding to 0.1% of the pixel points in the fog storage image after the brightness is reduced.
And taking the average intensity value of the pixel points corresponding to the fog storage image after the brightness is reduced as an atmospheric light estimated value.
It should be noted that the fog-storing image after the brightness is reduced is also divided into three color channels r, g and b, and each color channel corresponds to the component of the atmospheric light estimated value, namely A r 、A g 、A b I.e. atmospheric light estimation of each colour channelThe value components together form an atmospheric light estimated value, specifically, the average value of the intensities of the corresponding pixels on each color channel of the fog storage image after the brightness is reduced is taken as the atmospheric light estimated value component of the corresponding color channel, namely, the atmospheric light estimated value component is the component of the atmospheric light estimated value in the corresponding color channel of the fog storage image after the brightness is reduced, wherein the pixel corresponding to 0.1% of the pixels is found, and the average value of the intensities of the corresponding pixels of the corresponding color channel is taken as the atmospheric light estimated value.
In practice, the value of the small scale is set to 1.
The large scale value k, set according to the resolution of the image, is expressed as follows:
wherein w and h represent the width and height of the image, respectively;representing a rounded function.
It should be noted that the small-scale value is set to 1, namely pixel-level defogging, so that the defogging of the edge area of an object in an image can be finer, and the halation artifact phenomenon can be better avoided; the large scale can be adjusted adaptively according to the image by setting the large scale according to the resolution of the image, so that the large scale selection of pictures with different sizes can be better adapted, and the adaptability is stronger.
In implementation, the large-scale transmission diagram t pa (x) And a small-scale transmission diagram t pi (x) The expression is as follows:
where α represents the distant view fog addition parameter, and is generally 0.95，A c Representing the corresponding color channel value of the atmospheric light estimate in the dark channel, i.e. A r 、A g 、A b Omega (x) denotes a local area centered on the pixel point x, I c And (y) is a color channel diagram corresponding to the original fog storage image.
In implementation, the significance value in the significance map is taken as a weight, and the significance extraction defogging model (Significance Extract Dehazing, SED) is obtained by carrying out weighted fusion on the large-scale transmission map and the small-scale transmission map, and is expressed as follows:
wherein t is sed (x) Represents the saliency extraction defogging transmission diagram at the pixel point x, W sig (x) Andrespectively representing t in the saliency extraction defogging model on the pixel point x pi (x) And t pa (x) Ratio of W sig (x) At t pi (x) Saliency value of saliency map corresponding to the point and +.>
It should be noted that the original fog-storing image is subjected to saliency division, and the region with high saliency is subjected to small-scale defogging, wherein the region with high saliency contains the edge of an object, so that the phenomenon of halation artifact generated on the edge of the object can be effectively avoided; and the defogging mode is extracted based on the saliency of the saliency map fusion, so that the quality of the defogging image can be improved more effectively.
Specifically, the method can extract the salient region of the original fog storage image based on the attention mechanism to obtain the salient map of the original fog storage image, and comprises the following steps:
firstly, an attention mechanism model is established through Bayesian rules, and the formula is as follows:
wherein s is z Representing the significance of pixel z; z represents the specific pixel point to be classified, and classification refers to saliency and non-saliency; c is a binary variable of whether the point is the target classification, 1 represents yes, 0 represents no; l is a random variable at the point; f is the visual characteristic of the point; f (f) z Is the eigenvalue at z; l (L) z Is the location of the point.
Then, assuming that the eigenvalues are independent of position, and given c=1, we get:
p(F＝f z ,L＝l z )＝p(F＝f z )p(L＝L z )，
p(F＝f z ,L＝l z |C＝1)＝p(F＝f z |C＝1)p(L＝l z |C＝1)，
thus, the probability distribution equation is obtained after the attention mechanism model takes logarithms:
log s z ＝-logp(F＝f z )+logp(L＝l z |C＝1)+p(C＝1)p(L＝l z )，
finally, computing the features as linear response of a gaussian differential (DoG) filter yields a saliency map.
It should be noted that, by adopting the attention-based mechanism to obtain the saliency map, the saliency map can be obtained according to the attention degree of people to different objects in the image, so that the defogging image obtained by the saliency map is more consistent with logic based on the attention of human eyes.
During implementation, the saliency extraction defogging model is constrained by utilizing L2 regularization, and a fused transmission diagram is obtained, and the method comprises the following steps:
the defogging model constraints are extracted for significance by L2 regularization as follows:
wherein lambda is t Represents regularization coefficient, R (t sed (x) A smoothing term for the transmission map;
it should be noted that the constraint on weights by L2 regularization is to find the most appropriate value under the weight effect so that the first three terms in the above equation can reach the minimum value, so that the optimal t is found sed (x) So that the edge area and t in the fog storage image pi (x) Closest to other regions and t pa (x) Closest, at the same time, the third term is to avoid invalid output, thereby obtaining the optimal t sed (x) The motion sickness artifact generated by the edge of the object in the image is better restrained, so that the defogging image quality is better.
R (t) sed (x) Neglecting), obtaining:
smoothing the above equation by using a guide filter to obtain smoothed t sed (x)。
When the method is implemented, the defogging image is obtained by inverse solution of the atmospheric scattering model, and is expressed as follows:
wherein J is sed (x) And (3) representing defogging images, and I (x) representing original fog storage images under the mine.
Specifically, the atmospheric scattering model is expressed as:
I(x)＝J sed (x)t sed (x)+A(1-t sed (x))，
the defogging image J can be obtained by inverse solution of the atmospheric scattering model sed (x)：
Example 2
The embodiment 2 of the application discloses a mine-oriented image saliency extraction defogging device, which comprises:
the fog storage image acquisition module is used for acquiring an original fog storage image under the mine;
the atmospheric light estimation module is used for carrying out brightness reduction processing on the original fog storage image based on histogram analysis and calculating an atmospheric light estimation value according to the fog storage image after brightness reduction;
the fusion transmission diagram obtaining module is used for obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, the preset small-scale value and the large-scale value; fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram;
and the defogging image obtaining module is used for obtaining defogging images by reversely solving the atmospheric scattering model based on the fused transmission diagram and the atmospheric light estimated value.
It should be noted that, the relevant points of the present embodiment and embodiment 1 can be referred to each other, and the description is repeated here. The specific calculation method of the atmospheric light estimation module in embodiment 2 may be selected as the calculation method of the atmospheric light estimation value in embodiment 1; in this embodiment 2, the fusion mode of the transmission map obtaining module is selected, and the specific fusion mode of embodiment 1 is selected.
Example 3
The embodiment 3 of the application discloses a face detection method for mines, which is used for processing collected images in the mines according to the image saliency extraction defogging method for the mines in the embodiment 1 to obtain defogging images; and detecting the face in the defogging image by using a plane rotation face detection method.
Specifically, in this embodiment, when a face in the defogging image is detected by using a planar rotation face detection method, full planar rotation (Rotation In Plane, RIP) face detection is adopted, and the method is divided into three layers of networks:
detecting a face candidate object facing downwards for the first layer network and rotating the face candidate object by 180 degrees, namely reducing the RIP range from [ -180 degrees, 180 degrees ] to [ -90 degrees, 90 degrees ]; detecting a face candidate target processed by the first layer network and rotating the face candidate target by 45 degrees, namely reducing the RIP range from [ -90 degrees, 90 degrees ] to [ -45 degrees, 45 degrees ], judging whether the face candidate target is a face or not, and discarding the face candidate target if the face candidate target is not the face; and finally, fine tuning through a third layer of network, judging whether the face is a human face or not, and predicting an accurate RIP angle.
In this embodiment, the steps from thick to thin are adopted, each step detects a face and calibrates the RIP direction to be vertical, and only one face is detected in each step, so that the network is simple, the time cost is low, and as shown in fig. 3, the specific implementation process is as follows:
layer one network:
firstly, obtaining all face candidate targets through a sliding window and an image pyramid principle; then removing candidates of non-face targets by using each P layer, returning the candidates of the face targets and adjusting the RIP direction to be an upright direction; and finally merging the candidate targets with high overlapping degree by using non-maximum suppression (NMS). That is, the first tier network has three goals: face/non-face classification; frame regression; calibration is expressed as follows:
[f,t,g]＝F 1 (x)，
wherein x represents each input window; f (F) 1 Representing a small CNN network detector; f, face confidence score; t frame regression vector; g orientation score.
The first target face/non-face classification uses a softmax penalty function L cls ：
L cls ＝ylogf+(1-y)log(1-f)，
Where y=1 represents a face and y=0 represents a non-face.
Second target frame regression vector loss function L reg ：
L reg (t,t * )＝S(t-t * )，
Wherein t and t represent predicted and actual values, respectively; s denotes the robust smoothing l1 loss.
The bounding box regression objective contains 3 aspects:
t w ＝w * /w，
t a ＝(a * +0.5w * -a-0.5w)/w * ，
t b ＝(b * +0.5w * -b-0.5w)/w * ，
wherein a, b and w respectively represent the abscissa and the ordinate of the frame and the line width;
thus, the layer one network may be defined as:
wherein L is cal Represents the coarse direction loss function of the predicted face, lambda reg And lambda (lambda) cal Representing a balancing factor for balancing the parameters of the different loss functions.
Layer two network:
further distinguishing faces from non-faces in the second network, returning to the bounding box, and calibrating the candidate faces. Unlike the first-tier network, the prediction of this stage of the direction of coarseness is the ternary classification angle range of RIP, i.e., [ -90, -45] or [ -45,45] or [45,90].
In the training stage of the second stage, the initial training image is uniformly rotated in the range of [90,90], and negative samples, namely samples of non-human faces, are filtered through the trained first layer of network. It will be appreciated that samples that are not within range will not contribute to calibration training.
Third layer network:
after passing through the second network, all candidate faces are calibrated to the vertical quarter RIP range, i.e., [ -45,45]. At this time, the third layer network can make a final decision to accurately determine whether it is a face and return to the bounding box. And as RIP has been narrowed to a small extent in the first few stages, the third tier network can directly return to the exact RIP angle of the candidate face rather than the coarse direction.
In the training phase of the third phase, the initial training image is uniformly rotated within the range of [45, 45 degrees ], and negative samples are filtered through the trained second layer network. Calibration branches are a regression task, trained with smooth l1 loss.
Compared with the prior art, the embodiment firstly carries out defogging treatment on the images under the mine to obtain defogged images, and then obtains face candidate targets in all defogged images through a sliding window and image pyramid principle; then removing candidates of non-face targets by using each P layer, returning the candidates of the face targets and adjusting the RIP direction to be an upright direction; and finally merging the candidate targets with high overlapping degree by using non-maximum suppression (NMS). By defogging the image, a clearer image can be obtained, which is more beneficial to face detection; by flipping the image three times, calibration based on coarse RIP prediction can be effectively achieved with little additional time overhead, i.e., rotating the original image 90 °, -90 ° and 180 ° to turn the image left, right and down. With accurate and rapid calibration, face candidates are calibrated step by step to upright, and are easier to detect. The face detection is carried out in steps, the task of each step is very simple, the time cost is reduced, and the face detection is better carried out. The detection steps are only performed from top to bottom, from left to right, the time cost is low, the rough calibration mode is more robust, and the RIP prediction is more accurate.
Those skilled in the art will appreciate that all or part of the flow of the methods of the embodiments described above may be accomplished by way of a computer program to instruct associated hardware, where the program may be stored on a computer readable storage medium. Wherein the computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.
The present application is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present application are intended to be included in the scope of the present application.
Claims (9)
1. The mine-oriented image saliency extraction defogging method is characterized by comprising the following steps of:
acquiring an original fog storage image under a mine;
performing brightness reduction processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimated value according to the fog storage image after brightness reduction;
obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, a preset small-scale value and a large-scale value;
fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model;
constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram;
based on the fused transmission diagram and the atmospheric light estimated value, reversely solving the atmospheric scattering model to obtain a defogging image;
the histogram analysis-based brightness reduction processing is carried out on the original fog storage image, and the method comprises the following steps:
dividing the original fog-storing image into three color channels of red, green and blue;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram;
calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channel c The formula is as follows:
in Max c 、Mea c And Mid c Respectively representing the maximum intensity value, the average intensity value and the median intensity of each color channel, c represents the color channel, and c epsilon { r, g, b }, r, g and b respectively represent three color channels of red, green and blue in the color channel;
and comparing the brightness factor of each color channel with a brightness threshold set by an image, setting the intensity of a pixel point with the intensity larger than the maximum value of 0.9 times of the intensity in the channel as the intensity average value of the channel if the brightness factor is smaller than or equal to the brightness threshold, otherwise, not reducing the intensity.
2. The mine-oriented image saliency extraction defogging method according to claim 1, wherein the atmospheric light estimated value is calculated according to the fog storage image after the brightness is reduced, and the method comprises the following steps:
obtaining a dark channel diagram according to the fog storage image with the brightness reduced;
finding out the pixel point with the maximum intensity of 0.1% in the dark channel diagram;
finding out the pixel point corresponding to the 0.1% pixel point in the fog storage image after the brightness is reduced;
and taking the average intensity value of the pixel points corresponding to the fog storage image after the brightness is reduced as an atmospheric light estimated value.
3. The mine-oriented image saliency extraction defogging method of claim 1, wherein the small scale value is set to 1;
the large-scale value k is set according to the resolution of the image and is expressed as follows:
where w and h represent the width and height of the image, respectively.
4. The mine-oriented image saliency extraction defogging method of claim 3, wherein the large-scale transmission map t pa (x) And a small-scale transmission diagram t pi (x) The expression is as follows:
wherein alpha represents a distant view fog adding parameter, A c Representing the color channel value corresponding to the atmospheric light estimated value in the dark channel, Ω (x) represents a local area centered on the pixel point x, I c And (y) is a color channel diagram corresponding to the original fog storage image.
5. The mine-oriented image saliency extraction defogging method of claim 4, wherein a saliency value in a saliency map is used as a weight, and a large-scale transmission map and a small-scale transmission map are subjected to weighted fusion to obtain the saliency extraction defogging model, which is expressed as follows:
t sed (x)＝W sig (x)·t pi (x)+W sig (x)·t pa (x)，
wherein t is sed (x) Represents the saliency extraction defogging transmission diagram at the pixel point x, W sig (x) And W is sig (x) Respectively representing t in the saliency extraction defogging model on the pixel point x pi (x) And t pa (x) Ratio of W sig (x) At t pi (x) Saliency value of saliency map corresponding to the point and W sig (x)+W sig (x)＝1。
6. The mine-oriented image saliency extraction defogging method of claim 5, wherein the L2 regularization is utilized to constrain the saliency extraction defogging model to obtain a fused transmission map, and the method comprises the following steps:
extracting defogging model constraints on the salience through L2 regularization as follows:
wherein lambda is t Represents regularization coefficient, R (t sed (x) A smoothing term for the transmission map;
r (t) sed (x) Neglecting)The method comprises the following steps of:
smoothing the above equation by using a guide filter to obtain smoothed t sed (x)。
7. The mine-oriented image saliency extraction defogging method of claim 6, wherein said inverse atmospheric scattering model obtains defogging images expressed as:
wherein J is sed (x) And (3) representing defogging images, and I (x) representing original fog storage images under the mine.
8. The utility model provides a fog device is removed in image saliency extraction towards mine which characterized in that includes:
the fog storage image acquisition module is used for acquiring an original fog storage image under the mine;
the atmosphere light estimation module is used for carrying out brightness value reduction processing on the original fog storage image based on histogram analysis and calculating an atmosphere light estimation value according to the fog storage image after brightness value reduction;
the fusion transmission diagram obtaining module is used for obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimated value, the preset small-scale value and the large-scale value; fusing the large-scale transmission map and the small-scale transmission map based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; constraining the saliency extraction defogging model by using L2 regularization to obtain a fused transmission diagram;
the defogging image obtaining module is used for obtaining defogging images by reversely solving the atmospheric scattering model based on the fused transmission diagram and the atmospheric light estimated value;
the histogram analysis-based brightness reduction processing is carried out on the original fog storage image, and the method comprises the following steps:
dividing the original fog-storing image into three color channels of red, green and blue;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram;
calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channel c The formula is as follows:
in Max c 、Mea c And Mid c Respectively representing the maximum intensity value, the average intensity value and the median intensity of each color channel, c represents the color channel, and c epsilon { r, g, b }, r, g and b respectively represent three color channels of red, green and blue in the color channel;
and comparing the brightness factor of each color channel with a brightness threshold set by an image, setting the intensity of a pixel point with the intensity larger than the maximum value of 0.9 times of the intensity in the channel as the intensity average value of the channel if the brightness factor is smaller than or equal to the brightness threshold, otherwise, not reducing the intensity.
9. A face detection method for mines, characterized in that the image saliency extraction defogging method according to any one of claims 1-7 processes the collected images in the mines to obtain defogged images; and detecting the face in the defogging image by using a plane rotation face detection method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110330453.0A CN112907582B (en) | 2021-03-24 | 2021-03-24 | Mine-oriented image saliency extraction defogging method and device and face detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110330453.0A CN112907582B (en) | 2021-03-24 | 2021-03-24 | Mine-oriented image saliency extraction defogging method and device and face detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112907582A CN112907582A (en) | 2021-06-04 |
| CN112907582B true CN112907582B (en) | 2023-09-29 |
Family
ID=76109122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110330453.0A Active CN112907582B (en) | 2021-03-24 | 2021-03-24 | Mine-oriented image saliency extraction defogging method and device and face detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112907582B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060056304A (en) * | 2006-05-04 | 2006-05-24 | 한양대학교 산학협력단 | Apparatus and method for adaptive image enhancement |
| CN105761227A (en) * | 2016-03-04 | 2016-07-13 | 天津大学 | Underwater image enhancement method based on dark channel prior algorithm and white balance |
| CN106530246A (en) * | 2016-10-28 | 2017-03-22 | 大连理工大学 | Image dehazing method and system based on dark channel and non-local prior |
| CN107301625A (en) * | 2017-06-05 | 2017-10-27 | 天津大学 | Image defogging algorithm based on brightness UNE |
| CN108596853A (en) * | 2018-04-28 | 2018-09-28 | 上海海洋大学 | Underwater picture Enhancement Method based on bias light statistical model and transmission map optimization |
| CN109961412A (en) * | 2019-03-18 | 2019-07-02 | 浙江大华技术股份有限公司 | A kind of video frame images defogging method and equipment |
| CN110046670A (en) * | 2019-04-24 | 2019-07-23 | 北京京东尚科信息技术有限公司 | Feature vector dimension reduction method and device |
| CN110175972A (en) * | 2019-05-29 | 2019-08-27 | 南京信息职业技术学院 | Infrared image enhancement method based on transmission map fusion |
| CN111785224A (en) * | 2019-04-04 | 2020-10-16 | 海信视像科技股份有限公司 | Brightness driving method |
| CN111932469A (en) * | 2020-07-21 | 2020-11-13 | 泉州职业技术大学 | Significance weight quick exposure image fusion method, device, equipment and medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6566663B2 (en) * | 2015-03-10 | 2019-08-28 | キヤノン株式会社 | Image display apparatus and control method thereof |
| CN107356515B (en) * | 2017-07-20 | 2020-04-03 | 上海睿钰生物科技有限公司 | Method and system for determining fluorescence intensity of fluorescence image |
-
2021
- 2021-03-24 CN CN202110330453.0A patent/CN112907582B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060056304A (en) * | 2006-05-04 | 2006-05-24 | 한양대학교 산학협력단 | Apparatus and method for adaptive image enhancement |
| CN105761227A (en) * | 2016-03-04 | 2016-07-13 | 天津大学 | Underwater image enhancement method based on dark channel prior algorithm and white balance |
| CN106530246A (en) * | 2016-10-28 | 2017-03-22 | 大连理工大学 | Image dehazing method and system based on dark channel and non-local prior |
| CN107301625A (en) * | 2017-06-05 | 2017-10-27 | 天津大学 | Image defogging algorithm based on brightness UNE |
| CN108596853A (en) * | 2018-04-28 | 2018-09-28 | 上海海洋大学 | Underwater picture Enhancement Method based on bias light statistical model and transmission map optimization |
| CN109961412A (en) * | 2019-03-18 | 2019-07-02 | 浙江大华技术股份有限公司 | A kind of video frame images defogging method and equipment |
| CN111785224A (en) * | 2019-04-04 | 2020-10-16 | 海信视像科技股份有限公司 | Brightness driving method |
| CN110046670A (en) * | 2019-04-24 | 2019-07-23 | 北京京东尚科信息技术有限公司 | Feature vector dimension reduction method and device |
| CN110175972A (en) * | 2019-05-29 | 2019-08-27 | 南京信息职业技术学院 | Infrared image enhancement method based on transmission map fusion |
| CN111932469A (en) * | 2020-07-21 | 2020-11-13 | 泉州职业技术大学 | Significance weight quick exposure image fusion method, device, equipment and medium |
Non-Patent Citations (8)
| Title |
|---|
| A comparative study of histogram equalization based image enhancement techniques for brightness preservation and contrast enhancement;Omprakash Patel等;《Signal & Image Processing : An International Journal (SIPIJ)》;第4卷(第5期);11-25 * |
| Gray Scale Image Restoration Using Advanced Atmospheric Light Estimation;Sasmita Mahakud等;《2019 Fifth International Conference on Image Information Processing (ICIIP)》;80-85 * |
| Jop-alarm: Detecting jump-oriented programming-based anomalies in applications;Fan Yao等;《2013 IEEE 31st International Conference on Computer Design (ICCD)》;467-470 * |
| Multi-scale Optimal Fusion model for single image dehazing;Dong Zhao等;《Signal Processing: Image Communication》;第74卷;253–265 * |
| Single Image Haze Removal with Improved Atmospheric Light Estimation;Yincui Xu等;《Journal of Physics: Conference Series》;第1098卷(第1期);1-6 * |
| 图像及视频去雾算法研究及实现;郭聪;《中国优秀硕士学位论文全文数据库：信息科技辑》(第2018年第08期);I138-610 * |
| 基于天空识别的单幅图像去雾方法;鲍唤唤;《中国优秀硕士学位论文全文数据库：信息科技辑》(第2020年第02期);I138-1817 * |
| 基于暗通道和多正则化约束的图像去雾方法;舒巧玲等;《计算机工程与设计》;第40卷(第7期);1950-1955+2018 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112907582A (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110543837B (en) | Visible light airport airplane detection method based on potential target point | |
| CN108596849B (en) | Single image defogging method based on sky region segmentation | |
| CN111079556A (en) | Multi-temporal unmanned aerial vehicle video image change area detection and classification method | |
| CN105404884B (en) | Image analysis method | |
| US8553086B2 (en) | Spatio-activity based mode matching | |
| CN107167810B (en) | Method for rapidly extracting underwater target by side-scan sonar imaging | |
| CN109255326B (en) | Traffic scene smoke intelligent detection method based on multi-dimensional information feature fusion | |
| CN113989613A (en) | Light-weight high-precision ship target detection method coping with complex environment | |
| GB2426137A (en) | Tracking object contours in a sequence of images | |
| CN109242032B (en) | Target detection method based on deep learning | |
| CN111489330B (en) | Weak and small target detection method based on multi-source information fusion | |
| CN103186887A (en) | Image demisting device and image demisting method | |
| CN111062293A (en) | Unmanned aerial vehicle forest flame identification method based on deep learning | |
| CN110060221B (en) | Bridge vehicle detection method based on unmanned aerial vehicle aerial image | |
| Niu et al. | A moving objects detection algorithm based on improved background subtraction | |
| CN111985314B (en) | Smoke detection method based on ViBe and improved LBP | |
| CN112200746A (en) | Defogging method and device for traffic scene image in foggy day | |
| CN111681262A (en) | Method for detecting infrared dim target under complex background based on neighborhood gradient | |
| Pal et al. | Visibility enhancement techniques for fog degraded images: a comparative analysis with performance evaluation | |
| CN112907582B (en) | Mine-oriented image saliency extraction defogging method and device and face detection | |
| CN110188693B (en) | Improved complex environment vehicle feature extraction and parking discrimination method | |
| CN108985216B (en) | Pedestrian head detection method based on multivariate logistic regression feature fusion | |
| CN116740572A (en) | Marine vessel target detection method and system based on improved YOLOX | |
| CN112949389A (en) | Haze image target detection method based on improved target detection network | |
| CN113920168A (en) | Image tracking method in audio and video control equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Kou Qiqi Inventor after: Cheng Deqiang Inventor after: You Yangyang Inventor after: Wang Zhenyu Inventor before: Cheng Deqiang Inventor before: You Yangyang Inventor before: Wang Zhenyu Inventor before: Kou Qiqi |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |