CN107798665B - Underwater image enhancement method based on structure-texture layering - Google Patents
Underwater image enhancement method based on structure-texture layering Download PDFInfo
- Publication number
- CN107798665B CN107798665B CN201711086095.3A CN201711086095A CN107798665B CN 107798665 B CN107798665 B CN 107798665B CN 201711086095 A CN201711086095 A CN 201711086095A CN 107798665 B CN107798665 B CN 107798665B
- Authority
- CN
- China
- Prior art keywords
- image
- texture
- layer
- transmittance
- enhancement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000002834 transmittance Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 26
- 238000012937 correction Methods 0.000 claims abstract description 13
- 230000002708 enhancing effect Effects 0.000 claims abstract description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 11
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 10
- 238000005457 optimization Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 6
- 239000004576 sand Substances 0.000 claims description 5
- 238000002474 experimental method Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 238000000149 argon plasma sintering Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000002787 reinforcement Effects 0.000 claims description 3
- 230000008602 contraction Effects 0.000 claims description 2
- 238000007670 refining Methods 0.000 claims description 2
- 235000019646 color tone Nutrition 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 15
- 238000010521 absorption reaction Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 230000003321 amplification Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000003199 nucleic acid amplification method Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 239000002245 particle Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007903 penetration ability Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the field of computer vision, and aims to improve the contrast of an image and keep good details, inhibit noise and keep the naturalness of color tones. The invention adopts the technical scheme that an underwater image enhancement method based on structure-texture layering is characterized in that firstly, color correction is carried out through histogram equalization, and then a color corrected image is decomposed into a low-frequency structure layer and a high-frequency texture layer. The method comprises the steps of enabling noise to be remained in a texture layer, accurately estimating the transmittance from a structure layer without noise based on a proposed fog line model, further performing enhancement processing, enhancing the texture layer by using a gradient residue minimization method, and reconstructing the enhanced structure layer, the texture layer and a refined edge mask into a final enhancement image through proper scale expansion, so that the problem which cannot be processed in the prior art is solved. The invention is mainly applied to underwater image enhancement and processing occasions.
Description
Technical Field
The invention belongs to the field of computer vision, and relates to an underwater image enhancement method based on structure-texture layering. Specifically, the image is divided into a high-frequency texture layer and a low-frequency structural layer through layering processing, so that the problem of noise amplification in the enhancement process is avoided, details and edges are well reserved, and the contrast and the tone naturalness are improved, namely the underwater image enhancement method based on structure-texture layering.
Background
Clear and high-visibility underwater image analysis and identification have very important significance for research in underwater exploration, underwater video, military affairs and the like. However, in complex underwater imaging environments, image quality is severely degraded due to the effects of absorption and scattering of large suspended particles (like turbid particles in water) during transmission. Therefore, the typical underwater image has the following problems:
1) converging light to illuminate to form uneven background gray distribution;
2) the absorption and scattering effects cause the problems of non-uniform brightness and detail blurring of an underwater image;
3) poor lighting conditions cause problems of hue attenuation and contrast reduction;
due to the fact that the quality of the shot underwater images is generally poor, preprocessing is needed before target recognition is carried out on the underwater images. The existing underwater image enhancement method mainly has the following development stages:
the first stage is as follows: processing underwater images by using a traditional image enhancement method, such as: gray level change, histogram equalization, image spatial domain smoothing and sharpening, pseudo-color processing and the like. However, these conventional methods do not provide ideal processing results due to the uniqueness of the underwater imaging characteristics.
And a second stage: according to the similarity of imaging characteristics in underwater and haze, researchers begin to research underwater image enhancement methods based on a defogging technology, but due to the difference between transmissivity and absorption scattering characteristics in water and air, the direct application of the defogging method to underwater image enhancement has problems, such as loss of details in dark regions and overexposure in bright regions.
And a third stage: with the study of underwater refraction and scattering characteristics, researchers have proposed some specialized underwater mathematical imaging models and some methods for underwater image enhancement. The main starting points of the methods are background light estimation, transmissivity estimation, compensation of light wave attenuation, enhancement based on a fusion principle and the like. For example: the background light is estimated with a layer-based approach as mentioned in the reference; the method comprises the following steps of (1) estimating global background light based on quadtree decomposition and image blocking, and providing a transmittance estimation algorithm aiming at the minimum information loss of underwater imaging characteristics; a system method for enhancing underwater images by an artificial light compensation method; the visibility of the underwater degraded image is increased by fusing the four weights. These methods suppress overexposure and noise problems to some extent and improve the hue of the image, but still have the problems of partial loss of detail, noise amplification, unnatural hue, oversaturation, and the like. These problems can have a great impact on underwater research, and therefore, it is necessary to design a better underwater image enhancement method, which can improve the contrast of the image as much as possible, highlight interesting details and features, improve the visual effect of the image, and provide a clear image suitable for analysis.
Disclosure of Invention
The invention aims to make up for the defects of the prior art, namely, the purpose of keeping good details while improving the image contrast is realized, and the naturalness of the tone is kept while the noise is suppressed. The invention adopts the technical scheme that an underwater image enhancement method based on structure-texture layering is characterized in that firstly, color correction is carried out through histogram equalization, and then a color corrected image is decomposed into a low-frequency structure layer and a high-frequency texture layer. The method comprises the steps of enabling noise to be remained in a texture layer, accurately estimating the transmittance from a structure layer without noise based on a proposed fog line model, further performing enhancement processing, enhancing the texture layer by using a gradient residue minimization method, and reconstructing the enhanced structure layer, the texture layer and a refined edge mask into a final enhancement image through proper scale expansion, so that the problem which cannot be processed in the prior art is solved.
The method comprises the following specific steps:
1) color correction of the captured image: firstly, combining the histogram characteristics of the color distribution of the underwater image, moving the average color distribution of each color channel to a desired range, and then performing linear normalization processing, namely, solving the color shift problem by simple histogram equalization, specifically, for the pixel value of each color channel of each image
The lower limit of the boundary is calculated as follows
And upper limit of
Where c is ∈ { R, G, B }, μcAnd σcThe distribution range is effectively adjusted for the mean and standard deviation of each color channel, λ being a hue parameter, respectively, and the pixel value range of the image is then truncated to 0, 255 using a truncation function chip ()]And then obtaining a color corrected image:
Ic(x) Is a colorPixel values for each channel of the corrected image;
2) constructing a layered model: defining an underwater image with a defogging model:
Ic(x)＝Jc(x)t(x)+Ac(1-t(x))+E(x), (4)
where x is the pixel value, Jc(x) Is a restored image, AcIs global background light, t (x) is throw ratio in water, definition E (x) is noise in input image, and image enhancement is to obtain underwater image I from capturec(x) Resume Jc(x) This process involves the estimation of transmittance and background light;
3) structure-texture layering implementation: using TV-L1Separating the structural layer and the texture layer:
wherein the content of the first and second substances,
is to normalize the gradient of the structure layer by a 1 norm, xi is a weight parameter to control the smoothness degree, through TV-L1Structural layer IsSolved out, TV-L1The method is an optimization method for image decomposition by using total variation as a regular term, and a texture layer passes through It＝I-IsCalculating;
4) and (3) structural layer reinforcement: and (3) estimating A by adopting a global background light scattering estimation method:
wherein the content of the first and second substances,
is to reach the maximum position in the dark channel of the structural layer, i.e.
Ω (y) is a field of y;
according to dark channel prior theory, it is first assumed that the transmittance in each window is constant, defined as t0The value a has been calculated by equation (9), so that two minimum operations are performed on both sides of equation (6), and after the term shift, an estimated value of the transmittance t (x) is obtained:
where ω is a correction factor;
5) denoising the texture layer: through the above TV-L1And decomposing, and capturing the noise in the graph to remain in the texture layer of high frequency. Enhancing the texture layer by using a gradient residual minimization method, wherein the enhancement of the texture layer is obtained by optimizing a formula (15):
wherein δ and η are control weight parameters, which control the smoothness and fineness of the processing result, and too large or too small η can cause most of the texture of the enhanced image to be lost along with the denoising processing, so that properly selecting δ and η has a great influence on the experimental result, and making Z equal to J-I, the above optimization problem becomes two sub-problems, as shown in the following formula:
this can turn equation (16) into the classical TV problem, equation (17) is approximated with a soft threshold;
6) refining a mask: using a binary mask M to apply TV-L1The decomposed texture layer is separated into a smooth area and a detail area, and then is deleted inResidual details in the enhanced low-frequency structural layer smooth area are specifically detected by using discrete cosine transform coefficients to detect the similarity between blocks in a scene, whether an area is smooth or not is judged, and I is definedtB, then the similarity of each block in the scene detail is represented by equation (18):
where x, y are coordinate locations in the DCT, calculated except for B1,1,B1,2And B2,1Then using threshold value to judge similarity of every block, defining initial edge mask of texture layer as M, setting threshold value kappa as 0.1, when rho is greater than kappa, M of said block is 1, otherwise, M is 0, so as to obtain a rough binary edge information image
Combining structural layer I by soft mapping methodsThe resulting mapped labrador matrix corrects this initial coarse M:
where M and M 'are vector representations of M and M', LsIs from IsThe resulting mapped labrador matrix is then used,
is a normalization parameter, set to 10 in this document5；
7) And (3) reconstruction: finally, we reconstruct the final enhancement processing result by equation (20):
wherein, tauIs a scaling factor, introduces enhanced details, set to in experiments
And
respectively a low-frequency structural layer and a high-frequency texture layer which are subjected to enhancement treatment.
In a weak light environment, a high-frequency part of an image not only contains texture information, but also contains a large amount of noise, in order to avoid influencing the accuracy of transmissivity estimation, the image is firstly subjected to layering processing, then the transmissivity is estimated in a low-frequency structural layer, denoising processing is carried out in a high-frequency texture layer, and a graph is defined as J-Js+Jt，JsAnd JtStructural and texture layers of the image, respectively, then equation (4) is rewritten as:
Ic(x)＝(Js c(x)+Jt c(x))t(x)+Ac(1-t(x))+E(x)
＝Js c(x)t(x)+Ac(1-t(x))+Jt c(x)t(x)+E(x). (5)
wherein the background light AcIs a relatively smooth signal, and the estimated transmittance t is also close to Js cAnd thus, the captured image can be correspondingly written as: i ═ Is+ItObtaining:
Is c(x)＝Js c(x)t(x)+Ac(1-t(x)) (6)
It c(x)＝Jt c(x)t(x)+E(x) (7)
wherein, Is cIs a low-frequency structure layer of the captured image, containing most of the structure outline of the image, It cIs the high frequency texture layer of the captured image, containing most of the texture information and noise, by which the transmittance t (x) is derived from Is(x) And the influence of noise is avoided.
The estimated value of the transmittance t (x) is more refined: converting the formula (6) into a 3D coordinate system to obtain a linear transmittance estimated value t' (x) of transmittance in a three-dimensional coordinate system, and changing the formula (6) into the following form by taking the background light A as a coordinate origin:
Is0(x)＝t'(x)·Js0(x) (11)
wherein, Is0(x)＝Is(x)-A，Js0(x)＝Js(x) -a, thus calculating a linear transmittance estimate for each pixel value according to the following formula:
given a limit value of the transmission
Then, by optimizing equation (13), a fine transmittance map is obtained:
where α and β are weight parameters of the control data term and the smoothing term, and σ (x) is
The standard deviation at each pixel value, N (x) denotes the four neighborhood pixel positions around each pixel value in the structural layer, A and
after the value of (c) is calculated, then this enhanced structural layer can be derived from equation (14):
the invention has the technical characteristics and effects that:
aiming at the problems of noise amplification, tone offset and the like of underwater image enhancement, the method of the invention respectively enhances and denoises by dividing the image into a structural layer and a texture layer, thereby realizing the purpose of improving the image contrast and simultaneously keeping good details, and simultaneously inhibiting the noise and simultaneously keeping the naturalness of the tone. The invention has the following characteristics:
1. the main innovation point of the algorithm is based on the idea of structure-texture layering, and the enhancement and denoising processing is respectively carried out on two layers, so that the noise amplification influence is avoided.
2. This document presents a simple, yet universally applicable solution to the color shift problem.
3. The method has good results for various underwater image enhancements and has certain universality.
4. The algorithm provided by the invention can be applied to the enhancement of underwater videos and has certain expansibility.
Drawings
FIG. 1 is a framework diagram of an algorithm;
FIG. 2 is a captured underwater image on the left and a corresponding histogram on the right;
FIG. 3 shows the left image after color correction and the right image corresponding to the histogram;
FIG. 4 is a decomposition model verification diagram: FIG. 4 (a) is a composite degraded image I, (b) is a structural layer of I, (c) is a texture layer of I, (d) is a sharp image J, (e) is a structural layer of J, (f) is a texture layer of J);
FIG. 5 is a decomposition result diagram;
FIG. 6 is a graph of the reinforcement results of a structural layer;
FIG. 7 is a graph of enhancement results for a texture layer;
FIG. 8 is a redefined graph of an edge mask;
FIG. 9 is a graph of the results of the final reconstruction;
FIG. 10 is a graph comparing the effects of this document and other references (the first column is the input graph; the second, third and fourth columns are the results of three examples of prior art; the fifth column is the results of this document);
FIG. 11 is a graph of the enhancement effect of the hierarchy and non-hierarchy (the first row is a graph of the results of the non-hierarchy; the second row is a graph of the results of the corresponding hierarchy);
Detailed Description
The invention provides an underwater image enhancement method based on structure-texture layering, which is used for enabling noise to be remained in a texture layer, and a structural layer without noise influence meets a fog line model. We then enhance the texture layer with a gradient residual minimization method by enhancing the structural layer based on the fog line model. And then, reconstructing the enhanced structural layer, the texture layer and the refined mask into a final enhanced image through proper scale expansion and contraction. That is, the image contrast is improved while retaining good details, and the naturalness of the color tone is maintained while suppressing noise. The present invention will be described in detail below with reference to the accompanying drawings and examples.
First, in order to overcome some of the disadvantages of the existing underwater image enhancement method, we propose a hierarchical-based image processing method, whose outline diagram is shown in fig. 1. First, color correction is performed by histogram equalization. Second, with TV-L1(Total variational (TV energy) as regularization term for image decomposition (called ROF or TV-L)1Model) optimization method) to perform layered processing on the image, and then perform enhancement and denoising processing on the structural layer and the texture layer respectively. And finally, reconstructing the enhanced structural layer, the enhanced texture layer and the refined mask into a final result graph. The following are specific implementation methods:
1) and carrying out color correction on the shot image. By observing the color distribution histogram characteristics of an underwater image as shown in figure 2,
8) we see that since different light segments are subject to different intensities of absorption and scattering in water, only the blue-green light in the 480-570 wavelength band has the smallest attenuation coefficient and the strongest penetration ability in water, so that the captured image in water mostly presents light green color tone. To solve this color shift problem, we use simple histogram equalization. This principle is straightforward, i.e. moving the average color distribution of each color channel to the desired rangeAnd then linear normalization processing is carried out. For each color channel
(
Pixel values representing each color channel of the image), the lower limit of the boundary is calculated as follows
(
Minimum value of pixel values representing each color channel of an image) and an upper limit
(
Maximum value of pixel values representing each color channel of the image):
where c is ∈ { R, G, B }, μcAnd σcThe mean and standard deviation of each color channel, respectively. λ is a hue parameter that effectively adjusts the distribution range beyond which pixel values will be truncated. Finally, this color corrected image is obtained by minimizing equation (3).
Ic(x)＝Jc(x)t(x)+Ac(1-t(x))+E(x), (4)
Wherein clip (·) is a clip function that controls the truncation of a range of pixel values to [0, 255 ]. The comparison between fig. 2 and fig. 3 shows that the method can effectively correct colors of underwater images.
2) And constructing a hierarchical model. Since the attenuation of the optical path in turbid water and in an atmosphere with haze is similar, the underwater map model is defined by the classical dark channel defogging model of zemmine:
Ic(x)＝Jc(x)t(x)+Ac(1-t(x))+E(x), (4)
where x is the pixel value, Ic(x) Is a color corrected image for each channel, Jc(x) Is the restored image. A. thecIs global background light, and t (x) is the throw in water. Define e (x) as the noise in the input image. The purpose of the image enhancement is to obtain an underwater image I from the capturec(x) Resume Jc(x) This process involves the estimation of transmittance and background light
Generally, an image contains not only important texture information but also a large amount of noise in a high frequency portion, especially in a low light environment. These factors can reduce the accuracy of the transmittance estimation and thus affect the final enhancement. In order to overcome the problem, the image is layered firstly, then the transmissivity is estimated on the structural layer respectively to avoid the influence of noise, and the denoising processing is carried out on the texture layer independently. We define JsAnd JtStructural and texture layers, respectively, of a clean image, then J ═ Js+JtThen equation (4) can be rewritten as:
Ic(x)＝(Js c(x)+Jt c(x))t(x)+Ac(1-t(x))+E(x)
＝Js c(x)t(x)+Ac(1-t(x))+Jt c(x)t(x)+E(x). (5)
wherein the background light AcIs a smooth signal and the estimated transmission t is also relatively smooth, approaching Js cThe ideal transmittance of (1). Thus, the observed captured image may be divided into two layers (I ═ I)s+It) Ideally, one can obtain:
Is c(x)＝Js c(x)t(x)+Ac(1-t(x)) (6)
wherein, Is cIs the structural layer that captures the image, containing most of the details of the image. I ist cIs the texture layer of the captured image, which contains most of the texture information and noise. By this means, the transmission t (x) can be varied from Is(x) And (4) estimating to avoid the influence of noise.
3) Layering is achieved. The structural layer has larger gradient at the contour and boundary of the object, but has smaller gradient at the texture layer. We use TV-L1The decomposition model separates the structural and texture layers well:
wherein the content of the first and second substances,
is to normalize the gradient of the structure layer by a 1 norm, xi is a weight parameter to control the smoothness degree, through TV-L1(Total variational (TV energy) as regularization term for image decomposition (called ROF or TV-L)1Model) optimization method) can be applied to the structural layer IsSolving out that the texture layer passes through It＝I-IsAnd (6) calculating.
An important problem is the above TV-L1Whether the decomposition model happens to be able to decompose the captured map according to equations (6) and (7). FIG. 4 shows an example of decomposition, the input image being I, J synthesized from J according to equation (4)sAnd JtIs by TV-L1Optimizing the decomposed two-layer image, as shown, the structural layer (I) of the degraded image I (x)s) Not including high frequency texture details and noise, is visually consistent with the structural layers of the original sharp image. For better visual effect we magnify the texture layer by a factor of 10.
4) And estimating background light. Considering the absorption and scattering effects of light waves in water, we estimate a using a global backlight light scattering estimation method:
wherein the content of the first and second substances,
is to reach the maximum position in the dark channel of the structural layer, i.e.
Ω (y) is a field of y, and its size is set to 17.
According to dark channel prior theory, it is first assumed that the transmittance in each window is constant, defined as t0The value a has been calculated by equation (9), so we find the minimum value operation twice on both sides of equation (6), and after the term shift, we can obtain the estimated value of the transmittance t (x):
where ω is a correction factor, set herein to 0.95, in order to bring the recovery result graph closer to a realistic picture. However, there is a significant incongruity in the result graph recovered by using the estimated value of the transmittance t (x), so that the estimated value needs to be refined to obtain a better enhancement effect. We observed a distribution I without noisesThe pixel distribution in (1) conforms to the rule of 'fog lines'. Therefore, we can convert equation (6) into a 3D coordinate system to obtain a transmittance t' (x) in a three-dimensional coordinate system (hereinafter referred to as a linear transmittance estimated value). Taking the background light a as the origin of coordinates, equation (6) can be changed to the following form:
Is0(x)＝t'(x)·Js0(x) (11)
wherein, Is0(x)＝Is(x)-A，Js0(x)＝Js(x) -A. Therefore, we can calculate a linear transmittance estimate for each pixel value according to the following formula:
given a limit value of the transmission
Then, we can get a fine transmittance map by optimizing equation (12):
where α and β are weight parameters of the control data term and the smoothing term, and σ (x) is
The standard deviation at each pixel value, n (x), represents the four domain pixel position for each pixel value in the structural layer. Once A and
is calculated, then this enhanced structural layer can be derived from equation (14):
5) the texture layer is enhanced. Through the above TV-L1After decomposition, the noise in the captured image remains in the texture layer. We enhance the texture layer by the gradient residual minimization method, i.e. the enhanced texture layer can be obtained by optimizing equation (15):
wherein, δ and η are control weight parameters, which control the smoothness and fineness of the processing result, and too large or too small η can cause most of the texture of the enhanced image to be lost along with the denoising process, so that the value of δ and η is properly selected to greatly influence the experimental result. Let Z be J-I, the above optimization problem can become two subproblems, as shown in the following equation.
Equation (16) becomes a classical TV problem, which can be referred to [15 ]]TV-L in1And (6) solving the method. Equation (17) may be approximated with a soft threshold.
6) And defining an edge mask. From FIG. 5, it can be seen that the enhancement is
May also include a portion of unwanted noise, especially in slippery areas. Therefore, we use an edge mask M to mask TV-L1And separating the decomposed texture layer into smooth and detailed areas, and further deleting the residual details of the smooth areas in the reinforced structural layer. We use the discrete cosine transform coefficient to detect the similarity between blocks in the scene, judge whether it is smooth, define ItB, then the similarity of each block in the scene detail can be represented by equation (18):
where x, y are coordinate locations in the DCT, we compute except for B1,1,B1,2And B2,1And then using a thresholdThe similarity of each block is judged. We define the initial detail mask of the texture layer as M, the threshold k is 0.1, when p is greater than k, M of the block is 1, otherwise M is 0. To obtain a fine detail mask
Our soft mapping method combines structural layer IsThe resulting mapped labrador matrix corrects this initial coarse M.
Where M and M 'are vector representations of M and M', LsIs from IsThe resulting mapped labrador matrix is then used,
is a normalization parameter, set to 10 in this document5。
7) And (5) reconstructing the image. The structural layer and the texture layer are reinforced respectively to obtain the edge mask M, so that the final enhancement processing result can be reconstructed by the formula (20):
where τ is the scaling factor, enhanced details are introduced, set to in the experiment
And
respectively a structural layer and a texture layer to be reinforced.
The color correction effect, the layering effect, the denoising effect and the enhancement effect of the algorithm are tested in the experiment and compared with the algorithm of the reference, and the specific results are shown in fig. 2, fig. 3, fig. 4, fig. 5, fig. 6, fig. 7, fig. 9, fig. 10 and fig. 11. It can be seen in fig. 10 that the restoration effect of the prior art example 1 amplifies noise and the hue of the enhanced image is unnatural. The recovery results of prior art example 2 are low in contrast and also noise is significant. The processing result of the prior art example 3 improves the contrast of the image to a certain extent and reduces noise, however, because the method is not directed to underwater image enhancement, the color deviation exists in the recovery result, and some details do not achieve good enhancement effect. Fig. 11 shows a comparison of the layered and non-layered effects, and it can be seen that a part of the details of the enhancement result are lost, like the bubble in the first column of the figure, which further illustrates that the underwater image enhancement method based on the structure-texture layering has better effect. In other words, the method mentioned herein achieves the best enhancement, does not reduce the noise very effectively, but retains good detail and natural hue.
Claims (3)
1. An underwater image enhancement method based on structure-texture layering is characterized in that firstly, color correction is carried out through histogram equalization, then a graph after the color correction is decomposed into a low-frequency structural layer and a high-frequency texture layer, noise is remained in the texture layer, then the transmissivity is accurately estimated from the structural layer without the noise based on a proposed fog line model, then enhancement processing is carried out, the texture layer is enhanced through a gradient residue minimization method, and then the enhanced structural layer, the texture layer and a refined edge mask are reconstructed into a final enhancement graph through proper scale expansion and contraction, so that the problem which cannot be processed in the prior art is solved; the concrete steps are detailed as follows:
1) color correction of the captured image: firstly, combining the histogram characteristics of the color distribution of the underwater image, moving the average color distribution of each color channel to a desired range, and then performing linear normalization processing, namely, solving the color shift problem by simple histogram equalization, specifically, for each color of each imagePixel value of channel
The lower limit of the boundary is calculated as follows
And upper limit of
Where c is ∈ { R, G, B }, μcAnd σcThe distribution range is effectively adjusted for the mean and standard deviation of each color channel, λ being a hue parameter, respectively, and the pixel value range of the image is then truncated to 0, 255 using a truncation function chip ()]And then obtaining a color corrected image:
Ic(x) Is the pixel value of each channel of the color corrected image;
2) constructing a layered model: defining an underwater image with a defogging model:
Ic(x)＝Jc(x)t(x)+Ac(1-t(x))+E(x)， (4)
where x is the pixel value, Jc(x) Is a restored image, AcIs global background light, t (x) is throw ratio in water, definition E (x) is noise in input image, and image enhancement is to obtain underwater image I from capturec(x) Resume Jc(x) This is thatThe process involves the estimation of transmittance and background light;
3) structure-texture layering implementation: using TV-L1Separating the structural layer and the texture layer:
wherein the content of the first and second substances,
is to normalize the gradient of the structure layer by a 1 norm, xi is a weight parameter to control the smoothness degree, through TV-L1Structural layer IsSolved out, TV-L1The method is an optimization method for image decomposition by using total variation as a regular term, and a texture layer passes through It＝I-IsCalculating;
4) and (3) structural layer reinforcement: and (3) estimating A by adopting a global background light scattering estimation method:
wherein the content of the first and second substances,
is to reach the maximum position in the dark channel of the structural layer, i.e.
Ω (y) is a field of y;
according to dark channel prior theory, it is first assumed that the transmittance in each window is constant, defined as t0The value a has been calculated by equation (9), so that two minimum operations are performed on both sides of equation (6), and after the term shift, an estimated value of the transmittance t (x) is obtained:
where ω is a correction factor;
5) denoising the texture layer: through the above TV-L1Decomposing, capturing the texture layer with noise residual in the graph at high frequency, and enhancing the texture layer by using a gradient residual minimization method, wherein the enhancement of the texture layer is obtained by optimizing an equation (15):
wherein δ and η are control weight parameters, which control the smoothness and fineness of the processing result, and too large or too small η can cause most of the texture of the enhanced image to be lost along with the denoising processing, so that properly selecting δ and η has a great influence on the experimental result, and making Z equal to J-I, the above optimization problem becomes two sub-problems, as shown in the following formula:
this can turn equation (16) into the classical TV problem, equation (17) is approximated with a soft threshold;
6) refining a mask: using a binary mask M to apply TV-L1Separating the decomposed texture layer into a smooth area and a detail area, further deleting the residual details in the enhanced smooth area of the low-frequency structural layer, specifically detecting the similarity between blocks in the scene by using a discrete cosine transform coefficient, judging whether one area is smooth or not, and defining ItB, then the similarity of each block in the scene detail is represented by equation (18):
where x, y are coordinate locations in the DCT, calculated except for B1，1，B1，2And B2，1Then using threshold value to judge similarity of every block, defining initial edge mask of texture layer as M, setting threshold value kappa as 0.1, when rho is greater than kappa, M of said block is 1, otherwise, M is 0, so as to obtain a rough binary edge information image
Combining structural layer I by soft mapping methodsThe resulting mapped labrador matrix corrects this initial coarse M:
where M and M 'are vector representations of M and M', LsIs from IsThe resulting mapped labrador matrix is then used,
is a normalization parameter, set to 10 in this document5；
7) And (3) reconstruction: finally, we reconstruct the final enhancement processing result by equation (20):
where τ is the scaling factor, enhanced details are introduced, set to in the experiment
And
respectively a low-frequency structural layer and a high-frequency texture layer which are subjected to enhancement treatment.
2. The underwater image enhancement method based on structure-texture layering as claimed in claim 1, wherein the high frequency part of the image in the low light environment contains not only texture information but also a great deal of noise, in order to avoid affecting the accuracy of the transmittance estimation, the image is layered first, then the transmittance is estimated in the low frequency structure layer, the denoising is performed in the high frequency texture layer, and a graph is defined as J ═ Js+Jt，JsAnd JtStructural and texture layers of the image, respectively, then equation (4) is rewritten as:
Ic(x)＝(Js c(x)+Jt c(x))t(x)+Ac(1-t(x))+E(x)
＝Js c(x)t(x)+Ac(1-t(x))+Jt c(x)t(x)+E(x). (5)
wherein the background light AcIs a relatively smooth signal, and the estimated transmittance t is also close to Js cAnd thus, the captured image can be correspondingly written as: i ═ Is+ItObtaining:
Is c(x)＝Js c(x)t(x)+Ac(1-t(x)) (6)
wherein, Is cIs a low-frequency structure layer of the captured image, containing most of the structure outline of the image, It cIs the high frequency texture layer of the captured image, containing most of the texture information and noise, by which the transmittance t (x) is derived from Is(x) And the influence of noise is avoided.
3. The underwater image enhancement method based on structure-texture layering as claimed in claim 1, wherein the pre-estimated value of the transmittance t (x) is further refined by the following steps: converting the formula (6) into a 3D coordinate system to obtain a linear transmittance estimated value t' (x) of transmittance in a three-dimensional coordinate system, and changing the formula (6) into the following form by taking the background light A as a coordinate origin:
Is0(x)＝t'(x)·Js0(x) (11)
wherein, Is0(x)＝Is(x)-A，Js0(x)＝Js(x) -a, thus calculating a linear transmittance estimate for each pixel value according to the following formula:
given a limit value of the transmission
Then, by optimizing equation (13), a fine transmittance map is obtained:
where α and β are weight parameters of the control data term and the smoothing term, and σ (x) is
The standard deviation at each pixel value, N (x) denotes the four neighborhood pixel positions around each pixel value in the structural layer, A and
after the value of (c) is calculated, then this enhanced structural layer can be derived from equation (14):
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711086095.3A CN107798665B (en) | 2017-11-07 | 2017-11-07 | Underwater image enhancement method based on structure-texture layering |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711086095.3A CN107798665B (en) | 2017-11-07 | 2017-11-07 | Underwater image enhancement method based on structure-texture layering |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107798665A CN107798665A (en) | 2018-03-13 |
| CN107798665B true CN107798665B (en) | 2021-07-09 |
Family
ID=61549431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711086095.3A Expired - Fee Related CN107798665B (en) | 2017-11-07 | 2017-11-07 | Underwater image enhancement method based on structure-texture layering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107798665B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108550120B (en) * | 2018-03-29 | 2020-03-27 | 青岛大学 | Method for restoring underwater image under variable frame |
| CN109118446B (en) * | 2018-07-30 | 2021-08-24 | 西南财经大学 | Underwater image restoration and denoising method |
| CN109308380B (en) * | 2018-08-24 | 2022-08-02 | 云南大学 | Embroidery artistic style simulation method based on non-photorealistic sense |
| CN109272475B (en) * | 2018-08-31 | 2022-02-18 | 深圳纳瓦科技有限公司 | Method for rapidly and effectively repairing and strengthening underwater image color |
| CN109272539A (en) * | 2018-09-13 | 2019-01-25 | 云南大学 | The decomposition method of image texture and structure based on guidance figure Total Variation |
| CN109816604B (en) * | 2019-01-14 | 2022-09-16 | 苏州长风航空电子有限公司 | Self-adaptive clustering infrared image enhancement method |
| CN109859129A (en) * | 2019-01-29 | 2019-06-07 | 哈工大机器人（岳阳）军民融合研究院 | A kind of underwater picture enhancing treating method and apparatus |
| CN110111261B (en) * | 2019-03-28 | 2021-05-28 | 瑞芯微电子股份有限公司 | Adaptive balance processing method for image, electronic device and computer readable storage medium |
| WO2021042270A1 (en) * | 2019-09-03 | 2021-03-11 | 中山大学 | Compression artifacts reduction method based on dual-stream multi-path recursive residual network |
| CN110634112B (en) * | 2019-10-15 | 2022-02-22 | 中国矿业大学(北京) | Method for enhancing noise-containing image under mine by double-domain decomposition |
| CN110738626B (en) * | 2019-10-24 | 2022-06-28 | 广东三维家信息科技有限公司 | Rendering graph optimization method and device and electronic equipment |
| CN111340711B (en) * | 2020-05-21 | 2020-09-08 | 腾讯科技（深圳）有限公司 | Super-resolution reconstruction method, device, equipment and storage medium |
| CN111753896A (en) * | 2020-06-16 | 2020-10-09 | 江西中科九峰智慧医疗科技有限公司 | Chest radiography exposure problem classification method and system based on deep learning |
| CN112184566B (en) * | 2020-08-27 | 2023-09-01 | 北京大学 | Image processing method and system for removing adhered water mist and water drops |
| CN113012037B (en) * | 2021-03-11 | 2022-08-05 | 桂林电子科技大学 | Deep learning training data set synthesis method for underwater image restoration |
| CN113313702A (en) * | 2021-06-11 | 2021-08-27 | 南京航空航天大学 | Aerial image defogging method based on boundary constraint and color correction |
| CN116152116B (en) * | 2023-04-04 | 2023-07-21 | 青岛哈尔滨工程大学创新发展中心 | Underwater image enhancement method based on visual self-attention model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985125A (en) * | 2014-05-19 | 2014-08-13 | 清华大学 | Complicated background SAR image naval ship tail track detection method |
| CN104458766A (en) * | 2014-12-31 | 2015-03-25 | 江南大学 | Cloth surface blemish detection method based on structure texture method |
| CN104504664A (en) * | 2014-12-29 | 2015-04-08 | 河海大学常州校区 | System and method for automatically enhancing underwater images in NSCT (non-subsampled contourlet transform) regions on basis of human visual characteristics |
| CN106897972A (en) * | 2016-12-28 | 2017-06-27 | 南京第五十五所技术开发有限公司 | A kind of self-adapting histogram underwater picture Enhancement Method of white balance and dark primary |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9098926B2 (en) * | 2009-02-06 | 2015-08-04 | The Hong Kong University Of Science And Technology | Generating three-dimensional façade models from images |
| CN105450923A (en) * | 2014-09-25 | 2016-03-30 | 索尼公司 | Image processing method, image processing device and electronic device |
-
2017
- 2017-11-07 CN CN201711086095.3A patent/CN107798665B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985125A (en) * | 2014-05-19 | 2014-08-13 | 清华大学 | Complicated background SAR image naval ship tail track detection method |
| CN104504664A (en) * | 2014-12-29 | 2015-04-08 | 河海大学常州校区 | System and method for automatically enhancing underwater images in NSCT (non-subsampled contourlet transform) regions on basis of human visual characteristics |
| CN104458766A (en) * | 2014-12-31 | 2015-03-25 | 江南大学 | Cloth surface blemish detection method based on structure texture method |
| CN106897972A (en) * | 2016-12-28 | 2017-06-27 | 南京第五十五所技术开发有限公司 | A kind of self-adapting histogram underwater picture Enhancement Method of white balance and dark primary |
Non-Patent Citations (8)
| Title |
|---|
| "A Contrast Enhancement Framework with JPEG Artifacts Suppression";Li Y.等;《Computer Vision–ECCV 2014》;20141231;第174-188页 * |
| "An approach to underwater image enhancement based on image structural decomposition";Ji T.等;《Journal of Ocean University of China》;20150220;第14卷;全文 * |
| "Contrast enhancement of noisy low-light images based on structure-texture-noise decomposition";Jaemoon Lim等;《Journal of Visual Communication and Image Representation》;20170531;第45卷;全文 * |
| "Image Clarification Method Based on Structure-Texture Decomposition with Texture Refinement";Toda M.等;《Image Analysis and Processing-ICIAP 2015》;20150821;第352-362页 * |
| "Non-local Image Dehazing";D. Berman等;《2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)》;20161212;第1674-1682页 * |
| "Robust Image and Video Dehazing with Visual Artifact Suppression via Gradient Residual Minimization";Chen C.等;《Computer Vision-ECCV 2016》;20160917;第576-591页 * |
| "TV-L1分解模型下的地震图像增强方法研究";杨宁等;《地球物理学进展》;20161215;第31卷(第6期);全文 * |
| "基于图像结构—纹理分解的医学图像去噪算法的研究";付磊;《中国优秀硕士学位论文全文数据库·信息科技辑》;20160715;第2016年卷(第7期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107798665A (en) | 2018-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107798665B (en) | Underwater image enhancement method based on structure-texture layering | |
| Hu et al. | Single image defogging based on illumination decomposition for visual maritime surveillance | |
| Chiang et al. | Underwater image enhancement: using wavelength compensation and image dehazing (WCID) | |
| EP2740100A1 (en) | Method and system for removal of fog, mist or haze from images and videos | |
| Park et al. | Single image haze removal with WLS-based edge-preserving smoothing filter | |
| Hou et al. | Non-uniform illumination underwater image restoration via illumination channel sparsity prior | |
| CN112837233A (en) | Polarization image defogging method for acquiring transmissivity based on differential polarization | |
| CN110135434A (en) | Underwater picture increased quality algorithm based on color line model | |
| CN111462022B (en) | Underwater image sharpness enhancement method | |
| Wang et al. | Review on underwater image restoration and enhancement algorithms | |
| Wang et al. | Single Underwater Image Enhancement Based on $ L_ {P} $-Norm Decomposition | |
| CN117830134A (en) | Infrared image enhancement method and system based on mixed filtering decomposition and image fusion | |
| Prasath et al. | Application of different techniques for underwater image processing-a systematic review | |
| Wang et al. | Review of single image defogging | |
| Zhao et al. | Color channel fusion network for low-light image enhancement | |
| Mi et al. | Multi-scale gradient domain underwater image enhancement | |
| Song et al. | An adaptive real-time video defogging method based on context-sensitiveness | |
| Malinverni et al. | Image enhancement comparison to improve underwater cultural heritage survey | |
| Bhat et al. | Underwater Image Enhancement with Feature Preservation using Generative Adversarial Networks (UIEFP GAN) | |
| Kansal et al. | Effect of non uniform illumination compensation on dehazing/de-fogging techniques | |
| Lu et al. | A novel underwater scene reconstruction method | |
| CN108447034B (en) | Marine foggy day image defogging method based on illumination decomposition | |
| Huo et al. | Underwater image restoration based on color correction and red channel prior | |
| CN111429389A (en) | Visible light and near infrared image fusion method capable of maintaining spectral characteristics | |
| Zhang et al. | Single image dehazing method under the influence of un-uniform illumination |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210709 |