CN111784603B - RAW domain image denoising method, computer device and computer readable storage medium - Google Patents
RAW domain image denoising method, computer device and computer readable storage medium Download PDFInfo
- Publication number
- CN111784603B CN111784603B CN202010606247.3A CN202010606247A CN111784603B CN 111784603 B CN111784603 B CN 111784603B CN 202010606247 A CN202010606247 A CN 202010606247A CN 111784603 B CN111784603 B CN 111784603B
- Authority
- CN
- China
- Prior art keywords
- chromaticity
- subgraph
- brightness
- pixel
- denoising
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000001914 filtration Methods 0.000 claims abstract description 89
- 238000004364 calculation method Methods 0.000 claims abstract description 41
- 238000004590 computer program Methods 0.000 claims description 21
- 230000004927 fusion Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 description 22
- 230000008569 process Effects 0.000 description 11
- 230000009467 reduction Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 239000003086 colorant Substances 0.000 description 4
- 239000000284 extract Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention provides a RAW domain image denoising method, a computer device and a computer readable storage medium, wherein the method comprises the steps of obtaining an initial image, calculating the brightness value of each pixel according to the chromaticity value of each pixel of the initial image, and obtaining an initial brightness map; extracting chromaticity subgraphs of each color of the initial image and corresponding brightness subgraphs; performing guided filtering on each color degree subgraph by taking the brightness subgraph as a guide graph to obtain a primary denoising brightness subgraph and a primary denoising color degree subgraph; performing joint filtering on the primary denoising brightness subgraph and the corresponding primary denoising chromaticity subgraph to obtain a secondary denoising chromaticity subgraph; and calculating the output chromaticity value of each pixel by applying the chromaticity value of each pixel of the secondary denoising chromaticity subgraph, and performing inverse interpolation calculation on the plurality of chromaticity subgraphs based on the output chromaticity value of each pixel to obtain an output image. The invention also provides a computer device for realizing the method and a computer readable storage medium. The invention can improve the quality of the denoising image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a RAW domain image denoising method, a computer device for realizing the method and a computer readable storage medium.
Background
Many existing intelligent electronic devices have an image capturing function, for example, a smart phone, a tablet computer, a vehicle recorder and the like are all provided with an image capturing device, and the image capturing device is usually provided with a CMOS sensor to obtain an image. Typically, an image includes a large number of pixels, and color information of each pixel may be represented by an RGB value or a YUV value.
Currently, an image format directly collected by a common image sensor such as a CCD sensor and a CMOS sensor is called a RAW format (or Bayer format), and an output image needs to be converted into a color domain image format (RGB or YUV format) for output after being processed by an Image Signal Processor (ISP). Image sensors are susceptible to various factors during image acquisition and transmission to generate noise, such that images directly acquired by the image sensor are typically noisy images. Since noise signals are mixed with image signals, there are problems of insignificant image characteristics and low definition, noise reduction is generally required to improve the signal-to-noise ratio of images.
Image noise can be generally classified into luminance noise, which is generally high frequency noise, and color noise, which is generally low frequency noise, from a frequency perspective. Color noise is particularly noticeable in low brightness environments because the human eye is more sensitive to color noise than to brightness noise.
Since the Bayer image in the RAW domain is directly collected by the image sensor and then output, each pixel unit only contains data of one of the three primary colors of red, green and blue, and after the processing of the image signal processor, a series of operations such as demosaicing, automatic exposure, dark corner removal and the like change the original noise characteristics, so that the denoising process becomes more complex. Thus, in the prior art, luminance denoising is usually performed once in the RAW domain and then color denoising is performed in the color domain, but the following problems generally exist: firstly, brightness information is influenced while color noise is removed, so that brightness information is blurred; second, color aliasing/color shift/image saturation reduction may occur; third, the color noise cannot be removed in the RAW domain, resulting in a problem that the color noise is amplified after being processed by the image signal processor.
For example, in the prior art, there are methods of extracting chromaticity subgraphs of each color from an original image according to the color of each pixel, and denoising the chromaticity subgraphs, but this solution does not consider the difference between luminance noise and color noise, and affects the luminance noise while removing the color noise, so that luminance information is easy to be blurred. Moreover, the denoising process of the image in the prior art is usually first-stage filtering, and structural noise is easy to generate.
Disclosure of Invention
The invention mainly aims to provide a RAW domain image denoising method capable of removing brightness noise and color noise and avoiding structural noise.
Another object of the present invention is to provide a computer apparatus for implementing the above RAW domain image denoising method.
It is still another object of the present invention to provide a computer readable storage medium implementing the above RAW domain image denoising method.
In order to achieve the main purpose of the invention, the RAW domain image denoising method provided by the invention comprises the steps of obtaining an initial image, calculating the brightness value of each pixel according to the chromaticity value of each pixel of the initial image, and obtaining an initial brightness map; extracting chromaticity subgraphs of each color of the initial image and corresponding brightness subgraphs; performing guided filtering on each color degree subgraph by taking the brightness subgraph as a guide graph to obtain a primary denoising brightness subgraph and a primary denoising color degree subgraph; performing joint filtering on the primary denoising brightness subgraph and the corresponding primary denoising chromaticity subgraph to obtain a secondary denoising chromaticity subgraph; and calculating the output chromaticity value of each pixel by applying the chromaticity value of each pixel of the secondary denoising chromaticity subgraph, and performing inverse interpolation calculation on the plurality of chromaticity subgraphs based on the output chromaticity value of each pixel to obtain an output image.
According to the scheme, the luminance subgraph is used as a guide graph to conduct guide filtering on each color subgraph, and the guide filtering is mainly used for removing the luminance noise in the initial image, so that the influence on the removal of the luminance noise caused by interference on the luminance noise when the color noise is removed later is avoided. And by carrying out the channel separation processing on each color, the mutual interference of the noise of each color in the denoising process can be avoided, the occurrence of color aliasing or color deviation is avoided, and the problem of structural noise can also be avoided. In addition, the first-stage filtering is the guide filtering, the second-stage filtering is the joint filtering, the joint filtering takes the luminance subgraph and the chromaticity subgraph as references, and the luminance noise is removed during the guide filtering, so that serious color shift caused by overlarge luminance noise can be avoided during the joint filtering in the process of filtering the color noise, and most of the color noise can be removed.
In a preferred embodiment, after the primary denoising luminance subgraph and the primary denoising chromaticity subgraph are obtained, the following steps are further performed: performing downsampling interpolation calculation on the primary denoising brightness subgraph and the primary denoising chromaticity subgraph; the performing joint filtering on the primary denoising luminance subgraph and the corresponding primary denoising chromaticity subgraph comprises: and applying the primary denoising brightness subgraph after downsampling to perform joint filtering on the primary denoising chromaticity subgraph.
Therefore, the high-frequency characteristics of the primary denoising luminance subgraph and the primary denoising chrominance subgraph can be further filtered through downsampling interpolation calculation on the primary denoising luminance subgraph and the primary denoising chrominance subgraph, and the luminance noise can be further filtered through downsampling because the luminance noise is mainly the high-frequency characteristics, so that the removal of the color noise due to the existence of the luminance noise during the color noise filtering is avoided.
The further scheme is that the obtaining of the secondary denoising chromaticity subgraph comprises the following steps: and performing up-sampling interpolation calculation on the image subjected to joint filtering by the primary denoising brightness subgraph and the primary denoising chromaticity subgraph after the down-sampling is applied to obtain a secondary denoising chromaticity subgraph.
Therefore, the total number of the pixel points of the secondary denoising chromaticity subgraph and the primary denoising chromaticity subgraph can be ensured to be equal after upsampling, and the calculation of the output chromaticity value of each pixel is facilitated.
In a further aspect, calculating the output chromaticity value of each pixel using the pixel chromaticity value of the secondary denoising chromaticity subgraph includes: and calculating the output chromaticity value of each pixel by using the secondary denoising chromaticity subgraph and the primary denoising chromaticity subgraph and the chromaticity value of each pixel of the initial image.
More preferably, calculating the output chromaticity value of each pixel using the pixel chromaticity value of the secondary denoising chromaticity sub-image includes: and carrying out fusion calculation on the chromaticity values of each pixel of the primary image by using the secondary denoising chromaticity subgraph and the primary denoising chromaticity subgraph according to a preset weight value to obtain the output chromaticity value of each pixel.
Therefore, the output chromaticity value of each pixel is calculated according to the secondary denoising chromaticity subgraph, the primary denoising chromaticity subgraph and the chromaticity value of each pixel of the initial image, the calculated output chromaticity value of each pixel can be ensured to be closer to the true color, and the color of the denoised image is more vivid.
In a preferred embodiment, the performing the guided filtering on each color level sub-graph with the luminance sub-graph as the guide graph includes: acquiring a first brightness search window of a brightness sub-graph and a first chroma search window of each chroma sub-graph, traversing each pixel of the first brightness search window, carrying out matching calculation of a matching window on each pixel, calculating an average value of brightness values of all pixels meeting matching requirements in the first brightness search window as a guiding filtering brightness value of a current pixel, and calculating an average value of chroma values of all pixels meeting matching requirements in the first chroma search window as a guiding filtering chroma value of the current pixel; the matching requirement is that the brightness value of the pixel in the matching window meets the preset brightness threshold requirement.
Therefore, in the process of conducting guide filtering on each color subgraph by taking the brightness subgraph as the guide graph, only judging whether the brightness value of the pixel in the matching window meets the preset brightness threshold requirement, namely, only taking the brightness information as the main parameter of the filter, so that the image after guide filtering can retain color noise, the initial denoising color subgraph used in the joint filtering still retains the color noise, and the color noise is prevented from being removed when the brightness noise is removed.
Still further, the matching the brightness value of the pixel in the window to meet the preset brightness threshold requirement includes: the preset error arithmetic value of the brightness value of each pixel in the matching window is smaller than the preset brightness threshold value.
It can be seen that, whether the current matching window meets the preset requirement is determined by the preset brightness threshold, the calculated amount is reduced, and when the algorithm of the invention is realized by the hardware circuit, the hardware circuit is simple, and the realization cost of the invention is reduced.
In a further aspect, performing joint filtering on the primary denoising luminance subgraph and the corresponding primary denoising chrominance subgraph includes: acquiring a second brightness search window of the primary denoising brightness subgraph and a second chromaticity search window of each primary denoising chromaticity subgraph, traversing each pixel of the second brightness search window and the corresponding second chromaticity search window, carrying out matching calculation of a matching window on each pixel, calculating average values of brightness values of all pixels meeting matching requirements in the second brightness search window and the corresponding second chromaticity search window as joint filtering brightness values of the current pixel, and calculating average values of chromaticity values of all pixels meeting matching requirements in the second brightness search window and the corresponding second chromaticity search window as joint filtering chromaticity values of the current pixel; wherein, satisfy the matching requirement and be: the luminance values of the pixels in the matching window meet the preset luminance threshold requirements, and the chrominance values of the pixels in the matching window meet the preset chrominance threshold requirements.
Therefore, in the process of the joint filtering, the conditions of the brightness value and the chromaticity value of each pixel are considered at the same time, that is, the brightness value and the chromaticity value are taken as main parameters of the filter together for filtering, so that the brightness noise and the color noise of each pixel can be filtered at the same time. Since the luminance noise is already filtered in the pilot filtering, the influence of the luminance noise on the color noise can be avoided in the joint filtering.
In order to achieve the above another object, the present invention provides a computer apparatus including a processor and a memory, the memory storing a computer program, the computer program implementing each step of the above RAW domain image denoising method when executed by the processor.
In order to achieve the above-mentioned still another object, the present invention provides a computer program stored on a computer readable storage medium, which when executed by a processor, implements the steps of the above-mentioned RAW domain image denoising method.
Drawings
Fig. 1 is a flowchart of an embodiment of a RAW domain image denoising method according to the present invention.
Fig. 2 is a schematic diagram of the arrangement of the chromaticity values of each pixel of the initial image.
FIG. 3 is a schematic illustration of a plurality of neighborhood windows in an initial image.
Fig. 4 is a schematic diagram of extracting a plurality of chromaticity subgraphs from an initial image.
Fig. 5 is a schematic diagram of extracting a plurality of luminance subgraphs from an initial image.
Fig. 6 is a schematic diagram of a search window and a matching window.
Fig. 7 is a schematic diagram of downsampling in an embodiment of the RAW domain image denoising method according to the present invention.
The invention is further described below with reference to the drawings and examples.
Detailed Description
The RAW domain image denoising method is applied to intelligent electronic equipment, preferably, the intelligent electronic equipment is provided with an image pickup device, such as a camera, and the like, the image pickup device is provided with an image sensor, such as a CMOS (complementary metal oxide semiconductor), and the intelligent electronic equipment acquires an initial image by using the image pickup device. Preferably, the intelligent electronic device is provided with a processor and a memory, wherein the memory stores a computer program, and the processor implements the RAW domain image denoising method by executing the computer program.
RAW domain image denoising method embodiment:
in this embodiment, referring to fig. 1, step S1 is first performed to obtain an initial image, and specifically, an initial image output by a CCD sensor or a CMOS sensor is obtained. Typically, the color information of the initial image is RGB information. Taking a Bayer image as an example, the Bayer image format is shown in fig. 2, where the original image has a large number of pixels each having one color information, for example, the color information of the first row of pixels is the color information of red R or green Gr, the red R pixels and the green Gr pixels are arranged at intervals, the color information of the second row of pixels is the color information of green Gb or blue B, and the green Gb pixels and the blue B pixels are arranged at intervals. The color information of each pixel is a chromaticity value, which is typically a binary number of 0 to 255.
Then, step S2 is performed to calculate the luminance value of each pixel, and an initial luminance map is formed. Specifically, for each pixel, interpolation calculation is performed on the pixel neighborhood 3×3 pixels. As shown in fig. 3, if the center pixel of a certain pixel is red R, the structure of the neighborhood 3×3 pixels of the pixel is as shown in fig. 3 (a), and the luminance value of the red R pixel can be obtained by the following formula:
y= (4×r+2×Σg+Σb)/16 (formula 1)
If the center pixel of a certain pixel is blue B, the structure of the neighborhood 3×3 pixels of the pixel is as shown in fig. 3 (d), and the luminance value of the blue B pixel can be obtained by the following formula:
y= (4×b+2×Σg+Σr)/16 (formula 2)
Accordingly, if the center pixel of a certain pixel is green Gr or Gb, the structure of the neighborhood 3×3 pixels of the pixel is as shown in fig. 3 (b) or fig. 3 (c), and the luminance value of the green Gr or Gb pixel can be obtained by the following formula:
Y＝(4×G 0 +2×Σr+2×Σb)/8 (formula 3)
In the above equations 1, 2 and 3, ΣR is the sum of the chromaticity values of all red R pixels in the 3×3 neighborhood, ΣB is the sum of the chromaticity values of all green B pixels in the 3×3 neighborhood, ΣG is the sum of the chromaticity values of all green Gr and Gb pixels in the 3×3 neighborhood, and G 0 The chromaticity value of the center pixel is the chromaticity value of the center pixel when the center pixel is green Gr or green Gb pixel.
After calculating the luminance value of each pixel, each pixel in the initial image will have a corresponding luminance value, and each pixel is represented by the luminance value, i.e. an initial luminance map is formed.
Next, step S3 is executed to extract chromaticity subgraphs of each color, and extract luminance subgraphs corresponding to each chromaticity subgraph. Specifically, all red R pixels in the initial image are extracted to form a chromaticity subgraph of red R pixels, all blue B pixels in the initial image are extracted to form a chromaticity subgraph of blue B pixels, all green Gr pixels in the initial image are extracted to form a chromaticity subgraph of green Gr pixels, all green Gb pixels in the initial image are extracted to form a chromaticity subgraph of green Gb pixels, thereby forming four chromaticity subgraphs, and fig. 4 shows the structure of the four chromaticity subgraphs.
When each chromaticity subgraph is extracted, the luminance subgraph corresponding to each color channel is correspondingly extracted from the initial luminance graph according to the color channel of each pixel, as shown in fig. 5, for red R pixels, the luminance values of all red R pixels are extracted and the luminance subgraph of red R pixels is formed, the luminance values of all blue B pixels are extracted and the luminance subgraph of blue B pixels is formed, the luminance values of all green Gr pixels are extracted and the luminance subgraph of green Gr pixels is formed, and the luminance values of all green Gb pixels are extracted and the luminance subgraph of green Gb pixels is formed, so that four luminance subgraphs are formed. It can be seen that each luminance sub-graph of the present embodiment corresponds to one chromaticity sub-graph, i.e. the chromaticity sub-graph of the red R pixel corresponds to the luminance sub-graph of the red R pixel, and so on.
Since the number of the red R pixels in the initial image is 1/4 of that of the initial image, the width and the height of the chromaticity subgraph of the red R pixels obtained by extraction and the brightness subgraph of the corresponding red R pixels are 1/2 of that of the initial image. Similarly, for the blue B pixel, the green Gr pixel and the green Gb pixel, the width and the height of the extracted chromaticity subgraph and the extracted luminance subgraph are 1/2 of the width and the height of the initial image.
And then, executing step S4, taking the brightness subgraph as a guide graph, and carrying out guide filtering on the chromaticity subgraph corresponding to the brightness subgraph to obtain a primary denoising brightness subgraph and a primary denoising chromaticity subgraph. In this embodiment, when the guided image is used to perform filtering noise reduction, the calculated filter parameters are directly used to guide the corresponding chromaticity subgraph to perform filtering operation, and when the filter parameters are calculated, only the data of the luminance subgraph, that is, the luminance value of each pixel in the luminance subgraph is used as the reference standard of filtering, so that the calculated primary denoising luminance subgraph and the primary denoising chromaticity subgraph are images for denoising luminance, that is, the guided filtering is mainly to remove the luminance noise in the images, the color noise in the images is kept as much as possible, and the color noise is filtered in the subsequent joint filtering.
The specific steps of the guided filtering are described in detail below. The guided filtering in this embodiment is an operation performed for each pixel of the luminance sub-graph and the chrominance sub-graph, that is, for each pixel, it is necessary to calculate the chrominance value and the luminance value after the primary denoising. Firstly, for a pixel to be calculated, a search window with a neighborhood size of 9×9 is extracted by taking the current pixel as a center, and if the search window is extracted under a luminance subgraph, the search window is a first luminance search window. Fig. 6 shows the extraction of a first luminance search window 10 of size 9 x 9 from the luminance subgraph.
Then, a matching window 11 with a size of 3×3 centered on the current pixel is set, then all pixels in the search window 10 are traversed, a neighborhood window (the size of the neighborhood window is also 3×3) of each pixel in the search window 10 is taken as a matching window, for example, a matching window 12, the center pixel of the matching window 12 is a pixel to be matched, and the matching window 12 of each pixel and the matching window 11 of the current pixel are subjected to matching operation. The matching operation of the present embodiment is performed using a sum of absolute error algorithm (SAD algorithm). Specifically, the absolute value of the difference between the luminance value of each pixel in the matching window 11 of the current pixel and the luminance value of the corresponding pixel in the matching window 12 is calculated, for example, the absolute value of the difference between the luminance value of the pixel in the upper left corner of the matching window 11 and the luminance value of the pixel in the upper left corner of the matching window 12 is calculated, the absolute values of the differences of the luminance values corresponding to the other eight pixels are sequentially calculated, then the sum of the absolute values of the differences of the luminance values of the nine pixels is calculated, whether the sum of the absolute values of the differences of the luminance values of the nine pixels is larger than a preset luminance threshold is judged, and if not, the center pixel of the matching window 12 is considered to be similar to the center pixel of the matching window 11 of the current pixel, namely, the pixel to be matched is considered to be similar to the current pixel.
After traversing all pixels of the current search window 10, determining all pixels to be matched similar to the current pixel, accumulating the brightness values of all similar pixels to be matched, and calculating the average value of the brightness values of all similar pixels to be matched, wherein the average value is used as the brightness value of the current pixel after primary filtering. After the brightness value of each pixel after primary filtering is calculated, a brightness subgraph after primary denoising can be obtained.
And extracting a first chroma search window for the chroma subgraph, wherein each pixel of the first chroma search window and each pixel of the first brightness search window are in one-to-one correspondence. Then, after all pixels to be matched similar to the current pixel are determined, calculating the average value of the chrominance values of all similar pixels to be matched, and taking the average value of the chrominance values as the chrominance value of the current pixel after primary filtering. After the first filtered chrominance value of each pixel is calculated, a first denoised chrominance subgraph can be obtained. It should be noted that, the criterion for determining whether a pixel is similar to the current pixel is that the sum of absolute values of differences of luminance values of nine pixels in the matching window is not greater than a preset luminance threshold, that is, the criterion for determining is based on the luminance value instead of the chrominance value, so that the guiding filtering is filtering performed by using a luminance subgraph as a guiding graph, and parameters of the filter are related to the luminance value and are unrelated to the chrominance value of the pixel. In this way, after the guided filtering, most of the brightness noise in the image can be removed, that is, the denoising operation of the brightness noise is performed once.
Through the operation, four primary denoising chromaticity subgraphs of four colors and corresponding four primary denoising chromaticity subgraphs can be obtained. Then, step S5 is executed to perform downsampling interpolation calculation on each primary denoising chromaticity subgraph and the corresponding primary denoising luminance subgraph. Specifically, the downsampling calculation may be implemented by using a Binning manner, as shown in fig. 7, which is a schematic diagram of 2 times downsampling interpolation calculation, and the image is sampled by 2 pixels at intervals, that is, adjacent 2×2 pixels are averaged to obtain a downsampled value, for example, for a 2×2 pixel window 15, an average value of chromaticity values of four pixels in the window 15 is obtained as a chromaticity value of the downsampled pixel 16, and the average value of luminance values of four pixels in the window 15 is obtained as a luminance value of the downsampled pixel 16.
In this way, after carrying out 2 times of downsampling interpolation calculation on the primary denoising chromaticity subgraph, the resolution of the obtained downsampled image is 1/2 of that of the primary denoising chromaticity subgraph; similarly, if 4 times of downsampling interpolation calculation is performed, the resolution of the obtained image is 1/4 of that of the primary denoising chromaticity subgraph. The luminance subgraph is subjected to downsampling interpolation calculation, that is, the luminance values of the pixels are filtered, that is, the average value of the luminance values of the four pixels is used instead of the luminance values of the four pixels.
And then, executing step S6, and carrying out joint filtering on each channel chromaticity subgraph after downsampling and the corresponding brightness subgraph to obtain a secondary denoising chromaticity subgraph. In this embodiment, the calculation of the joint filtering is similar to the guided filtering calculation, except that the calculation of the filter coefficients is performed according to the luminance value and the chrominance value of each pixel at the same time, instead of the calculation depending on the data of the luminance value alone.
The specific steps of the joint filtering are described in detail below. Firstly, for a pixel to be calculated, taking the current pixel as a center, extracting a search window with the neighborhood size of 9×9 from the primary de-noised luminance subgraph after downsampling, wherein the search window is a second luminance search window, and extracting a search window with the neighborhood size of 9×9 from the primary de-noised chrominance subgraph after downsampling, wherein the search window is a second chrominance search window.
Then, setting a matching window with the current pixel as a center and having a size of 3×3 for the second brightness search window, traversing all pixels in the second brightness search window, taking a neighborhood window (the size of the neighborhood window is also 3×3) of each pixel in the second brightness search window as a matching window of the pixel to be matched, and performing matching operation, such as absolute error sum algorithm (SAD algorithm), on the matching window of the current pixel and the matching window of the pixel to be matched, wherein the matching operation of the absolute error sum algorithm is not repeated.
And, for the second chromaticity search window, a matching window with the current pixel as the center and the size of 3×3 is set, then all pixels in the second chromaticity search window are traversed, a neighborhood window (the size of the neighborhood window is also 3×3) of each pixel in the second chromaticity search window is taken as a matching window of the pixel to be matched, and then the matching window of the current pixel and the matching window of the pixel to be matched are subjected to matching operation, such as absolute error sum algorithm (SAD algorithm) matching operation. For the second chromaticity search window, the chromaticity value of each pixel, not the luminance value, is used when the matching operation of the absolute error and the algorithm is performed.
In the joint filtering calculation, the factors of the luminance value and the chrominance value need to be considered simultaneously, namely, a certain pixel to be matched is confirmed to be similar to the current pixel, and the following conditions need to be met: under the downsampled primary denoising luminance subgraph, a matching operation result of a matching window of the current pixel and a matching window of the pixel to be matched is not larger than a luminance threshold value, and under the downsampled primary denoising chrominance subgraph, a matching operation result of the matching window of the current pixel and the matching window of the pixel to be matched is not larger than a chrominance threshold value. If a certain pixel to be matched meets the above condition, the pixel to be matched is considered to be similar to the current pixel, otherwise, the pixel to be matched is not considered to be similar to the current pixel.
After traversing the second brightness searching window and all pixels of the second chromaticity searching window, determining all pixels to be matched similar to the current pixel, accumulating chromaticity values of all similar pixels to be matched, calculating an average value of the chromaticity values of all similar pixels to be matched, and taking the average value as the chromaticity value of the current pixel after secondary filtering. After calculating the secondary filtered chromaticity value of each pixel, a secondary denoised chromaticity subgraph can be obtained.
In the calculation process of the joint filtering noise reduction, the parameter design of the filter refers to the data of the brightness value and the chromaticity value at the same time, main brightness noise is removed before the joint filtering noise reduction, and the downsampling interpolation calculation is carried out on the brightness subgraph, so that most of color noise can be removed through the joint filtering noise reduction, and the interference of the brightness noise on the color noise is very small.
Then, step S7 is executed to perform up-sampling interpolation calculation on the secondary denoising chromaticity subgraph obtained by the joint filtering. Because the step S5 performs the downsampling interpolation calculation on the primary denoising chromaticity subgraph, the step S7 performs the upsampling interpolation calculation according to the same multiplying power, and after the upsampling interpolation calculation, the obtained pixels of the secondary denoising chromaticity subgraph are the same as the pixels of the primary denoising chromaticity subgraph. In this embodiment, there may be various methods for up-sampling interpolation calculation, such as bilinear interpolation, bicubic interpolation, and Cubic interpolation, which are not limited in this embodiment.
Next, step S8 is executed, where the output chromaticity value of each pixel is calculated by applying the chromaticity value of each pixel of the secondary denoising chromaticity subgraph subjected to the upsampling interpolation calculation, and specifically, in this embodiment, the upsampling secondary denoising chromaticity subgraph, the primary denoising chromaticity subgraph and the chromaticity value of each pixel in the initial image are applied to perform fusion calculation according to a certain weight, so as to obtain the output chromaticity value of each pixel, so as to maintain original image details as far as possible, and reduce the influence of the denoising algorithm on the initial image. Specifically, the fusion calculation is to perform weighted fusion calculation on the filtering result of the two times and the initial image according to the set fusion weight, for example, the calculation is performed by using the following formula:
P out ＝w 2 ×P filt2 +(1-w 2 )×(w 1 ×P filt1 +(1-w 1 )×P ori ) (4)
Wherein P is out For the output chromaticity value of a pixel, w 1 Weight, w, of primary denoising chromaticity subgraph 2 Weights for secondary denoising chromaticity subgraphs, P filt1 For the chroma value, P, of the pixel in the primary denoising chroma subgraph filt2 For the chroma value, P, of the pixel in the upsampled secondary denoising chroma sub-graph ori Is the chrominance value of that pixel in the original image. It will be appreciated that the above formula is an exemplary formula, and other variations of the above formula are possible in practical application, and the present embodiment is not limited thereto.
Finally, step S9 is executed, performing an inverse interpolation calculation according to the plurality of chromaticity subgraphs, and outputting the denoised image. Since the step S3 is to extract the chromaticity subgraphs of each color channel from the initial image, that is, divide the initial image into four chromaticity subgraphs according to the color of each pixel, the step S9 is to restore the positions of each pixel in the initial image according to the Bayer format of the initial image in the reverse process of the step S3, thereby forming an output image, and the chromaticity value of each pixel of the output image is the output chromaticity value calculated in the step S8.
The invention firstly extracts the chromaticity subgraph and the corresponding brightness subgraph of each color channel according to the colors of each pixel, and carries out guiding filtering and combined filtering, so that the chromaticity values and the brightness values among the pixels with different colors in the filtering process can not interfere with each other, the conditions of color aliasing and color shift of the filtered image are avoided, the problem of structural noise is avoided, and the quality of the filtered image is improved.
In addition, as the color noise is filtered before the primary brightness noise is filtered, the invention performs two-stage filtering, compared with the mode of only one-stage filtering in the prior art, the invention can avoid the problem of fuzzy brightness noise information and has better image denoising effect. In addition, the calculation amount of the invention is not large, and when the algorithm of the invention is realized by using hardware, a hardware circuit is not complex, so that the realization cost of the invention can be reduced.
Of course, the above examples are preferred embodiments of the present invention, and the following changes may be made during practical application:
in the processes of pilot filtering and joint filtering noise reduction, the matching algorithm used is not limited to calculating pixel similarity by using an absolute error sum algorithm, but may also be calculated in other manners to achieve similar effects, such as using a mean absolute difference algorithm (MAD), a sum of absolute error algorithm (SAD), a sum of square error algorithm (SSD), a sum of square error algorithm (MSD), a normalized product correlation algorithm (NCC), a Sequential Similarity Detection Algorithm (SSDA), a hadamard transformation algorithm (SATD), and the like.
In addition, in the processes of guiding filtering and combined filtering noise reduction, the filtering of similar pixels by using the mean filter is not limited, and other filtering noise reduction algorithms can be adopted to achieve similar effects, such as non-local mean (NLM), bilateral filtering and Gaussian filtering methods. In addition, the joint filtering noise reduction process is not limited to filtering noise reduction through similarity pixels, and can also adopt guide filtering or frequency domain filtering to achieve similar effects.
Finally, the filter window size of the pilot filter and the joint filter can also be adjusted, and the above embodiment uses a 9×9 neighborhood as the search window and a 3×3 neighborhood as the matching window, but similar effects can be achieved with other sizes, for example, 5×5, 7×7, 11×11 neighborhood as the search window and 5×5, 7×7 neighborhood as the matching window.
Computer apparatus embodiment:
the computer device of the present embodiment may be an intelligent electronic device, and the computer device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, where the processor executes the computer program to implement each step of the RAW domain image denoising method. Of course, the intelligent electronic device further comprises an image capturing device for acquiring the initial image.
For example, a computer program may be split into one or more modules, which are stored in memory and executed by a processor to perform the various modules of the invention. One or more of the modules may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program in the terminal device.
The processor referred to in the present invention may be a central processing unit (Central Processing Unit, CPU), or other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being a control center of the terminal device, and the various interfaces and lines being used to connect the various parts of the overall terminal device.
The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory, and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
Computer-readable storage medium embodiments:
the computer program stored in the above-mentioned computer means may be stored in a computer readable storage medium if it is implemented in the form of software functional units and sold or used as a separate product. Based on such understanding, the present invention may implement all or part of the procedures in the above-described embodiment method, or may be implemented by instructing related hardware by a computer program, which may be stored in a computer readable storage medium, and the computer program may implement the steps of the RAW domain image denoising method when executed by a processor.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the content of the computer readable medium can be appropriately increased or decreased according to the requirements of the jurisdiction's jurisdiction and the patent practice, for example, in some jurisdictions, the computer readable medium does not include electrical carrier signals and telecommunication signals according to the jurisdiction and the patent practice.
Finally, it should be emphasized that the invention is not limited to the above-described embodiments, such as variations of the filtering template, or variations of the specific algorithm for performing the mean filtering, etc., which are also intended to be included in the scope of the claims.
Claims (10)
1. A RAW domain image denoising method, comprising:
acquiring an initial image, calculating the brightness value of each pixel according to the chromaticity value of each pixel of the initial image, and acquiring an initial brightness map;
extracting chromaticity subgraphs of each color of the initial image and corresponding brightness subgraphs;
the method is characterized in that:
performing guided filtering on each chromaticity subgraph by taking the luminance subgraphs as a guide graph to obtain primary denoising luminance subgraphs and primary denoising chromaticity subgraphs;
performing joint filtering on the primary denoising brightness subgraph and the corresponding primary denoising chromaticity subgraph to obtain a secondary denoising chromaticity subgraph;
calculating the output chromaticity value of each pixel by applying the chromaticity value of each pixel of the secondary denoising chromaticity subgraph, and performing inverse interpolation calculation on a plurality of chromaticity subgraphs based on the output chromaticity value of each pixel to obtain an output image;
wherein performing guided filtering on each chromaticity subgraph by taking the luminance subgraph as a guide graph comprises: acquiring a first brightness search window of the brightness subgraph and a first chromaticity search window of each chromaticity subgraph, traversing each pixel of the first brightness search window, carrying out matching calculation of a matching window on each pixel, calculating an average value of brightness values of all pixels meeting matching requirements in the first brightness search window as a guiding filtering brightness value of a current pixel, and calculating an average value of chromaticity values of all pixels meeting matching requirements in the first chromaticity search window as a guiding filtering chromaticity value of the current pixel;
the performing joint filtering on the primary denoising brightness subgraph and the corresponding primary denoising chromaticity subgraph comprises: obtaining a second brightness search window of the primary denoising brightness subgraph and a second chroma search window of each primary denoising chroma subgraph, traversing each pixel of the second brightness search window and the corresponding second chroma search window, carrying out matching calculation of a matching window on each pixel, calculating average values of brightness values of all pixels meeting matching requirements in the second brightness search window and the corresponding second chroma search window as joint filtering brightness values of current pixels, and calculating average values of the brightness values of all pixels meeting matching requirements in the second brightness search window and the corresponding second chroma search window as joint filtering chroma values of the current pixels.
2. The RAW domain image denoising method according to claim 1, wherein:
after the primary denoising brightness subgraph is obtained, the primary denoising chromaticity subgraph is further executed: performing downsampling interpolation calculation on the primary denoising brightness subgraph and the primary denoising chromaticity subgraph;
the performing joint filtering on the primary denoising brightness subgraph and the corresponding primary denoising chromaticity subgraph comprises: and applying the primary denoising brightness subgraph after downsampling and the primary denoising chromaticity subgraph to perform joint filtering.
3. The RAW domain image denoising method according to claim 2, wherein:
the obtaining the secondary denoising chromaticity subgraph comprises the following steps: and carrying out up-sampling interpolation calculation on the image subjected to the joint filtering on the primary denoising brightness subgraph and the primary denoising chromaticity subgraph after the down-sampling is applied to obtain the secondary denoising chromaticity subgraph.
4. A RAW domain image denoising method according to any one of claims 1 to 3, wherein:
the calculating of the output chromaticity value of each pixel using the pixel chromaticity value of the secondary denoising chromaticity subgraph includes: and calculating the output chromaticity value of each pixel by applying the secondary denoising chromaticity subgraph and the primary denoising chromaticity subgraph and the chromaticity value of each pixel of the initial image.
5. The RAW domain image denoising method according to claim 4, wherein:
the calculating of the output chromaticity value of each pixel using the pixel chromaticity value of the secondary denoising chromaticity subgraph includes: and carrying out fusion calculation on the secondary denoising chromaticity subgraph, the primary denoising chromaticity subgraph and the pixel chromaticity values of the initial image according to a preset weight value to obtain the output chromaticity value of each pixel.
6. A RAW domain image denoising method according to any one of claims 1 to 3, wherein:
and when the average value of the brightness values of all the pixels meeting the matching requirement in the first brightness searching window is calculated as the guiding filtering brightness value of the current pixel, the brightness value meeting the matching requirement is that the brightness value of the pixel in the matching window meets the preset brightness threshold requirement.
7. The RAW domain image denoising method according to claim 6, wherein:
the matching window includes: and the preset error arithmetic value of the brightness value of each pixel in the matching window is smaller than the preset brightness threshold value.
8. A RAW domain image denoising method according to any one of claims 1 to 3, wherein:
when calculating the average value of the brightness values of all pixels meeting the matching requirement in the second brightness search window and the corresponding second chromaticity search window as the joint filtering brightness value of the current pixel, the matching requirement is met: the brightness value of the pixel in the matching window meets the preset brightness threshold requirement, and the chromaticity value of the pixel in the matching window meets the preset chromaticity threshold requirement.
9. Computer arrangement, characterized by comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, implements the steps of the RAW domain image denoising method according to any one of claims 1 to 8.
10. A computer readable storage medium having stored thereon a computer program characterized by: the computer program, when executed by a processor, implements the steps of the RAW domain image denoising method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010606247.3A CN111784603B (en) | 2020-06-29 | 2020-06-29 | RAW domain image denoising method, computer device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010606247.3A CN111784603B (en) | 2020-06-29 | 2020-06-29 | RAW domain image denoising method, computer device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111784603A CN111784603A (en) | 2020-10-16 |
| CN111784603B true CN111784603B (en) | 2024-01-26 |
Family
ID=72760787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010606247.3A Active CN111784603B (en) | 2020-06-29 | 2020-06-29 | RAW domain image denoising method, computer device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111784603B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112446848A (en) * | 2020-12-18 | 2021-03-05 | 维沃移动通信有限公司 | Image processing method and device and electronic equipment |
| CN114723613A (en) * | 2021-01-05 | 2022-07-08 | 北京小米移动软件有限公司 | Image processing method and device, electronic device and storage medium |
| CN112862717B (en) * | 2021-02-10 | 2022-09-20 | 山东英信计算机技术有限公司 | Image denoising and blurring method, system and medium |
| CN113284065A (en) * | 2021-05-26 | 2021-08-20 | Oppo广东移动通信有限公司 | Video image denoising method and device, electronic equipment and readable storage medium |
| CN113240607A (en) * | 2021-05-26 | 2021-08-10 | Oppo广东移动通信有限公司 | Image denoising method and device, electronic equipment and storage medium |
| CN115908210A (en) * | 2021-08-05 | 2023-04-04 | 中兴通讯股份有限公司 | Image processing method, electronic device, computer-readable storage medium |
| CN115988311A (en) * | 2021-10-14 | 2023-04-18 | 荣耀终端有限公司 | Image processing method and electronic equipment |
| CN114331899A (en) * | 2021-12-31 | 2022-04-12 | 上海宇思微电子有限公司 | Image noise reduction method and device |
| CN116912132B (en) * | 2023-09-11 | 2023-12-26 | 归芯科技(深圳)有限公司 | Bayer domain 3DNR image denoising method and device thereof |
| CN117376485B (en) * | 2023-10-23 | 2024-04-23 | 上海为旌科技有限公司 | Method and filter for removing image chroma noise by YUV (YUV) combination |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7983511B1 (en) * | 2010-11-29 | 2011-07-19 | Adobe Systems Incorporated | Methods and apparatus for noise reduction in digital images |
| CN105430357A (en) * | 2015-11-26 | 2016-03-23 | 上海富瀚微电子股份有限公司 | Demosaicing method and device for image sensor |
| CN106027854A (en) * | 2016-06-22 | 2016-10-12 | 凌云光技术集团有限责任公司 | United filtering denoising method which is applied to a camera and is applicable to be realized in FPGA (Field Programmable Gate Array) |
| CN111161188A (en) * | 2019-12-30 | 2020-05-15 | 珠海全志科技股份有限公司 | Method for reducing image color noise, computer device and computer readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8488031B2 (en) * | 2011-01-14 | 2013-07-16 | DigitalOptics Corporation Europe Limited | Chromatic noise reduction method and apparatus |
-
2020
- 2020-06-29 CN CN202010606247.3A patent/CN111784603B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7983511B1 (en) * | 2010-11-29 | 2011-07-19 | Adobe Systems Incorporated | Methods and apparatus for noise reduction in digital images |
| CN105430357A (en) * | 2015-11-26 | 2016-03-23 | 上海富瀚微电子股份有限公司 | Demosaicing method and device for image sensor |
| CN106027854A (en) * | 2016-06-22 | 2016-10-12 | 凌云光技术集团有限责任公司 | United filtering denoising method which is applied to a camera and is applicable to be realized in FPGA (Field Programmable Gate Array) |
| CN111161188A (en) * | 2019-12-30 | 2020-05-15 | 珠海全志科技股份有限公司 | Method for reducing image color noise, computer device and computer readable storage medium |
Non-Patent Citations (3)
| Title |
|---|
| RGBX格式图像传感器的去马赛克算法;董鹏宇;;集成电路应用(第05期);全文 * |
| 基于多引导滤波的图像增强算法;刘杰;张建勋;代煜;;物理学报(第23期);全文 * |
| 基于轮廓波变换的图像混合噪声去除算法研究;许友雷;舒忠;;现代计算机(第33期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111784603A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784603B (en) | RAW domain image denoising method, computer device and computer readable storage medium | |
| US8363123B2 (en) | Image pickup apparatus, color noise reduction method, and color noise reduction program | |
| CN111784605B (en) | Image noise reduction method based on region guidance, computer device and computer readable storage medium | |
| EP2089848B1 (en) | Noise reduction of panchromatic and color image | |
| CN110246087B (en) | System and method for removing image chroma noise by referring to multi-resolution of multiple channels | |
| CN111260580B (en) | Image denoising method, computer device and computer readable storage medium | |
| JP2020166628A (en) | Image processing method, image processing device, program, image processing system, and learned model manufacturing method | |
| CN111161188B (en) | Method for reducing image color noise, computer device and readable storage medium | |
| EP3891693A1 (en) | Image processor | |
| JP2020144488A (en) | Image processing method, image processing device, program, image processing system, and method of producing trained model | |
| CN110852953B (en) | Image interpolation method and device, storage medium, image signal processor and terminal | |
| US8471933B2 (en) | Image processing apparatus, image processing method and computer program | |
| US8189066B2 (en) | Image processing apparatus, image processing method, and computer-readable medium | |
| CN113168669A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| EP3275169B1 (en) | Downscaling a digital raw image frame | |
| KR101600312B1 (en) | Apparatus and method for processing image | |
| JP5291788B2 (en) | Imaging device | |
| CN112907468A (en) | Image noise reduction method, device and computer storage medium | |
| KR100869134B1 (en) | Image processing apparatus and method | |
| WO2014115453A1 (en) | Image processing device, image-capturing device, image processing method, and program | |
| US9007494B2 (en) | Image processing apparatus, method for controlling the same and storage medium | |
| KR101327790B1 (en) | Image interpolation method and apparatus | |
| CN108810320B (en) | Image quality improving method and device | |
| CN112907653B (en) | Image processing method, image pickup device and storage medium | |
| KR102189923B1 (en) | Method and apparatus for removing artifact in demosaicked image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |