CN115002297A - Image denoising method, training method, device, electronic equipment and storage medium - Google Patents
Image denoising method, training method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115002297A CN115002297A CN202110230532.4A CN202110230532A CN115002297A CN 115002297 A CN115002297 A CN 115002297A CN 202110230532 A CN202110230532 A CN 202110230532A CN 115002297 A CN115002297 A CN 115002297A
- Authority
- CN
- China
- Prior art keywords
- image
- noise reduction
- sample
- pair
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention provides an image noise reduction method, a training device, electronic equipment and a storage medium, wherein the image noise reduction method comprises the following steps: determining an image sensor corresponding to an image to be subjected to noise reduction, and determining shooting parameters of the image to be subjected to noise reduction; acquiring an image noise reduction model corresponding to the image sensor and the shooting parameters, wherein the image noise reduction model is obtained by training based on a sample image pair acquired by the image sensor, and the sample image pair comprises an original image and a synthesized image; and carrying out noise reduction processing on the image to be subjected to noise reduction through the image noise reduction model to obtain a noise-reduced image. According to the invention, because the original image in the sample image pair is an image with noise in a real scene, the image denoising model obtained through training of the sample image pair can improve the image denoising effect, and no side effect is caused.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image denoising method, a training method, an apparatus, an electronic device, and a storage medium.
Background
In the traditional noise reduction algorithm, a bilateral filtering algorithm is used, and a pre-designed filtering kernel is utilized to perform convolution operation on a noise image, so that a noise-reduced image is obtained. The noise reduction method needs to synthesize noise on a noise-free image by assuming that the appearance of the noise satisfies a gaussian distribution or a certain mathematical distribution, and then change the variance of the gaussian distribution, thereby changing the noise intensity. In practical application, the noise form which is supposed to satisfy a certain mathematical distribution cannot satisfy the real situation, so that the traditional algorithm has a common phenomenon that the noise cannot be completely reduced when applied to a practical product.
In order to achieve the purpose of noise reduction in practical application, the traditional noise reduction algorithm can perform tuning teaching of more parameters, the tuning teaching can bring certain improvement of the noise reduction effect, and meanwhile, the problem that more macroscopic images are introduced, such as smearing, graininess and the like, can be solved. The traditional noise reduction algorithm needs to be trained in a large number of later stages between the noise reduction effect and the generated side effect, the workload is large, the requirement on modeling experience workers is high, the development cost of a multi-line product is high, and in actual application, the traditional noise reduction algorithm after full training still cannot completely eliminate the negative effect brought by the algorithm.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide an image denoising method, a training method, an apparatus, an electronic device, and a storage medium that overcome or at least partially solve the above problems.
According to a first aspect of the embodiments of the present invention, there is provided an image noise reduction method, including:
determining an image sensor corresponding to an image to be denoised, and determining shooting parameters of the image to be denoised;
acquiring an image noise reduction model corresponding to the image sensor and the shooting parameters, wherein the image noise reduction model is obtained by training based on a sample image pair acquired by the image sensor, and the sample image pair comprises an original image and a synthesized image;
and carrying out noise reduction processing on the image to be subjected to noise reduction through the image noise reduction model to obtain a noise-reduced image.
According to a second aspect of the embodiments of the present invention, there is provided a training method of an image noise reduction model, including:
acquiring at least one original image and at least one synthesized image corresponding to a target image sensor from electronic equipment, and determining at least one sample image pair according to the at least one original image and the at least one synthesized image, wherein the shooting parameters of each sample image pair are the same;
and training a convolutional neural network model based on at least one sample image pair with shooting parameters meeting the same set condition to obtain an image noise reduction model corresponding to the target image sensor and the set condition.
According to a third aspect of the embodiments of the present invention, there is provided a sample image pair acquiring method, including:
closing part of functional modules in an image processor ISP, and determining a target image sensor for acquiring an original image and a synthesized image, wherein the part of functional modules comprise a sharpening module and a noise reduction module;
acquiring an original image of a target scene through the target image sensor and the image processor ISP;
under the same shooting parameters as the original image, acquiring multi-frame images of the target scene through the target image sensor and the image processor ISP, and synthesizing the multi-frame images to obtain a synthesized image;
determining the original image and the composite image as the sample image pair.
According to a fourth aspect of the embodiments of the present invention, there is provided an image noise reduction device including:
the image information determining module is used for determining an image sensor corresponding to an image to be subjected to noise reduction and determining shooting parameters of the image to be subjected to noise reduction;
a model obtaining module, configured to obtain an image noise reduction model corresponding to the image sensor and the shooting parameters, where the image noise reduction model is obtained by training a sample image pair obtained by the image sensor, and the sample image pair includes an original image and a synthesized image;
and the image denoising module is used for denoising the image to be denoised through the image denoising model to obtain a denoised image. According to a fifth aspect of the embodiments of the present invention, there is provided a training apparatus for an image noise reduction model, including:
the system comprises an image pair acquisition module, a processing module and a display module, wherein the image pair acquisition module is used for acquiring at least one original image and at least one synthesized image corresponding to a target image sensor from electronic equipment, and determining at least one sample image pair according to the at least one original image and the at least one synthesized image, wherein the shooting parameters of each sample image pair are the same;
and the model training module is used for training the convolutional neural network model based on at least one sample image pair with shooting parameters meeting the same set condition to obtain an image noise reduction model corresponding to the target image sensor and the set condition.
According to a sixth aspect of the embodiments of the present invention, there is provided a sample image pair acquiring apparatus including:
the image processing device comprises a setting module, a processing module and a processing module, wherein the setting module is used for closing partial functional modules in an image processor ISP and determining a target image sensor for acquiring an original image and a synthesized image, and the partial functional modules comprise a sharpening module and a noise reduction module;
the original image acquisition module is used for acquiring an original image of a target scene through the target image sensor and the image processor ISP;
a synthetic image obtaining module, configured to obtain, through the target image sensor and the image processor ISP, a multi-frame image of the target scene under the same shooting parameter as the original image, and synthesize the multi-frame image to obtain a synthetic image;
a sample image pair determination module to determine the original image and the composite image as the sample image pair.
According to a seventh aspect of the embodiments of the present invention, there is provided an electronic apparatus including: a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the image noise reduction method as described in the first aspect, or implementing the training method of the image noise reduction model as described in the second aspect, or implementing the sample image pair acquisition method as described in the third aspect.
According to an eighth aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image denoising method according to the first aspect, or implements the training method of the image denoising model according to the second aspect, or implements the sample image pair acquiring method according to the third aspect.
According to the image denoising method, the training method and the training device, the electronic equipment and the storage medium provided by the embodiment of the invention, the image sensor corresponding to the image to be denoised is determined, the shooting parameters of the image to be denoised are determined, the image denoising model corresponding to the image sensor and the shooting parameters is obtained, and the denoising processing is carried out on the image to be denoised through the image denoising model, so that the denoised image is obtained. Because the image noise reduction model is obtained by training based on the sample image pair acquired by the image sensor, and the original image in the sample image pair is the noise image of the real scene, the noise form does not need to be assumed to meet certain mathematical distribution, the image noise reduction effect can be improved through the image noise reduction model obtained by training through the sample image pair, and no side effect is brought.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
FIG. 1is a flowchart illustrating steps of an image denoising method according to an embodiment of the present invention;
FIG. 2a is an exemplary diagram of an image to be denoised in an embodiment of the present invention;
FIG. 2b is an exemplary diagram of an image obtained by denoising the image to be denoised shown in FIG. 2a according to the embodiment of the present invention;
FIG. 3 is a flowchart illustrating steps of a method for training an image denoising model according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating steps of a method for obtaining a sample image pair according to an embodiment of the present invention;
fig. 5 is a block diagram of an image noise reduction apparatus according to an embodiment of the present invention;
FIG. 6 is a block diagram of a training apparatus for an image denoising model according to an embodiment of the present invention;
fig. 7 is a block diagram of a sample image pair acquiring apparatus according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Fig. 1is a flowchart of steps of an image denoising method according to an embodiment of the present invention, where the method may be applied to an electronic device such as a mobile phone, a tablet computer, or a digital camera, as shown in fig. 1, the method may include:
When a camera in an electronic device such as a mobile phone includes a plurality of image sensors, different noise forms exist for each image sensor, and different image noise reduction models are respectively trained in advance for each image sensor when shooting parameters meet different setting conditions in order to better reduce noise of an image acquired by each image sensor. Therefore, when the noise of the image to be subjected to noise reduction is reduced, an appropriate image noise reduction model needs to be selected for noise reduction, which is to determine an image sensor for acquiring the image to be subjected to noise reduction and determine the shooting parameters of the image to be subjected to noise reduction so as to determine the image noise reduction model corresponding to the image sensor and the shooting parameters.
Wherein the photographing parameter includes at least one of an exposure time, a sensitivity (ISO), and an ISP parameter. ISO is the meaning of sensitivity and is an abbreviation of International Standardization Organization (International Standardization Organization), which makes quantification regulations on sensitivity. Sensitivity is a measure of the sensitivity of a film to light and is determined by sensitivity metrology and measurements of several values, and is ISO 6. For a film that is less sensitive to light, it takes longer to expose to achieve the same image as a more sensitive film, and is therefore commonly referred to as a slow film. Highly sensitive negatives are thus referred to as fast negatives.
The composite image is obtained by synthesizing a plurality of images obtained under the same shooting parameters and the same scene as the original image.
The method comprises the steps of configuring a plurality of image noise reduction models corresponding to different setting conditions which are met by an image sensor and shooting parameters in the electronic equipment, after the shooting parameters of an image to be subjected to noise reduction are determined, determining the setting conditions corresponding to the shooting parameters, obtaining the image noise reduction models corresponding to the image sensor and the determined setting conditions from the plurality of pre-configured image noise reduction models, and then performing noise reduction on the image to be subjected to noise reduction by using the image noise reduction models.
When an image noise reduction model corresponding to an image sensor and a set condition is trained, the shooting parameters of a plurality of sample image pairs used are within the set condition, and the shooting parameters of each sample image pair are the same. The sample image pair can be acquired through electronic equipment such as a mobile phone, a camera, a PAD and a computer which are going to be sold on the market formally, and the sample image pair can also be acquired through an engineering machine. For example, some finished products and semi-finished products made by manufacturers, such as mobile phones, cameras, PADs, computers, etc., which are not yet put on the market formally, may be collectively referred to as "engineering machines". The following describes an image denoising method according to an embodiment of the present application, taking an engineering machine as an example.
In some embodiments, when acquiring a sample Image pair, an electronic device such as an engineering machine is required to turn off a part of functional modules in an ISP (Image Signal Processor). Illustratively, the partial functional modules are, for example, a sharpening module and a noise reduction module, so that the sharpening module and the noise reduction module can be prevented from performing nonlinear processing to change the noise form, so as to improve the noise reduction effect of the trained image noise reduction model in a real scene. The embodiment of the present application does not limit the part of the functional modules, and those skilled in the art can determine the part of the functional modules according to actual requirements. Electronic equipment such as an engineering machine and the like acquires an original image of a target scene through an image sensor and an ISP (internet service provider), and stores the original image to a first storage position; acquiring a plurality of frames of images of a target scene under the same shooting parameters as the original image through an image sensor and an ISP (internet service provider), determining a mean image of the plurality of frames of images, taking the mean image as a composite image, and storing the composite image to a second storage position. The method includes the steps of respectively deriving original images and synthesized images from a first storage position and a second storage position of electronic equipment such as an engineering machine, forming a sample image pair by each original image and the corresponding synthesized image according to a derivation sequence to obtain a plurality of sample image pairs, and training an image noise reduction model based on the plurality of sample images to obtain an image noise reduction model corresponding to the image sensor and set conditions.
In one embodiment of the present invention, the acquiring an image noise reduction model corresponding to the image sensor and the shooting parameters includes: when the shooting parameters comprise sensitivity, determining a sensitivity range to which the sensitivity of the image to be subjected to noise reduction belongs; and acquiring an image noise reduction model corresponding to the image sensor and the sensitivity range.
The sensitivity range is a preset sensitivity range corresponding to different image noise reduction models, and each image noise reduction model is trained based on a sample image pair which is acquired by the image sensor and has sensitivity within a certain sensitivity range. Since noise increases with increasing sensitivity, in order to obtain a suitable image noise reduction model by training, the maximum sensitivity range of the electronic device may be divided into a plurality of small sensitivity ranges, that is, each sensitivity range, and for each divided sensitivity range, a corresponding image noise reduction model is trained based on a corresponding sample image pair.
When the shooting parameters comprise sensitivity, determining a sensitivity range to which the sensitivity of the image to be subjected to noise reduction belongs based on the sensitivity of the image to be subjected to noise reduction and a plurality of preset sensitivity ranges. For example, the sensitivity of the image to be noise-reduced is 1500, and the preset sensitivity ranges are 0-1000, 1000-. After a sensitivity range to which the sensitivity of the image to be subjected to noise reduction belongs is determined, an image noise reduction model corresponding to the image sensor and the sensitivity range is obtained from a plurality of image noise reduction models which are configured in advance, and then the image noise reduction model is used for reducing the noise of the image to be subjected to noise reduction.
And 103, carrying out noise reduction processing on the image to be subjected to noise reduction through the image noise reduction model to obtain a noise-reduced image.
After an image noise reduction model corresponding to an image sensor and shooting parameters is obtained, the image to be subjected to noise reduction is input into the image noise reduction model, and noise reduction processing is carried out on the image to be subjected to noise reduction through the image noise reduction model, so that a noise-reduced image is obtained.
The image noise reduction model comprises an encoder and a decoder, wherein the encoder comprises a convolution layer, a pooling layer and a deconvolution layer, and the decoder comprises a convolution layer, a pooling layer and a deconvolution layer. Each encoder and decoder are connected in combination by a convolutional layer, a pooling layer, and a deconvolution layer. After the image to be denoised is input into the image denoising model, the image to be denoised is coded by a coder to obtain a characteristic diagram, and then the characteristic diagram is decoded by a decoder to obtain the image after denoising.
Taking the image to be denoised shown in fig. 2a as an example, the image to be denoised is denoised by the image denoising model to obtain the denoised image, as shown in fig. 2b, it can be seen that the image to be denoised by the image denoising model has a good effect without introducing side effects such as smearing, graininess and the like.
The image noise reduction method provided by this embodiment obtains an image noise reduction model corresponding to the image sensor and the shooting parameters by determining the image sensor corresponding to the image to be noise reduced and determining the shooting parameters of the image to be noise reduced, and obtains the image after noise reduction by performing noise reduction processing on the image to be noise reduced through the image noise reduction model.
On the basis of the technical scheme, the method further comprises the following steps: and closing part of functional modules in the image processor ISP, wherein the part of functional modules comprise a sharpening module and a noise reduction module, and determining a target image sensor for acquiring an original image and a synthesized image.
Since the sharpening module and the noise reduction module in the image processor ISP perform nonlinear processing on the image, the form of noise can be changed, and the noise reduction effect of the trained image noise reduction model in a real scene is affected, the sharpening module and the noise reduction module in the image processor ISP need to be closed, and then the image to be subjected to noise reduction is obtained through the image sensor and the ISP, so that the noise reduction effect of the image noise reduction model can be improved.
On the basis of the above technical solution, after the denoising processing is performed on the image to be denoised by the image denoising model to obtain a denoised image, the method further includes: and sharpening the noise-reduced image.
After the noise reduction processing is carried out on the image to be subjected to noise reduction through the image noise reduction model, the image subjected to noise reduction is sharpened so as to compensate the outline of the image subjected to noise reduction, the edge and the gray level jumping part of the image subjected to noise reduction are enhanced, and the image subjected to noise reduction becomes clearer.
Fig. 3 is a flowchart of steps of a training method for an image noise reduction model according to an embodiment of the present invention, where the method may be applied to an electronic device such as a server, and as shown in fig. 3, the method may include:
The shooting parameters of each sample image pair are the same, and a composite image is obtained by combining a plurality of images obtained under the same shooting parameters and the same scene as the original image, wherein at least one sample image pair is obtained by the target image sensor in the electronic equipment. The electronic device may not be the same device as the electronic device performing the training method of the image noise reduction model, and the electronic device may be an engineering machine because the engineering machine has a large space for customizing the underlying software.
At least one raw image and at least one composite image may be captured by a target image sensor of an electronic device, and the at least one raw image and the at least one composite image corresponding to the target image sensor may be acquired from the electronic device.
In one embodiment of the present invention, the acquiring at least one original image and at least one synthesized image corresponding to the target sensor from the electronic device includes: the at least one original image is obtained from a first storage location of the electronic device and the at least one composite image is obtained from a second storage location of the electronic device.
The method comprises the steps that a sample image pair used for training an image noise reduction model is acquired through electronic equipment, a customized application program (App) can be installed in the electronic equipment, the sample image pair is automatically acquired through the application program, before the sample image pair is acquired, a node for exporting images needs to be set, namely a first storage position of an original image is set, and a second storage position of a synthesized image is set, so that the application program stores the acquired original image to the first storage position, and stores the synthesized image to the second storage position, after the application program acquires one original image, multi-frame images are acquired under the same shooting parameters, the multi-frame images are synthesized into the synthesized image, and then the next original image and the synthesized image are acquired. When the image noise reduction model is trained, at least one original image is obtained from a first storage position of the electronic device, at least one synthesized image is obtained from a second storage position of the electronic device, the original image in the first storage position and the synthesized image in the second storage position are correspondingly stored, namely the first original image in the first storage position and the first synthesized image in the second storage position are images of the same scene and the same shooting parameters, and the two images form a sample image pair, so that the obtained at least one original image and the at least one synthesized image form a plurality of sample image pairs.
In one embodiment of the invention, part of functional modules in an image processor ISP are closed, and a target image sensor for acquiring an original image and a synthesized image is determined, wherein the part of functional modules comprise a sharpening module and a noise reduction module;
determining an image of a target scene acquired by the target image sensor and the image processor ISP as the original image; and acquiring multi-frame images of the target scene under the same shooting parameters through the target image sensor and the ISP, synthesizing the multi-frame images, and determining the synthesized image as the synthesized image.
Wherein the photographing parameter includes at least one of an exposure time, a sensitivity, and an ISP parameter.
The noise generated by the image shot by the electronic equipment such as the mobile phone, the tablet personal computer, the digital camera and the like mainly comprises two parts, wherein one part is generated by the electronic circuit and the image sensor when current passes through, and the other part is generated by the sensor gain (sensor gain) of the image sensor and the ISO gain (gain) of the ISP (internet service provider) which have the amplification effect on the current in the mobile phone circuit. The two parts of noise hardly satisfy Gaussian distribution and other assumptions, the real noise distribution is more complex, and the morphological change is richer. In order to enable noise data to completely accord with the situation of a real scene, the embodiment of the invention uses a customized App to store certain specific nodes in a photographing pipeline of electronic equipment such as a mobile phone and the like to obtain a native image containing noise from the electronic equipment, so that a plurality of assumptions for modeling the noise can be avoided, inaccuracy of the assumptions often has negative effects on noise reduction, noise rules under the real scene can be completely fitted, and the noise reduction effect is better.
In some embodiments, before the sample image is acquired by the electronic device, the sharpening module and the noise reduction module in the ISP need to be turned off, so that the sharpening module and the noise reduction module can be prevented from changing the noise form, and the noise of the image noise reduction model obtained by subsequent neural network training cannot be reduced well under a real condition. There is also a need to identify a target image sensor in an electronic device that captures both raw images and composite images in order to train an image noise reduction model corresponding to the target image sensor. Illustratively, determining a target image sensor to acquire a raw image and a composite image includes: a node in the electronic device from which the image is derived, i.e. a first storage location of the original image, is set as node 1, and a second storage location of the composite image is set as node 2. Node 1is an image which is a noisy image after an image flows into an ISP through a multi-frame noise reduction algorithm, i.e. an original image, and node 2 is a composite image of a plurality of frames (e.g. 200 frames) of images of the same scene which are acquired through one or more of the ISP fixed sensitivity, ISP parameters and exposure time and have the same shooting parameters, wherein the composite image has a very low noise level and can be regarded as a clean image, i.e. a noise-free image, so that the original image derived from node 1 and the composite image derived from node 2 serve as a training image of a pair of neural networks, i.e. a sample image pair.
After the above-mentioned setting of the electronic device is completed, a pair of sample images may be acquired using a target image sensor provided in the electronic device. After the electronic device collects one or a few frames (such as 5 frames) of images through the target image sensor, the images flow into the ISP, the ISP processes the images, original data of the images collected by the target image sensor is converted into a format supported by display, when a few frames exist, multi-frame noise reduction is carried out on the few frames of images, noise still exists after multi-frame noise reduction, the images containing the noise in a real scene are determined to be the original images, the original images are stored in a first storage position, and the original images can be stored in a YUV format for processing. The electronic equipment collects multi-frame images of the same target scene as the original image under the same shooting parameters through the target image sensor, the number of the multi-frame images is larger than that of the small number of the multi-frame images, the number of the multi-frame images can be 200, for example, the multi-frame images collected by the target image sensor sequentially flow into an ISP, the ISP sequentially processes the multi-frame images, the multi-frame images are respectively converted into formats supported by display, then the electronic equipment synthesizes the multi-frame images processed by the ISP, the synthesized images are determined to be synthesized images, the synthesized images are stored in a second storage position, and the synthesized images can be stored in a YUV format for convenience of processing. After acquiring the original image and the synthesized image of one target scene under the same shooting parameters, acquiring the next pair of the original image and the synthesized image, at this time, shooting the next pair of the original image and the synthesized image of the same target scene by changing the shooting parameters, or acquiring the next pair of the original image and the synthesized image of the next target scene by using the same shooting parameters, or acquiring the next pair of the original image and the synthesized image of the next target scene after changing the shooting parameters, so that after acquiring each pair of the original image and the synthesized image, acquiring the next pair of the original image and the synthesized image in such a way until acquiring a sample image pair enough for training the image noise reduction model. When different images are collected, the brightness of the laboratory lamp box can be adjusted, and the images with different sensitivities can be collected according to a certain sensitivity rule. All shooting parameters of the original image and the synthesized image are the same, that is, the exposure time, the sensitivity and the ISP parameters of the original image and the synthesized image are the same.
In one embodiment of the invention, a mean image of the plurality of frame images is determined as the composite image.
Due to the limitation of size, an image sensor in electronic equipment such as a mobile phone cannot effectively acquire enough light entering amount in a single shooting process, so that the noise of an image shot in a single time cannot be completely avoided. The mathematical expectation of noise on each image pixel is zero, which means the same scene, if a plurality of static scene images with completely consistent scenes can be obtained under the same shooting parameters (such as one or more of exposure time, ISO and ISP parameters and the like), then the images are subjected to averaging operation, the average image obtained in the way is determined as a composite image, if the number of the shot images is enough, the mathematical expectation of noise of each independent pixel in the composite image obtained through calculation is approximately equal to zero, and from the perspective of a whole image, a very clean composite image with extremely low noise level can be approximately obtained. And taking the synthesized image as a true value image corresponding to the original image to provide a training material for the convolutional neural network model.
During the process of acquiring a sample image pair (original image-synthesized image), the shooting parameters such as exposure time, ISO, ISP parameters, etc. need to be locked, and part of the functional modules in the ISP chip, such as the sharpening module and the noise reduction module, need to be turned off. Exposure time and ISO have a direct impact on the intensity of the noise, and in some embodiments, it is ensured that the original image in a single scene and all frames used to compose the composite image have the same exposure time, sensitivity, ISP parameters, and other capture parameters during the acquisition process. Thus, the original image and the composite image can be aligned in the dimensions of brightness, color and other dimensions, and the different dimensions are just the existence of noise information.
According to the embodiment of the invention, the original image with noise in a real scene is collected, and the average image of the multi-frame image is determined as the synthetic image, so that the obtained original image and the synthetic image are both images conforming to the real scene, and noise does not need to be assumed to meet certain mathematical distribution, so that the trained image noise reduction model can have a better noise reduction effect.
Wherein the convolutional neural network model comprises an encoder and a decoder, the encoder comprises a convolutional layer, a pooling layer and a deconvolution layer, and the decoder comprises a convolutional layer, a pooling layer and a deconvolution layer.
Since noise varies with different shooting parameters, a plurality of setting conditions may be set in advance, and for each setting condition, a corresponding image noise reduction model is trained. And acquiring at least one sample image pair with shooting parameters meeting the same set condition, and training the convolutional neural network model by using the at least one sample image pair to obtain an image noise reduction model corresponding to the target image sensor and the set condition.
In an embodiment of the present invention, training a convolutional neural network model based on at least one sample image pair whose shooting parameters satisfy the same setting condition to obtain an image noise reduction model corresponding to the target image sensor and the setting condition includes:
when the shooting parameters comprise sensitivity, training a convolutional neural network model based on at least one sample image pair belonging to the same sensitivity range to obtain an image noise reduction model corresponding to the target image sensor and the sensitivity range.
The method includes the steps of classifying acquired sample images according to sensitivity ranges to obtain at least one sample image pair belonging to each sensitivity range, and training a convolutional neural network by using the at least one sample image pair belonging to the same sensitivity range to obtain an image noise reduction model corresponding to a target image sensor and the sensitivity ranges.
In one embodiment of the invention, the method comprises: the method comprises the following steps: acquiring a plurality of sample image pairs collected according to a sensitivity step length, and determining sensitivity ranges corresponding to the plurality of sample image pairs; and training the convolutional neural network model through a plurality of sample image pairs corresponding to each sensitivity range respectively to obtain an image noise reduction model corresponding to each sensitivity range.
Wherein the sensitivity of the at least one sample image pair is acquired in sensitivity steps. The sample image pair of training data needs to be collected according to a certain rule, and the noise increases with the increase of the sensitivity (ISO), so that the sensitivity step size of the collection sensitivity needs to be set, and the sensitivity is covered from zero to the maximum supported sensitivity of the electronic equipment. For example, a sample image pair may be acquired for a cell phone with an ISO maximum of 6400 using 100 as a sensitivity step, and the ISO acquisition rule is shown in table 1.
TABLE 1ISO Collection rules
The setting condition is a set sensitivity range. Since noise increases with Increasing Sensitivity (ISO), in order to avoid a situation where one model cannot completely cover all sensitivities, a maximum sensitivity range of the electronic device may be divided into a plurality of small sensitivity ranges, and a corresponding image noise reduction model is trained for each sensitivity range, where the rules for model training may be as shown in table 2, and of course, other training rules may also be adopted, for example, in order to make the model noise reduction effect better, an ISO range may be determined every 500.
TABLE 2 model training rules
| ISO range | Convolutional neural network model |
| 0-1000 | Model 1 |
| 1000-2000 | Model 2 |
| 2000-3000 | Model 3 |
| 3000-4000 | Model 4 |
| 4000-5000 | Model 5 |
| 5000-6400 | |
If the sample image pairs of the target image sensor are not acquired according to each sensitivity range, the acquired sample image pairs can be classified according to the sensitivity ranges, the sample image pairs belonging to the same sensitivity range are classified into one class, and a convolutional neural network model is trained by using a plurality of similar sample image pairs to obtain an image noise reduction model corresponding to the target image sensor and the sensitivity range.
The structure of the convolutional neural network model mainly comprises an encoder and a decoder, wherein each encoder (encoder) and decoder (decoder) are connected in a combined mode through a convolutional layer, a pooling layer and a deconvolution layer. The convolutional neural network model may be trained using a gradient descent method, using the L1 loss as a loss function of the model. The model can be better converged after more than 50 iterations, and a better noise reduction effect can be obtained when noise reduction is carried out.
According to the training method of the image noise reduction model, at least one original image and at least one synthesized image corresponding to a target image sensor are obtained from electronic equipment, at least one sample image pair is determined according to the at least one original image and the at least one synthesized image, the convolutional neural network model is trained on the basis of the at least one sample image pair with shooting parameters meeting the same set conditions, the image noise reduction model corresponding to the target image sensor and the set conditions is obtained, the original image is an image with noise in a real scene due to the fact that the shooting parameters of each sample image pair are the same, the noise form does not need to be assumed to meet certain mathematical distribution, the image noise reduction model obtained through training of the sample image pair can improve the image noise reduction effect, and side effects are not brought.
Fig. 4 is a flowchart of steps of a method for obtaining a sample image pair according to an embodiment of the present invention, where the method may be applied to an electronic device such as a mobile phone and a tablet computer, and as shown in fig. 4, the method may include:
In some embodiments, before a sample image is acquired, the sharpening module and the noise reduction module in the ISP need to be turned off, so that the sharpening module and the noise reduction module can be prevented from changing noise forms, and an image noise reduction model obtained by subsequent neural network training cannot be subjected to good noise reduction under a real condition. There is also a need to identify a target image sensor that captures both raw images and composite images in order to train an image noise reduction model corresponding to the target image sensor.
After one frame or a few frames (such as 5 frames) of images are collected by the target image sensor, the images flow into the ISP, the ISP processes the images, original data of the images collected by the target image sensor are converted into a format supported by display, when a few frames exist, multi-frame noise reduction is carried out on the few frames of images, noise still exists after multi-frame noise reduction, and the images containing the noise in the real scene are determined to be the original images.
And 403, acquiring multi-frame images of the target scene through the target image sensor and the image processor ISP under the same shooting parameters as the original image, and synthesizing the multi-frame images to obtain a synthesized image.
The method comprises the steps that a target image sensor collects multi-frame images of the same target scene as an original image under the same shooting parameters (namely exposure time, light sensitivity and ISP parameters are all the same), the number of the multi-frame images is larger than that of a few frames of images, the number of the multi-frame images can be 200, for example, the multi-frame images collected by the target image sensor sequentially flow into an ISP, the multi-frame images are sequentially processed by the ISP, the multi-frame images are respectively converted into formats supported by display, then electronic equipment synthesizes the multi-frame images processed by the ISP, namely, the mean image of the multi-frame images is calculated, and the mean image is determined to be a synthesized image. When the number of images taken is sufficiently large (e.g., the number of images is greater than 100), the noise of each individual pixel in the computed composite image is expected to be approximately equal to zero, and a very clean composite image with extremely low noise level can be obtained approximately from the perspective of the whole image.
Because the noise level of the synthetic image is extremely low, the original image and the synthetic image which are acquired under the same shooting parameter and the same target scene are determined as a sample image pair, and a training material can be provided for the convolutional neural network model.
In the method for obtaining a sample image pair provided by this embodiment, a part of function modules in an ISP is turned off, a target image sensor for acquiring an original image and a synthesized image is determined, the original image of a target scene is obtained through the target image sensor and the ISP, a plurality of frames of images of the target scene are obtained through the target image sensor and the ISP under the same shooting parameters as the original image, the plurality of frames of images are synthesized to obtain a synthesized image, and the original image and the synthesized image are determined as the sample image pair. Because the image pairs of the samples are collected after part of the function modules in the ISP are closed, the phenomenon that the noise forms of the part of the function modules in the ISP are changed so that the model cannot be subjected to good noise reduction under a real condition can be avoided; the original image is an image containing noise in a real scene, the noise form does not need to be assumed to meet certain mathematical distribution, the image noise reduction effect can be improved through the image noise reduction model obtained through training of the sample image, and no side effect is caused.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Fig. 5 is a block diagram of a structure of an image noise reduction apparatus according to an embodiment of the present invention, and as shown in fig. 5, the image noise reduction apparatus may include:
an image information determining module 501, configured to determine an image sensor corresponding to an image to be noise-reduced, and determine a shooting parameter of the image to be noise-reduced;
a model obtaining module 502, configured to obtain an image noise reduction model corresponding to the image sensor and the shooting parameters, where the image noise reduction model is obtained by training based on a sample image pair obtained by the image sensor, and the sample image pair includes an original image and a synthesized image;
and an image denoising module 503, configured to perform denoising processing on the image to be denoised through the image denoising model, so as to obtain a denoised image.
Optionally, the image noise reduction model includes an encoder and a decoder, the encoder includes a convolutional layer, a pooling layer, and a deconvolution layer, and the decoder includes a convolutional layer, a pooling layer, and a deconvolution layer.
Optionally, the shooting parameters include at least one of exposure time, sensitivity, and ISP parameters.
Optionally, the model obtaining module includes:
a sensitivity range determination unit configured to determine a sensitivity range to which sensitivity of the image to be noise-reduced belongs when the shooting parameter includes sensitivity;
a model acquisition unit configured to acquire an image noise reduction model corresponding to the image sensor and the sensitivity range.
Optionally, the composite image is obtained by synthesizing a plurality of images obtained under the same shooting parameters and the same scene as the original image.
Optionally, the apparatus further comprises:
and the ISP function closing module is used for closing part of functional modules in the image processor ISP and determining a target image sensor for acquiring an original image and a synthesized image, and the part of functional modules comprise a sharpening module and a noise reduction module.
Optionally, the apparatus further comprises:
and the sharpening processing module is used for carrying out noise reduction processing on the image to be subjected to noise reduction through the image noise reduction model to obtain a noise-reduced image and then sharpening the noise-reduced image.
The image noise reduction device provided by this embodiment determines, through the image information determination module, an image sensor corresponding to an image to be noise reduced, and determines a shooting parameter of the image to be noise reduced, the model acquisition module acquires an image noise reduction model corresponding to the image sensor and the shooting parameter, and the image noise reduction module performs noise reduction on the image to be noise reduced through the image noise reduction model to obtain a noise-reduced image.
Fig. 6 is a block diagram of a structure of a training apparatus for an image noise reduction model according to an embodiment of the present invention, and as shown in fig. 6, the training apparatus for an image noise reduction model may include:
an image pair obtaining module 601, configured to obtain at least one original image and at least one synthesized image corresponding to a target image sensor from an electronic device, and determine at least one sample image pair according to the at least one original image and the at least one synthesized image, where shooting parameters of each sample image pair are the same;
a model training module 602, configured to train a convolutional neural network model based on at least one sample image pair whose shooting parameters satisfy the same setting condition, so as to obtain an image noise reduction model corresponding to the target image sensor and the setting condition.
Optionally, the image pair obtaining module includes:
an image acquisition unit configured to acquire the at least one original image from a first storage location of the electronic device and acquire the at least one composite image from a second storage location of the electronic device.
Optionally, closing part of functional modules in the image processor ISP, and determining a target image sensor for acquiring an original image and a synthesized image, where the part of functional modules include a sharpening module and a noise reduction module;
determining an image of a target scene acquired by the target image sensor and the image processor ISP as the original image; acquiring multi-frame images of the target scene under the same shooting parameters through the target image sensor and the image processor ISP, synthesizing the multi-frame images, and determining the synthesized image as the synthesized image.
Optionally, the mean image of the multiple frames of images is determined as the composite image.
Optionally, the shooting parameter includes at least one of an exposure time, a sensitivity, and an ISP parameter.
Optionally, the model training module is specifically configured to:
and when the shooting parameters comprise sensitivity, training a convolutional neural network model based on at least one sample image pair belonging to the same sensitivity range to obtain an image noise reduction model corresponding to the target image sensor and the sensitivity range.
Optionally, the apparatus includes:
the system comprises a sensitivity range determining module, a sensitivity range determining module and a sensitivity range determining module, wherein the sensitivity range determining module is used for acquiring a plurality of sample image pairs collected according to a sensitivity step length and determining the sensitivity ranges corresponding to the sample image pairs;
and the light sensitivity corresponding model training module is used for training the convolutional neural network model through a plurality of sample image pairs corresponding to each light sensitivity range respectively to obtain an image noise reduction model corresponding to each light sensitivity range.
Optionally, the convolutional neural network model includes an encoder and a decoder, the encoder includes a convolutional layer, a pooling layer and a deconvolution layer, and the decoder includes a convolutional layer, a pooling layer and a deconvolution layer.
The training device of the image noise reduction model provided by this embodiment acquires at least one original image and at least one synthesized image corresponding to the target image sensor from the electronic device through the image pair acquisition module, determining at least one sample image pair from the at least one original image and the at least one synthesized image, the model training module based on the at least one sample image pair with the shooting parameters satisfying the same set condition, training the convolution neural network model to obtain an image noise reduction model corresponding to the target image sensor and the set conditions, because the shooting parameters of each sample image pair are the same, the original image is an image of a real scene containing noise, the noise form does not need to be assumed to meet certain mathematical distribution, the image denoising model obtained through training of the sample image can improve the image denoising effect, and side effects cannot be brought.
Fig. 7 is a block diagram of a sample image pair acquiring apparatus according to an embodiment of the present invention, and as shown in fig. 7, the sample image pair acquiring apparatus may include:
a setting module 701, configured to close part of functional modules in an image processor ISP, and determine a target image sensor for acquiring an original image and a synthesized image, where the part of functional modules include a sharpening module and a noise reduction module;
an original image acquisition module 702, configured to acquire an original image of a target scene through the target image sensor and the image processor ISP;
a synthesized image obtaining module 703, configured to obtain, through the target image sensor and the image processor ISP, a multi-frame image of the target scene under the same shooting parameters as the original image, and synthesize the multi-frame image to obtain a synthesized image;
a sample image pair determining module 704 for determining the original image and the composite image as the sample image pair.
The sample image pair obtaining apparatus provided in this embodiment closes a part of function modules in an ISP through a setting module, and determines a target image sensor for collecting an original image and a synthesized image, where the original image obtaining module obtains the original image of a target scene through the target image sensor and the ISP, and the synthesized image obtaining module obtains multi-frame images of the target scene through the target image sensor and the ISP under the same shooting parameters as the original images, and synthesizes the multi-frame images to obtain a synthesized image, and the sample image pair determining module determines the original image and the synthesized image as a sample image pair, and collects the sample image pair after closing a part of function modules in the ISP, so that it can be avoided that the noise form of the part of function modules in the ISP changes so that a model cannot be subjected to good noise reduction under a real condition, and the original image is an image with noise in a real scene, the noise form does not need to be assumed to meet certain mathematical distribution, the image noise reduction effect can be improved through the image noise reduction model obtained through training of the sample image, and side effects are not brought.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
Further, according to an embodiment of the present invention, there is provided an electronic apparatus including: a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the image noise reduction method or the training method of the image noise reduction model of the foregoing embodiments.
According to an embodiment of the present invention, there is also provided a computer readable storage medium including, but not limited to, a disk memory, a CD-ROM, an optical memory, etc., on which a computer program is stored, which when executed by a processor implements the image noise reduction method or the training method of the image noise reduction model of the foregoing embodiments.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one of skill in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the true scope of the embodiments of the present invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The image denoising method, the training device, the electronic device and the storage medium provided by the invention are introduced in detail, and a specific example is applied in the text to explain the principle and the implementation of the invention, and the description of the embodiment is only used to help understanding the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (21)
1. An image noise reduction method, comprising:
determining an image sensor corresponding to an image to be denoised, and determining shooting parameters of the image to be denoised;
acquiring an image noise reduction model corresponding to the image sensor and the shooting parameters, wherein the image noise reduction model is obtained by training based on a sample image pair acquired by the image sensor, and the sample image pair comprises an original image and a synthesized image;
and carrying out noise reduction processing on the image to be subjected to noise reduction through the image noise reduction model to obtain a noise-reduced image.
2. The method of claim 1, wherein the image noise reduction model comprises an encoder and a decoder, the encoder comprising a convolutional layer, a pooling layer, and a deconvolution layer, the decoder comprising a convolutional layer, a pooling layer, and a deconvolution layer.
3. The method according to claim 1 or 2, wherein the shooting parameters include at least one of exposure time, sensitivity, and ISP parameters.
4. The method of claim 3, wherein the obtaining an image noise reduction model corresponding to the image sensor and the capture parameters comprises:
when the shooting parameters comprise sensitivity, determining a sensitivity range to which the sensitivity of the image to be subjected to noise reduction belongs;
and acquiring an image noise reduction model corresponding to the image sensor and the sensitivity range.
5. The method according to any one of claims 1 to 4, wherein the composite image is obtained by synthesizing a plurality of images obtained under the same shooting parameters and the same scene as the original image.
6. The method of any one of claims 1-5, further comprising:
and closing part of functional modules in the image processor ISP, and acquiring the image to be denoised through the image sensor and the image processor ISP, wherein the part of functional modules comprise a sharpening module and a denoising module.
7. The method according to claim 6, after the performing noise reduction processing on the image to be noise-reduced by the image noise reduction model to obtain a noise-reduced image, further comprising:
and sharpening the noise-reduced image.
8. A training method of an image noise reduction model is characterized by comprising the following steps:
acquiring at least one original image and at least one synthesized image corresponding to a target image sensor from electronic equipment, and determining at least one sample image pair according to the at least one original image and the at least one synthesized image, wherein the shooting parameters of each sample image pair are the same;
and training a convolutional neural network model based on at least one sample image pair of which the shooting parameters meet the same set condition to obtain an image noise reduction model corresponding to the target image sensor and the set condition.
9. The method of claim 8, wherein obtaining at least one raw image and at least one composite image corresponding to a target sensor from an electronic device comprises:
the at least one original image is retrieved from a first storage location of the electronic device and the at least one composite image is retrieved from a second storage location of the electronic device.
10. The method according to claim 8 or 9, characterized in that:
closing part of functional modules in an image processor ISP of the electronic equipment, and determining a target image sensor for acquiring an original image and a synthesized image, wherein the part of functional modules comprise a sharpening module and a noise reduction module;
determining an image of a target scene acquired by the target image sensor and the image processor ISP as the original image; acquiring multi-frame images of the target scene under the same shooting parameters through the target image sensor and the image processor ISP, synthesizing the multi-frame images, and determining the synthesized image as the synthesized image.
11. The method of claim 10, wherein:
and determining the average image of the plurality of frames of images as the composite image.
12. The method according to any one of claims 8 to 11, wherein the shooting parameters include at least one of exposure time, sensitivity, and ISP parameters.
13. The method according to any one of claims 8 to 12, wherein training a convolutional neural network model based on at least one sample image pair whose shooting parameters satisfy the same set condition to obtain an image noise reduction model corresponding to the target image sensor and the set condition comprises:
when the shooting parameters comprise sensitivity, training a convolutional neural network model based on at least one sample image pair belonging to the same sensitivity range to obtain an image noise reduction model corresponding to the target image sensor and the sensitivity range.
14. The method according to any one of claims 8-13, comprising: acquiring a plurality of sample image pairs collected according to a sensitivity step length, and determining sensitivity ranges corresponding to the plurality of sample image pairs;
and training the convolutional neural network model through a plurality of sample image pairs corresponding to each sensitivity range respectively to obtain an image noise reduction model corresponding to each sensitivity range.
15. The method of any one of claims 8-14, wherein the convolutional neural network model comprises an encoder and a decoder, the encoder comprising a convolutional layer, a pooling layer, and a deconvolution layer, the decoder comprising a convolutional layer, a pooling layer, and an deconvolution layer.
16. A method for obtaining a sample image pair, comprising:
closing part of functional modules in an image processor ISP, and determining a target image sensor for acquiring an original image and a synthesized image, wherein the part of functional modules comprise a sharpening module and a noise reduction module;
acquiring an original image of a target scene through the target image sensor and the image processor ISP;
under the same shooting parameters as the original image, acquiring multi-frame images of the target scene through the target image sensor and the image processor ISP, and synthesizing the multi-frame images to obtain a synthesized image;
determining the original image and the composite image as the sample image pair.
17. An image noise reduction apparatus, comprising:
the image information determining module is used for determining an image sensor corresponding to an image to be subjected to noise reduction and determining shooting parameters of the image to be subjected to noise reduction;
a model obtaining module, configured to obtain an image noise reduction model corresponding to the image sensor and the shooting parameters, where the image noise reduction model is obtained by training a sample image pair obtained by the image sensor, and the sample image pair includes an original image and a synthesized image;
and the image denoising module is used for denoising the image to be denoised through the image denoising model to obtain a denoised image.
18. An apparatus for training an image noise reduction model, comprising:
the image pair acquisition module is used for acquiring at least one original image and at least one synthesized image corresponding to a target image sensor from the electronic equipment, and determining at least one sample image pair according to the at least one original image and the at least one synthesized image, wherein the shooting parameters of each sample image pair are the same;
and the model training module is used for training the convolutional neural network model based on at least one sample image pair with shooting parameters meeting the same set condition to obtain an image noise reduction model corresponding to the target image sensor and the set condition.
19. A sample image pair acquisition device, comprising:
the image processing device comprises a setting module, a processing module and a processing module, wherein the setting module is used for closing partial functional modules in an image processor ISP and determining a target image sensor for acquiring an original image and a synthesized image, and the partial functional modules comprise a sharpening module and a noise reduction module;
the original image acquisition module is used for acquiring an original image of a target scene through the target image sensor and the image processor ISP;
a synthetic image obtaining module, configured to obtain, through the target image sensor and the image processor ISP, a multi-frame image of the target scene under the same shooting parameter as the original image, and synthesize the multi-frame image to obtain a synthetic image;
a sample image pair determination module to determine the original image and the composite image as the sample image pair.
20. An electronic device, comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the image noise reduction method according to any one of claims 1 to 7, or implementing the training method of the image noise reduction model according to any one of claims 8 to 15, or implementing the sample image pair acquisition method according to claim 16.
21. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, implements the image denoising method according to any one of claims 1 to 7, or implements the training method of the image denoising model according to any one of claims 8 to 15, or implements the sample image pair acquisition method according to claim 16.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110230532.4A CN115002297A (en) | 2021-03-02 | 2021-03-02 | Image denoising method, training method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110230532.4A CN115002297A (en) | 2021-03-02 | 2021-03-02 | Image denoising method, training method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115002297A true CN115002297A (en) | 2022-09-02 |
Family
ID=83018769
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110230532.4A Pending CN115002297A (en) | 2021-03-02 | 2021-03-02 | Image denoising method, training method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115002297A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117934574A (en) * | 2024-03-22 | 2024-04-26 | 深圳市智兴盛电子有限公司 | Method, device, equipment and storage medium for optimizing image of automobile data recorder |
-
2021
- 2021-03-02 CN CN202110230532.4A patent/CN115002297A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117934574A (en) * | 2024-03-22 | 2024-04-26 | 深圳市智兴盛电子有限公司 | Method, device, equipment and storage medium for optimizing image of automobile data recorder |
| CN117934574B (en) * | 2024-03-22 | 2024-07-12 | 深圳市智兴盛电子有限公司 | Method, device, equipment and storage medium for optimizing image of automobile data recorder |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11127117B2 (en) | Information processing method, information processing apparatus, and recording medium | |
| CN109712102B (en) | Image fusion method and device and image acquisition equipment | |
| CN107408296B (en) | Real-time noise for high dynamic range images is eliminated and the method and system of image enhancement | |
| US7343040B2 (en) | Method and system for modifying a digital image taking into account it's noise | |
| Li et al. | Detail-preserving multi-exposure fusion with edge-preserving structural patch decomposition | |
| CN113992861B (en) | Image processing method and image processing device | |
| KR20190143785A (en) | Image processing method and apparatus, and electronic device | |
| CN108090886B (en) | High dynamic range infrared image display and detail enhancement method | |
| JP7212554B2 (en) | Information processing method, information processing device, and program | |
| CN108810418A (en) | Image processing method, device, mobile terminal and computer readable storage medium | |
| CN111598799A (en) | Image toning enhancement method and image toning enhancement neural network training method | |
| US20230056564A1 (en) | Image authenticity detection method and apparatus | |
| CN109711241B (en) | Object detection method and device and electronic equipment | |
| CN108897786A (en) | Recommended method, device, storage medium and the mobile terminal of application program | |
| Rahman et al. | Efficient image enhancement model for correcting uneven illumination images | |
| WO2023125750A1 (en) | Image denoising method and apparatus, and storage medium | |
| CN109934789A (en) | Image de-noising method, device and electronic equipment | |
| EP3913572A1 (en) | Loss function for image reconstruction | |
| US8055093B2 (en) | Method and system for noise level detection in image data | |
| Rabie | Adaptive hybrid mean and median filtering of high-ISO long-exposure sensor noise for digital photography | |
| CN111325671B (en) | Network training method and device, image processing method and electronic equipment | |
| CN115002297A (en) | Image denoising method, training method, device, electronic equipment and storage medium | |
| CN117152182B (en) | Ultralow-illumination network camera image processing method and device and electronic equipment | |
| Lee et al. | Image enhancement approach using the just-noticeable-difference model of the human visual system | |
| CN112839167A (en) | Image processing method, image processing device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |