CN111476741B - Image denoising method, image denoising device, electronic equipment and computer readable medium - Google Patents
Image denoising method, image denoising device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN111476741B CN111476741B CN202010353247.7A CN202010353247A CN111476741B CN 111476741 B CN111476741 B CN 111476741B CN 202010353247 A CN202010353247 A CN 202010353247A CN 111476741 B CN111476741 B CN 111476741B
- Authority
- CN
- China
- Prior art keywords
- image
- denoising
- target image
- processed
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000004458 analytical method Methods 0.000 claims abstract description 94
- 238000012545 processing Methods 0.000 claims abstract description 56
- 230000011218 segmentation Effects 0.000 claims description 30
- 230000008569 process Effects 0.000 claims description 27
- 238000010586 diagram Methods 0.000 claims description 24
- 230000015654 memory Effects 0.000 claims description 14
- 238000013135 deep learning Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 description 8
- 230000001815 facial effect Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000000717 retained effect Effects 0.000 description 3
- 210000004709 eyebrow Anatomy 0.000 description 2
- 210000004209 hair Anatomy 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 235000012434 pretzels Nutrition 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention provides a denoising method, a denoising device, electronic equipment and a computer readable medium for images, which relate to the technical field of image processing and comprise the steps of acquiring images to be processed; performing first denoising processing on the image to be processed to obtain a first target image, wherein the first denoising processing means removing significant noise in the image to be processed; carrying out face analysis on the first target image to obtain a face analysis result; and carrying out second denoising on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and apparatus for denoising an image, an electronic device, and a computer readable medium.
Background
With the development of smartphones, more and more photos are taken and transmitted, and thus, the image denoising technology is also becoming more important. Denoising has been mainly performed in the past by conventional image processing methods, such as bilateral filtering, non-Local means, and the like. With the rise of deep learning technology in recent years, the graph denoising effect based on the deep learning method is significantly superior to that of the traditional method. However, when the noise of the image is too strong, it is still difficult for the deep learning image denoising technique to completely remove the noise in the image, and image details are vulnerable to be lost in the denoising process.
Disclosure of Invention
Accordingly, an object of the present invention is to provide a method, an apparatus, an electronic device, and a computer readable medium for denoising an image, so as to alleviate the technical problem of poor denoising effect when denoising an image by using the existing image denoising method.
In a first aspect, an embodiment of the present invention provides a denoising method for an image, including: acquiring an image to be processed; performing first denoising processing on the image to be processed to obtain a first target image, wherein the first denoising processing means removing significant noise in the image to be processed; carrying out face analysis on the first target image to obtain a face analysis result; and carrying out second denoising processing on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising.
Further, performing a first denoising process on the image to be processed to obtain a first target image, where the first denoising process includes: and carrying out first denoising processing on the image to be processed by using a coarse denoising network to obtain the first target image.
Further, the coarse denoising network includes: a Unet network.
Further, performing face analysis on the first target image to obtain a face analysis result, where the face analysis result includes: and carrying out segmentation processing on the first target image by using a deep learning semantic segmentation network to obtain at least one face segmentation result, and determining the at least one face segmentation result as the face analysis result.
Further, the deep learning semantic segmentation network is a deeplab series network.
Further, performing face analysis on the first target image to obtain a face analysis result, where the face analysis result includes: processing the first target image by using a key point extraction network to obtain a face key point; generating a face key point thermodynamic diagram of the image to be processed based on the face key points, and determining the face key point thermodynamic diagram as the face analysis result.
Further, performing a second denoising process on the first target image pair by using the face analysis result, to obtain a second target image includes: and processing the face analysis result and the first target image by using a fine denoising network, and outputting to obtain the second target image.
Further, the fine denoising network includes: a multi-layered stacked residual network.
In a second aspect, an embodiment of the present invention provides an image denoising apparatus, including: an acquisition unit configured to acquire an image to be processed; the first denoising unit is used for performing first denoising processing on the image to be processed to obtain a first target image, wherein the first denoising processing means removing significant noise in the image to be processed; the face analysis unit is used for carrying out face analysis on the first target image to obtain a face analysis result; and the second denoising unit is used for performing second denoising processing on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any one of the first aspects when the computer program is executed.
In a fourth aspect, embodiments of the present invention provide a computer readable medium having non-volatile program code executable by a processor, the program code causing the processor to perform the steps of the method of any of the first aspects above.
In the embodiment of the invention, firstly, an image to be processed is acquired; performing first denoising treatment on the image to be processed to obtain a first target image; carrying out face analysis on the first target image to obtain a face analysis result; and finally, carrying out second denoising processing on the first target image by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising. In this embodiment, when the face analysis result is used to perform denoising processing on the first target image, the face component in the image to be processed can be protected according to the face analysis result, so that the face component is not removed in the process of performing denoising processing, and therefore the original face details are better retained while powerful noise is removed, and further the technical problem that the denoising effect is poor when the existing image denoising method is used to perform denoising processing on the image is solved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural view of an electronic device according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of denoising an image according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a coarse denoising network according to an embodiment of the present invention;
FIG. 4 is a schematic illustration of a face segmentation image according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a fine denoising network according to an embodiment of the present invention;
FIG. 6 is a schematic illustration of an image to be processed according to an embodiment of the invention;
FIG. 7 is a schematic diagram of an output image of a coarse denoising network according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an output image of a fine denoising network according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a facial key point thermodynamic diagram in accordance with an embodiment of the present invention;
fig. 10 is a schematic diagram of an image denoising apparatus according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1:
first, an electronic device 100 for implementing an embodiment of the present invention, which can be used to run the denoising method of an image of each embodiment of the present invention, will be described with reference to fig. 1.
As shown in fig. 1, electronic device 100 includes one or more processors 102, one or more memories 104, an input device 106, an output device 108, and an image capture device 110, which are interconnected by a bus system 112 and/or other forms of connection mechanisms (not shown). It should be noted that the components and structures of the electronic device 100 shown in fig. 1 are exemplary only and not limiting, as the electronic device may have other components and structures as desired.
The processor 102 may be implemented in hardware in at least one of a digital signal processor (DSP, digital Signal Processing), field programmable gate array (FPGA, field-Programmable Gate Array), programmable logic array (PLA, programmable Logic Array) and ASIC (Application Specific Integrated Circuit), and the processor 102 may be a central processing unit (CPU, central Processing Unit) or other form of processing unit having data processing and/or instruction execution capabilities and may control other components in the electronic device 100 to perform desired functions.
The memory 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that can be executed by the processor 102 to implement client functions and/or other desired functions in embodiments of the present invention as described below. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
The image acquisition device 110 is configured to acquire an image to be identified, where data acquired by a camera is denoised by a denoising method of the image to obtain a denoised image.
According to an embodiment of the present invention, there is provided an embodiment of a denoising method of an image, it being noted that the steps shown in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that herein.
Fig. 2 is a flowchart of a denoising method of an image according to an embodiment of the present invention, as shown in fig. 2, the method comprising the steps of:
step S202, a to-be-processed image is acquired.
In this embodiment, the image to be processed is an image including a face, and the image to be processed may be an image acquired by a camera device of a mobile terminal, for example, an image acquired by a camera device of a smart phone.
Step S204, performing a first denoising process on the image to be processed to obtain a first target image, where the first denoising process represents removing significant noise in the image to be processed.
In this embodiment, after the image to be processed is obtained, significant noise in the image to be processed may be removed, so as to implement a first denoising process of the image to be processed, and obtain a first target image.
In this application, the significant noise is determined according to the display degree of the noise in the image to be processed. For example, noise in the image to be processed, which is displayed to a degree higher than a certain threshold value, is determined as significant noise. For another example, a plurality of types of image noise may be determined according to the display degree of noise in the image to be processed, for example, gaussian noise and impulse noise are determined, and then gaussian noise and impulse noise are determined as significant noise.
Step S206, carrying out face analysis on the first target image to obtain a face analysis result.
In this embodiment, after the first denoising process is performed on the image to be processed to obtain the first target image, face analysis may be performed on the first target image, and a face analysis result may be obtained by the face analysis. The face analysis result may be a face segmentation result, which may be a result map as shown in fig. 4, or may be a face key point thermodynamic diagram, as shown in fig. 9, which is not specifically limited in this application.
In this application, the facial key point thermodynamic diagram is a thermodynamic diagram generated based on the facial key points, for example, each region surrounded by the facial key points may be determined, and then a corresponding thermodynamic diagram is generated based on each region surrounded by the facial key points, so as to obtain the facial key point thermodynamic diagram shown in fig. 9.
And step S208, performing second denoising processing on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising.
In this embodiment, after the face analysis result is obtained, the face analysis result may be used to perform the second denoising process on the first target image, so as to obtain the second target image. The second target image obtained after fine denoising is recorded as an image to be processed after denoising. By adopting the denoising mode, noise can be removed strongly, and the details of the original face can be kept well.
In this application, after the second denoising process is performed on the first target image pair, fine noise in the first target image may be removed, where the fine noise may be noise other than significant noise in the image to be processed. The fine noise can also be understood as: noise whose display degree is smaller than a certain threshold value in the image to be processed is determined as fine noise.
In the embodiment of the invention, firstly, an image to be processed is acquired; performing first denoising treatment on the image to be processed to obtain a first target image; carrying out face analysis on the first target image to obtain a face analysis result; and finally, carrying out second denoising processing on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising. In this embodiment, when the face analysis result is used to perform denoising processing on the first target image, the face component in the image to be processed can be protected according to the face analysis result, so that the face component is not removed in the process of performing denoising processing, and therefore the original face details are better retained while powerful noise is removed, and further the technical problem that the denoising effect is poor when the existing image denoising method is used to perform denoising processing on the image is solved.
In the present embodiment, the method described in the above steps S202 to S208 may be performed by one denoising network. Wherein, this denoising network includes: a coarse denoising network, a portrait parsing network and a fine denoising network. At this time, the method described in the above steps S202 to S208 may be described as:
firstly, acquiring an image to be processed; performing first denoising processing on the image to be processed through a denoising network to obtain a first target image; then, carrying out face analysis on the first target image through a face analysis network to obtain a face analysis result; and finally, performing second denoising processing on the first target image pair by using the face analysis result by the fine denoising network to obtain a second target image, and determining the second target image as a to-be-processed image after denoising. The above procedure will be described with reference to specific embodiments.
In an optional embodiment, the step S204 of performing the first denoising process on the image to be processed to obtain the first target image may be described as:
and carrying out first denoising processing on the image to be processed by using a coarse denoising network to obtain the first target image. Optionally, the coarse denoising network includes: a Unet network.
In this embodiment, the purpose of the coarse denoising network is to remove significant noise in the image to be processed, where the coarse denoising network may be a uiet structure. The network shown in fig. 3 is a network with a uiet structure. As shown in fig. 3, the whole network of the uiet is U-shaped, and represents a feature map of image information, in which the size is gradually reduced and then gradually restored to the original size.
The Unet structure can scale the feature map to different sizes, so that the large receptive field is possessed, and significant noise in the map can be effectively removed.
In this embodiment, after the first target image is obtained, face analysis may be performed on the first target image to obtain a face analysis result. In this embodiment, face analysis may be performed on the first target image in the following two ways.
Mode one: the face analysis result is a face segmentation map
And carrying out segmentation processing on the first target image by using a deep learning semantic segmentation network to obtain at least one face segmentation result, and determining the at least one face segmentation result as the face analysis result.
Specifically, in the present embodiment, the portrait parsing is actually a semantic segmentation task, so that the portrait parsing can be implemented using a deep learning semantic segmentation network (i.e., a portrait parsing network), such as a hole convolution deeplab series network. Wherein, the deeplab series network includes: deep v1 and deep v2, and deep v2 networks. Deep v1 is a network modified on the basis of VGG16, a specific modification can be described as: converting the full connection layer of the deep convolutional neural network VGG16 into convolution; the last two pooling layers of VGG16 are removed and then hole convolution is used. Deep v2 is an improvement over v1, and specific modifications can be described as: a multi-scale model was used in deep v2 to obtain better segmentation effect (e.g., spatial pyramid pooling Atous Spatial Pyramid Pooling (ASPP) was used); in deep v2, the base layer is converted from VGG16 to ResNet; deep v2 uses different learning strategies.
As shown in fig. 4, the at least one face segmentation result is at least one face segmentation result, as can be seen from fig. 4, including: the result of dividing the eyebrows, the eyes, the nose, the mouth, the chin, the hair and the whole face in the face.
In this embodiment, the deep learning semantic segmentation network may be trained on the face analysis data set; and loading the deep learning semantic segmentation network trained before into the denoising network when the denoising network is trained, and performing fine adjustment in the denoising network training process.
Mode two: the face analysis result is a face key point thermodynamic diagram
Firstly, processing the first target image by using a key point extraction network to obtain a face key point.
In this application, the facial keypoints may be the keypoints of the facial features, for example, the keypoints used to characterize the contours of the features. For example, the key points of each eye contour, the nose contour, the eyebrow contour, the mouth contour, and the hair contour may be described.
And then, generating a face key point thermodynamic diagram of the image to be processed based on the face key points, and determining the face key point thermodynamic diagram as the face analysis result.
In this embodiment, the first target image may be processed by using the key point extraction network to obtain the face key point, and then, a face key point thermodynamic diagram of the image to be processed is generated based on the face key point.
In this embodiment, the key point extraction network may be trained on the face analysis data set; and when the denoising network is trained, loading the previously trained key point extraction network into the denoising network, and performing fine adjustment in the denoising network training process.
In this embodiment, after the face analysis result and the first target image are obtained according to the above-described process, the second denoising process may be performed on the first target image pair by using the face analysis result to obtain a second target image, which specifically includes the following steps:
processing the face analysis result and the first target image by using a fine denoising network, and outputting to obtain the second target image, wherein the fine denoising network optionally comprises: a multi-layered stacked residual network.
Specifically, in the present embodiment, the fine denoising network uses the results of coarse denoising and face analysis (the first target image and the face analysis result) as input, and performs the second denoising processing on the first target image through the face analysis result, to obtain the second target image. The network structure of the fine denoising network may be a multi-layer stacked residual block. In this application, the fine denoising network may be a network such as SRResnet, EDSR, WDSR. For example, the network structure is the SRResnet network as shown in fig. 5.
In the fine denoising network, the size of the feature map is kept unchanged all the time, so that the fine denoising network is focused on fine improvement of image details.
In this embodiment, the mode of denoising the first target image by combining the face analysis result is adopted, so that the original face details can be better reserved while strong noise is removed, and further the technical problem that the denoising effect is poor when the existing image denoising method is adopted to denoise the image is solved.
The denoising method of the above image is described below with reference to fig. 6 to 9. In the present application, an image to be processed is first obtained, where the image to be processed is shown in fig. 6. After the image to be processed is acquired, coarse denoising processing is performed on the image to be processed through a uiet network as shown in fig. 3, so that significant noise in the image to be processed is removed, for example, gaussian noise, pretzel noise and the like in the image to be processed are removed, and obvious noise is displayed in the image to be processed, which is the image (i.e., the first target image) after the coarse denoising processing as shown in fig. 7. Next, face analysis is performed on the image shown in fig. 7 to obtain a face analysis result, where the face analysis result may be a face segmentation result shown in fig. 4 or a face key point thermodynamic diagram shown in fig. 9. After the face analysis result is obtained, the image shown in fig. 7 may be subjected to a second denoising process (i.e., a fine denoising process) using the face analysis result, to obtain a fine denoising result (i.e., a second target image) shown in fig. 8.
Example 3:
the embodiment of the invention also provides a denoising device for the image, which is mainly used for executing the denoising method for the image provided by the embodiment of the invention, and the denoising device for the image provided by the embodiment of the invention is specifically described below.
Fig. 10 is a schematic diagram of an image denoising apparatus according to an embodiment of the present invention, as shown in fig. 10, the image denoising apparatus mainly includes an acquisition unit 10, a first denoising unit 20, a face analysis unit 30, and a second denoising unit 40, wherein:
an acquisition unit 10 for acquiring an image to be processed;
a first denoising unit 20, configured to perform a first denoising process on the to-be-processed image to obtain a first target image, where the first denoising process represents removing significant noise in the to-be-processed image;
a face analysis unit 30, configured to perform face analysis on the first target image to obtain a face analysis result;
and a second denoising unit 40, configured to perform a second denoising process on the first target image pair by using the face analysis result, obtain a second target image, and determine the second target image as a denoising image to be processed.
In the embodiment of the invention, firstly, an image to be processed is acquired; performing first denoising treatment on the image to be processed to obtain a first target image; carrying out face analysis on the first target image to obtain a face analysis result; and finally, carrying out second denoising processing on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a to-be-processed image after denoising. In this embodiment, when the face analysis result is used to perform denoising processing on the first target image, the face component in the image to be processed can be protected according to the face analysis result, so that the face component is not removed in the process of performing denoising processing, and therefore the original face details are better retained while powerful noise is removed, and further the technical problem that the denoising effect is poor when the existing image denoising method is used to perform denoising processing on the image is solved.
Optionally, the first denoising unit is configured to: and carrying out first denoising processing on the image to be processed by using a coarse denoising network to obtain the first target image.
Optionally, the coarse denoising network includes: a Unet network.
Optionally, the face parsing unit is configured to: and carrying out segmentation processing on the first target image by using a deep learning semantic segmentation network to obtain at least one face segmentation result, and determining the at least one face segmentation result as the face analysis result.
Optionally, the deep learning semantic segmentation network is a deeplab series network.
Optionally, the face parsing unit is further configured to: processing the first target image by using a key point extraction network to obtain a face key point; generating a face key point thermodynamic diagram of the image to be processed based on the face key points, and determining the face key point thermodynamic diagram as the face analysis result.
Optionally, the second denoising unit is configured to: and processing the face analysis result and the first target image by using a fine denoising network, and outputting to obtain the second target image.
Optionally, the fine denoising network includes: a multi-layered stacked residual network.
The device provided by the embodiment of the present invention has the same implementation principle and technical effects as those of the foregoing method embodiment, and for the sake of brevity, reference may be made to the corresponding content in the foregoing method embodiment where the device embodiment is not mentioned.
In addition, in the description of embodiments of the present invention, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (11)
1. A method of denoising an image, comprising:
acquiring an image to be processed;
performing first denoising processing on the image to be processed to obtain a first target image, wherein the first denoising processing means removing significant noise in the image to be processed;
carrying out face analysis on the first target image to obtain a face analysis result;
performing second denoising processing on the first target image by using the face analysis result to obtain a second target image, and determining the second target image as a denoising image to be processed; the second denoising process is used for removing fine noise from a part of the image except for the face component part corresponding to the face analysis result in the first target image, wherein the fine noise is other noise except for significant noise in the image to be processed.
2. The method of claim 1, wherein performing a first denoising process on the image to be processed to obtain a first target image comprises:
and carrying out first denoising processing on the image to be processed by using a coarse denoising network to obtain the first target image.
3. The method of claim 2, wherein the coarse denoising network comprises: a Unet network.
4. A method according to any one of claims 1 to 3, wherein performing face analysis on the first target image to obtain a face analysis result includes:
and carrying out segmentation processing on the first target image by using a deep learning semantic segmentation network to obtain at least one face segmentation result, and determining the at least one face segmentation result as the face analysis result.
5. The method of claim 4, wherein the deep learning semantic segmentation network is a deeplab series network.
6. A method according to any one of claims 1 to 3, wherein performing face analysis on the first target image to obtain a face analysis result includes:
processing the first target image by using a key point extraction network to obtain a face key point;
generating a face key point thermodynamic diagram of the image to be processed based on the face key points, and determining the face key point thermodynamic diagram as the face analysis result.
7. The method of claim 1, wherein performing a second denoising process on the first target image pair using the face analysis result, to obtain a second target image comprises:
and processing the face analysis result and the first target image by using a fine denoising network, and outputting to obtain the second target image.
8. The method of claim 7, wherein the fine denoising network comprises: a multi-layered stacked residual network.
9. An apparatus for denoising an image, comprising:
an acquisition unit configured to acquire an image to be processed;
the first denoising unit is used for performing first denoising processing on the image to be processed to obtain a first target image, wherein the first denoising processing means removing significant noise in the image to be processed;
the face analysis unit is used for carrying out face analysis on the first target image to obtain a face analysis result;
the second denoising unit is used for performing second denoising processing on the first target image pair by using the face analysis result to obtain a second target image, and determining the second target image as a denoised image to be processed; the second denoising process is used for removing fine noise from a part of the image except for the face component part corresponding to the face analysis result in the first target image, wherein the fine noise is other noise except for significant noise in the image to be processed.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of the preceding claims 1 to 8 when the computer program is executed.
11. A non-volatile computer readable medium having program code executable by a processor, characterized in that the program code causes the processor to perform the steps of the method of any of the preceding claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010353247.7A CN111476741B (en) | 2020-04-28 | 2020-04-28 | Image denoising method, image denoising device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010353247.7A CN111476741B (en) | 2020-04-28 | 2020-04-28 | Image denoising method, image denoising device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111476741A CN111476741A (en) | 2020-07-31 |
| CN111476741B true CN111476741B (en) | 2024-02-02 |
Family
ID=71761994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010353247.7A Active CN111476741B (en) | 2020-04-28 | 2020-04-28 | Image denoising method, image denoising device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111476741B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112037142B (en) * | 2020-08-24 | 2024-02-13 | 腾讯科技（深圳）有限公司 | Image denoising method, device, computer and readable storage medium |
| CN112990215B (en) * | 2021-03-04 | 2023-12-12 | 腾讯科技（深圳）有限公司 | Image denoising method, device, equipment and storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101571949A (en) * | 2009-05-05 | 2009-11-04 | 南京信息工程大学 | PCNN-based method for de-noising wavelet domain ultrasonic medical image |
| CN104008537A (en) * | 2013-11-04 | 2014-08-27 | 无锡金帆钻凿设备股份有限公司 | Novel noise image fusion method based on CS-CT-CHMM |
| CN106874878A (en) * | 2017-02-21 | 2017-06-20 | 深圳市金立通信设备有限公司 | A kind of image denoising method and terminal |
| CN107169939A (en) * | 2017-05-31 | 2017-09-15 | 广东欧珀移动通信有限公司 | Image processing method and related product |
| CN108305222A (en) * | 2018-01-04 | 2018-07-20 | 浙江大华技术股份有限公司 | A kind of noise-reduction method of image, device, electronic equipment and storage medium |
| CN109086697A (en) * | 2018-07-20 | 2018-12-25 | 腾讯科技（深圳）有限公司 | A kind of human face data processing method, device and storage medium |
| CN109117760A (en) * | 2018-07-27 | 2019-01-01 | 北京旷视科技有限公司 | Image processing method, device, electronic equipment and computer-readable medium |
| CN109639982A (en) * | 2019-01-04 | 2019-04-16 | Oppo广东移动通信有限公司 | A kind of image denoising method, device, storage medium and terminal |
| WO2019119396A1 (en) * | 2017-12-22 | 2019-06-27 | 中国科学院深圳先进技术研究院 | Facial expression recognition method and device |
| EP3579180A1 (en) * | 2018-06-07 | 2019-12-11 | Beijing Kuangshi Technology Co., Ltd. | Image processing method and apparatus, electronic device and non-transitory computer-readable recording medium for selective image enhancement |
| CN110858391A (en) * | 2018-08-23 | 2020-03-03 | 通用电气公司 | Patient-specific deep learning image denoising method and system |
| WO2020057248A1 (en) * | 2018-09-21 | 2020-03-26 | 中兴通讯股份有限公司 | Image denoising method and apparatus, and device and storage medium |
| CN110929651A (en) * | 2019-11-25 | 2020-03-27 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8977012B2 (en) * | 2012-10-31 | 2015-03-10 | Google Inc. | Image denoising system and method |
| US10685429B2 (en) * | 2017-02-22 | 2020-06-16 | Siemens Healthcare Gmbh | Denoising medical images by learning sparse image representations with a deep unfolding approach |
-
2020
- 2020-04-28 CN CN202010353247.7A patent/CN111476741B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101571949A (en) * | 2009-05-05 | 2009-11-04 | 南京信息工程大学 | PCNN-based method for de-noising wavelet domain ultrasonic medical image |
| CN104008537A (en) * | 2013-11-04 | 2014-08-27 | 无锡金帆钻凿设备股份有限公司 | Novel noise image fusion method based on CS-CT-CHMM |
| CN106874878A (en) * | 2017-02-21 | 2017-06-20 | 深圳市金立通信设备有限公司 | A kind of image denoising method and terminal |
| CN107169939A (en) * | 2017-05-31 | 2017-09-15 | 广东欧珀移动通信有限公司 | Image processing method and related product |
| WO2019119396A1 (en) * | 2017-12-22 | 2019-06-27 | 中国科学院深圳先进技术研究院 | Facial expression recognition method and device |
| CN108305222A (en) * | 2018-01-04 | 2018-07-20 | 浙江大华技术股份有限公司 | A kind of noise-reduction method of image, device, electronic equipment and storage medium |
| EP3579180A1 (en) * | 2018-06-07 | 2019-12-11 | Beijing Kuangshi Technology Co., Ltd. | Image processing method and apparatus, electronic device and non-transitory computer-readable recording medium for selective image enhancement |
| CN109086697A (en) * | 2018-07-20 | 2018-12-25 | 腾讯科技（深圳）有限公司 | A kind of human face data processing method, device and storage medium |
| CN109117760A (en) * | 2018-07-27 | 2019-01-01 | 北京旷视科技有限公司 | Image processing method, device, electronic equipment and computer-readable medium |
| CN110858391A (en) * | 2018-08-23 | 2020-03-03 | 通用电气公司 | Patient-specific deep learning image denoising method and system |
| WO2020057248A1 (en) * | 2018-09-21 | 2020-03-26 | 中兴通讯股份有限公司 | Image denoising method and apparatus, and device and storage medium |
| CN109639982A (en) * | 2019-01-04 | 2019-04-16 | Oppo广东移动通信有限公司 | A kind of image denoising method, device, storage medium and terminal |
| CN110929651A (en) * | 2019-11-25 | 2020-03-27 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111476741A (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10949952B2 (en) | Performing detail enhancement on a target in a denoised image | |
| CN108961303B (en) | Image processing method and device, electronic equipment and computer readable medium | |
| CN106778928B (en) | Image processing method and device | |
| CN108805047B (en) | Living body detection method and device, electronic equipment and computer readable medium | |
| CN108038823B (en) | Training method of image morphing network model, image morphing method and computing device | |
| WO2016180224A1 (en) | Method and device for processing image of person | |
| WO2019178893A1 (en) | Motion blur image sharpening method and device, apparatus, and storage medium | |
| US11132544B2 (en) | Visual fatigue recognition method, visual fatigue recognition device, virtual reality apparatus and storage medium | |
| CN107944420B (en) | Illumination processing method and device for face image | |
| JP6866889B2 (en) | Image processing equipment, image processing methods and programs | |
| CN111325657A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| CN111476741B (en) | Image denoising method, image denoising device, electronic equipment and computer readable medium | |
| CN108932702B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN109785264B (en) | Image enhancement method and device and electronic equipment | |
| CN110705353A (en) | Method and device for identifying face to be shielded based on attention mechanism | |
| CN112214773B (en) | Image processing method and device based on privacy protection and electronic equipment | |
| JP7419080B2 (en) | computer systems and programs | |
| CN110503704B (en) | Method and device for constructing three-dimensional graph and electronic equipment | |
| WO2017173578A1 (en) | Image enhancement method and device | |
| CN110910512B (en) | Virtual object self-adaptive adjustment method, device, computer equipment and storage medium | |
| CN112714337A (en) | Video processing method and device, electronic equipment and storage medium | |
| CN112270747A (en) | Face recognition method and device and electronic equipment | |
| CN112001285A (en) | Method, device, terminal and medium for processing beautifying image | |
| KR20230086998A (en) | A computer program that creates virtual character contnet by adding noise to face segment regions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |