CN111899199B - Image processing method, device, equipment and storage medium - Google Patents
Image processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111899199B CN111899199B CN202010789107.4A CN202010789107A CN111899199B CN 111899199 B CN111899199 B CN 111899199B CN 202010789107 A CN202010789107 A CN 202010789107A CN 111899199 B CN111899199 B CN 111899199B
- Authority
- CN
- China
- Prior art keywords
- interference
- image
- sample
- network model
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 60
- 230000008030 elimination Effects 0.000 claims abstract description 47
- 238000003379 elimination reaction Methods 0.000 claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 abstract description 13
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an image processing method, an image processing device, image processing equipment and a storage medium. The method comprises the following steps: acquiring an image to be processed and the type of interference factors of the image to be processed; acquiring a corresponding interference elimination network model according to the type of the interference factor; the interference elimination network model is obtained by training a sample interference image based on a sample original image and after adding sample interference features based on the sample original image; and removing interference factor characteristics in the image to be processed based on the interference removing network model to obtain a result image. The method can identify the interference factor characteristics in the image to be processed, further realize the removal of interference factors in the image to be processed, improve the definition of the image to be processed, and further ensure the accuracy and reliability of feature extraction or labeling based on the image to be processed. In addition, the application also provides an image processing device, equipment and a storage medium, and the beneficial effects are the same as those described above.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image processing method, apparatus, device, and storage medium.
Background
At present, with the rapid development and perfection of deep learning, the computer vision technology has greatly advanced, and at present, the object features or data in the scene image can be identified and marked by the computer vision technology, and manual marking according to the content in the image is not needed in a manual mode.
The basis of the feature acquisition or labeling of the image based on the computer vision technology is that the image is clear, but under the application scene such as the entrance control or the safety monitoring of a vehicle detection and license plate recognition system, the image obtained by shooting the vehicle is often interfered by objective factors such as weather, so that the definition of the image is reduced, and the accuracy and the reliability of the feature extraction or labeling based on the image are difficult to ensure.
Therefore, the image processing method is provided to remove interference factors in the image, improve the definition of the image, and further ensure the accuracy and reliability of feature extraction or labeling based on the image, which is a problem to be solved by those skilled in the art.
Disclosure of Invention
The purpose of the application is to provide an image processing method, an image processing device, image processing equipment and a storage medium, so as to remove interference factors in an image, improve the definition of the image, and further ensure the accuracy and reliability of feature extraction or labeling based on the image.
In order to solve the above technical problems, the present application provides an image processing method, including:
acquiring an image to be processed and the type of interference factors of the image to be processed;
acquiring a corresponding interference elimination network model according to the type of the interference factor; the interference elimination network model is obtained by training a sample interference image based on a sample original image and after adding sample interference features based on the sample original image;
and removing interference factor characteristics in the image to be processed based on the interference removing network model to obtain a result image.
Preferably, the generating process of the interference-free network model includes:
acquiring a sample original image and sample interference characteristics corresponding to the type of the interference factors;
adding the sample interference features to the sample original image to obtain a sample interference image;
extracting interference extraction features in the sample interference images based on the network model to be trained, and removing the interference extraction features based on the sample interference images to obtain interference removal images;
calculating a first feature loss between the sample interference feature and the interference extraction feature, and a second feature loss between the interference removal image and the sample original image;
and updating the network model to be trained according to the first characteristic loss and the second characteristic loss to obtain the interference-free network model.
Preferably, the type of interference factor of the image to be processed includes a type of rain and fog interference;
correspondingly, the sample interference features comprise sample rain mark features and sample light features, and the interference extraction features comprise rain mark features and light features.
Preferably, extracting the interference extraction feature in the sample interference image based on the network model to be trained, and removing the interference extraction feature based on the sample interference image, includes:
extracting raindrop features in the sample interference image based on a first feature extraction model in the network model to be trained;
extracting light features in the sample interference image based on a second feature extraction model in the network model to be trained;
and removing raindrop features and light features in the sample interference image through a hybrid network in the network model to be trained.
Preferably, the first characteristic loss comprises a mean square error and the second characteristic loss comprises a countering loss.
Preferably, updating the network model to be trained according to the first feature loss and the second feature loss to obtain a de-interference network model, including:
generating a total feature loss based on the first feature loss and the second feature loss;
and updating the network model to be trained by reversely transmitting the total characteristic loss to the network model to be trained, so as to obtain the interference-free network model.
In addition, the present application also provides an image processing apparatus including:
the image acquisition module is used for acquiring the image to be processed and the interference factor type of the image to be processed;
the model acquisition module is used for acquiring a corresponding interference-free network model according to the type of the interference factors; the interference elimination network model is obtained by training a sample interference image based on a sample original image and after adding sample interference features based on the sample original image;
and the model processing module is used for removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image.
Preferably, the apparatus further comprises:
the original sample acquisition module is used for acquiring an original image of a sample and a sample interference characteristic corresponding to the type of the interference factor;
the interference sample generation module is used for adding the sample interference characteristics to the sample original image to obtain a sample interference image;
the extraction and removal module is used for extracting interference extraction features in the sample interference image based on the network model to be trained, and removing the interference extraction features based on the sample interference image to obtain an interference removal image;
the loss calculation module is used for calculating a first characteristic loss between the sample interference characteristic and the interference extraction characteristic and a second characteristic loss between the interference removal image and the sample original image;
and the model updating module is used for updating the network model to be trained according to the first characteristic loss and the second characteristic loss to obtain the interference-free network model.
In addition, the present application also provides an image processing apparatus including:
a memory for storing a computer program;
a processor for implementing the steps of the image processing method as described above when executing the computer program.
Furthermore, the present application also provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the image processing method as described above.
According to the image processing method, firstly, an image to be processed and an interference factor type corresponding to the image to be processed are obtained, and then, a corresponding interference elimination network model is obtained according to the interference factor type, wherein the interference elimination network model is obtained through training of a sample interference image based on a sample original image and a sample interference feature added based on the sample original image. And after the interference elimination network model is obtained, removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image. The method utilizes the interference elimination network model obtained based on the training of the sample original image and the sample interference image to identify the interference factor characteristics in the image to be processed, thereby realizing the removal of the interference factors in the image to be processed, improving the definition of the image to be processed, and further ensuring the accuracy and reliability of feature extraction or labeling based on the image to be processed. In addition, the application also provides an image processing device, equipment and a storage medium, and the beneficial effects are the same as those described above.
Drawings
For a clearer description of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an image processing method disclosed in an embodiment of the present application;
FIG. 2 is a flowchart of a process for generating a de-interference network model according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments herein without making any inventive effort are intended to fall within the scope of the present application.
The basis of the feature acquisition or labeling of the image based on the computer vision technology is that the image is clear, but under the application scene such as the entrance control or the safety monitoring of a vehicle detection and license plate recognition system, the image obtained by shooting the vehicle is often interfered by objective factors such as weather, so that the definition of the image is reduced, and the accuracy and the reliability of the feature extraction or labeling based on the image are difficult to ensure.
Therefore, the core of the application is to provide an image processing method to remove interference factors in an image, improve the definition of the image, and further ensure the accuracy and reliability of feature extraction or labeling based on the image.
In order to provide a better understanding of the present application, those skilled in the art will now make further details of the present application with reference to the drawings and detailed description.
Referring to fig. 1, an embodiment of the present application discloses an image processing method, which includes:
step S10: and acquiring the type of interference factors of the image to be processed.
It should be noted that, the image to be processed obtained in this step refers to an image captured under an actual scene, and the content of the image to be processed is different according to different types of actual scenes, including, but not limited to, a vehicle image captured by a vehicle detection and license plate recognition system under a parking lot entrance control or safety monitoring scene, and the like. Because the interference factors affecting the definition of the image in the corresponding scene often exist in the image to be processed acquired in the actual scene, the interference factor type of the image to be processed is further acquired after the image to be processed is acquired in the step, and the purpose is to remove the interference factors in the image to be processed in a targeted manner according to the interference factor type of the image to be processed in the subsequent step.
Step S11: and acquiring a corresponding interference elimination network model according to the interference factor type.
The interference elimination network model is obtained through training based on a sample original image and a sample interference image obtained by adding sample interference features based on the sample original image.
After the interference factor type of the image to be processed is obtained, the corresponding interference elimination network model is further obtained according to the interference factor type, and the interference elimination network model is obtained by training the sample interference image based on the sample original image and the sample interference feature added based on the sample original image, so that the interference elimination network model has the capability of eliminating the interference feature in the image.
Step S12: and removing interference factor characteristics in the image to be processed based on the interference removing network model to obtain a result image.
After the corresponding interference elimination network model is obtained according to the interference factor type, the interference elimination network model is further based on the interference elimination network model to eliminate the interference factor characteristics in the image to be processed, and a result image is obtained, wherein the result image is the image of the image to be processed after the interference factors are eliminated.
According to the image processing method, firstly, an image to be processed and an interference factor type corresponding to the image to be processed are obtained, and then, a corresponding interference elimination network model is obtained according to the interference factor type, wherein the interference elimination network model is obtained through training of a sample interference image based on a sample original image and a sample interference feature added based on the sample original image. And after the interference elimination network model is obtained, removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image. The method utilizes the interference elimination network model obtained based on the training of the sample original image and the sample interference image to identify the interference factor characteristics in the image to be processed, thereby realizing the removal of the interference factors in the image to be processed, improving the definition of the image to be processed, and further ensuring the accuracy and reliability of feature extraction or labeling based on the image to be processed.
Referring to fig. 2, an embodiment of the present application discloses a process for generating a de-interference network model, including:
step S20: and acquiring a sample original image and a sample interference characteristic corresponding to the interference factor type.
The original image of the sample obtained in the step refers to an image without interference features, which may be a high-definition image obtained by shooting under a specific scene, and the interference features of the sample obtained in the step refer to interference factor features of a specific interference factor type under a corresponding scene.
Step S21: and adding the sample interference features to the sample original image to obtain a sample interference image.
After the sample original image and the sample interference feature corresponding to the interference factor type are obtained, the sample interference feature is further added to the sample original image to obtain a sample interference image, so that the network model to be trained is trained based on the known sample interference feature, the sample interference image and the sample original image in the subsequent steps.
Step S22: and extracting interference extraction features in the sample interference image based on the network model to be trained, and removing the interference extraction features based on the sample interference image to obtain an interference removal image.
After the sample interference features are added to the sample original image to obtain a sample interference image, the interference extraction features in the sample interference image are further extracted based on the network model to be trained, and the interference extraction features are removed based on the sample interference image to obtain an interference removal image.
Step S23: a first feature loss between the sample interference feature and the interference extraction feature and a second feature loss between the interference-removed image and the sample original image are calculated.
After extracting the interference extraction feature in the sample interference image based on the network model to be trained and removing the interference extraction feature based on the sample interference image to obtain an interference removal image, the step further calculates a first feature loss between the sample interference feature and the interference extraction feature and a second feature loss between the interference removal image and the sample original image, wherein the first feature loss and the second feature loss are two types of feature difference parameters in relative terms, and represent differences between the interference extraction feature and the sample interference feature and differences between the interference removal image and the sample original image.
Step S24: and updating the network model to be trained according to the first characteristic loss and the second characteristic loss to obtain the interference-free network model.
After calculating a first characteristic loss between the sample interference characteristic and the interference extraction characteristic and a second characteristic loss between the interference removal image and the sample original image, updating the network model to be trained by the step of further updating the first characteristic loss and the second characteristic loss so as to achieve the purpose of training the network model to be trained through the sample interference image, and further obtaining the interference removal network model. In an actual scenario, in order to improve the usability of the interference-free network model, the cyclic network model to be trained needs to be trained through a certain number of sample interference images until the first characteristic loss and the second characteristic loss reach a specific threshold standard, and then the process of generating the interference-free network model based on the network model to be trained is considered to be ended.
According to the method, the sample interference image is obtained by adding the sample interference characteristics to the sample original image, and the interference-removing network model is obtained by training the network model to be trained based on the sample interference image, so that the reliability of the interference-removing network model is further ensured.
On the basis of the above embodiment, as a preferred implementation manner, the type of the interference factor of the image to be processed includes a rain and fog interference type;
correspondingly, the sample interference features comprise sample rain mark features and sample light features, and the interference extraction features comprise rain mark features and light features.
It should be noted that, because the image to be processed photographed in the actual scene often has the interference factors of rain marks and fog due to objective factors such as rain, the interference factor types of the image to be processed in the embodiment include rain and fog interference types, and further when the de-interference network model is generated, the sample original image and the sample interference feature corresponding to the interference factor type are obtained, and then the sample original image, the sample rain mark feature corresponding to the interference factor type and the sample light feature are specifically obtained; and extracting raindrop features and light features in the sample interference image based on the network model to be trained when extracting the interference extraction features in the sample interference image based on the network model to be trained. The light features may further include, among other things, atmospheric light component features, dark channel features, and the like. According to the method and the device for eliminating the rain and fog interference in the image to be processed, the types of the interference factors of the image to be processed are further divided, and the reliability of eliminating the interference factors of the rain and fog interference type in the image to be processed by the interference elimination network model is further ensured.
Still further, as a preferred embodiment, extracting the interference extraction feature in the sample interference image based on the network model to be trained, and removing the interference extraction feature based on the sample interference image, includes:
extracting raindrop features in the sample interference image based on a first feature extraction model in the network model to be trained;
extracting light features in the sample interference image based on a second feature extraction model in the network model to be trained;
and removing raindrop features and light features in the sample interference image through a hybrid network in the network model to be trained.
It should be noted that, the key point of this embodiment is that the network model to be trained includes a first feature extraction model, a second feature extraction model and a hybrid network, the rainmark feature in the sample interference image is extracted by the first feature extraction model, the light feature in the sample interference image is extracted by the second feature extraction model, and the rainmark feature and the light feature are removed from the sample interference image by the hybrid network, so that accuracy of the rainmark feature, the light feature and the interference removal image is further ensured, and reliability of the generated interference removal network model is further ensured.
Still further, the first characteristic loss comprises a mean square error and the second characteristic loss comprises a contrast loss.
In this embodiment, the first feature loss between the sample interference feature and the interference extraction feature includes a mean square error, and the second feature loss between the interference removal image and the sample original image includes a countering loss, so that the reliability of the generated interference removal network model is further ensured.
Furthermore, on the basis of the above-mentioned series of embodiments, as a preferred embodiment, updating the network model to be trained according to the first feature loss and the second feature loss to obtain a de-interference network model, including:
generating a total feature loss based on the first feature loss and the second feature loss;
and updating the network model to be trained by reversely transmitting the total characteristic loss to the network model to be trained, so as to obtain the interference-free network model.
It should be noted that, in the present embodiment, in updating the network model to be trained according to the first feature loss and the second feature loss, specifically, the total feature loss is generated based on the first feature loss and the second feature loss, specifically, the total feature loss may be obtained by accumulating the first feature loss and the second feature loss, and then the total feature loss is provided to the network model to be trained by back propagation, so as to update the network model to be trained. The back propagation can relatively improve the updating efficiency of the network model to be trained, so that the generating efficiency of the interference-removing network model can be relatively ensured.
Referring to fig. 3, an embodiment of the present application provides an image processing apparatus, including:
an image acquisition module 10, configured to acquire an image to be processed and an interference factor type of the image to be processed;
the model acquisition module 11 is used for acquiring a corresponding interference-free network model according to the type of the interference factors; the interference elimination network model is obtained by training a sample interference image based on a sample original image and after adding sample interference features based on the sample original image;
the model processing module 12 is configured to remove the interference factor features in the image to be processed based on the interference-free network model, and obtain a result image.
Further, as a preferred embodiment, the apparatus further comprises:
the original sample acquisition module is used for acquiring an original image of a sample and a sample interference characteristic corresponding to the type of the interference factor;
the interference sample generation module is used for adding the sample interference characteristics to the sample original image to obtain a sample interference image;
the extraction and removal module is used for extracting interference extraction features in the sample interference image based on the network model to be trained, and removing the interference extraction features based on the sample interference image to obtain an interference removal image;
the loss calculation module is used for calculating a first characteristic loss between the sample interference characteristic and the interference extraction characteristic and a second characteristic loss between the interference removal image and the sample original image;
and the model updating module is used for updating the network model to be trained according to the first characteristic loss and the second characteristic loss to obtain the interference-free network model.
The image processing device provided by the application firstly obtains an image to be processed and an interference factor type corresponding to the image to be processed, and further obtains a corresponding interference elimination network model according to the interference factor type, wherein the interference elimination network model is obtained based on a sample original image and a sample interference image training after a sample interference feature is added based on the sample original image. And after the interference elimination network model is obtained, removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image. The device can identify the interference factor characteristics in the image to be processed by utilizing the interference elimination network model obtained based on the sample original image and the sample interference image training, so that the interference factor in the image to be processed is eliminated, the definition of the image to be processed is improved, and the accuracy and the reliability of feature extraction or labeling based on the image to be processed are further ensured.
In addition, an embodiment of the present application further provides an image processing apparatus, including:
a memory for storing a computer program;
a processor for implementing the steps of the image processing method as described above when executing the computer program.
The image processing device provided by the application firstly obtains an image to be processed and an interference factor type corresponding to the image to be processed, and further obtains a corresponding interference elimination network model according to the interference factor type, wherein the interference elimination network model is obtained based on a sample original image and a sample interference image training after a sample interference feature is added based on the sample original image. And after the interference elimination network model is obtained, removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image. The device can identify the interference factor characteristics in the image to be processed by utilizing the interference elimination network model obtained based on the sample original image and the sample interference image training, so that the interference factor in the image to be processed is eliminated, the definition of the image to be processed is improved, and the accuracy and the reliability of feature extraction or labeling based on the image to be processed are further ensured.
In addition, the embodiment of the application further provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the image processing method are implemented.
The computer readable storage medium provided by the application is used for firstly acquiring an image to be processed and an interference factor type corresponding to the image to be processed, and further acquiring a corresponding interference elimination network model according to the interference factor type, wherein the interference elimination network model is obtained based on a sample original image and a sample interference image training after adding sample interference features based on the sample original image. And after the interference elimination network model is obtained, removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image. The computer readable storage medium can identify the interference factor characteristics in the image to be processed by utilizing the interference elimination network model obtained based on the sample original image and the sample interference image training, so that the interference factor in the image to be processed is eliminated, the definition of the image to be processed is improved, and the accuracy and the reliability of feature extraction or labeling based on the image to be processed are further ensured.
The above describes in detail an image processing method, apparatus, device and storage medium provided in the present application. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present application, and such improvements and modifications fall within the scope of the claims of the present application.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (8)
1. An image processing method, comprising:
acquiring an image to be processed and the type of interference factors of the image to be processed;
acquiring a corresponding interference elimination network model according to the interference factor type; the interference elimination network model is obtained by training a sample interference image based on a sample original image and a sample interference feature added based on the sample original image;
removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image;
the generating process of the interference-free network model comprises the following steps:
acquiring the original image of the sample and the sample interference characteristics corresponding to the interference factor type;
adding the sample interference features to the sample original image to obtain the sample interference image;
extracting interference extraction features in the sample interference image based on a network model to be trained, and removing the interference extraction features based on the sample interference image to obtain an interference removal image;
calculating a first feature loss between the sample interference feature and the interference extraction feature, and a second feature loss between the interference-removed image and the sample original image;
and updating the network model to be trained according to the first characteristic loss and the second characteristic loss to obtain the interference-free network model.
2. The image processing method according to claim 1, wherein the type of disturbance factor of the image to be processed includes a type of rain and fog disturbance;
correspondingly, the sample interference features comprise sample raindrop features and sample light features, and the interference extraction features comprise raindrop features and light features.
3. The image processing method according to claim 2, wherein the extracting the interference extraction features in the sample interference image based on the network model to be trained and removing the interference extraction features based on the sample interference image comprises:
extracting the raindrop features in the sample interference image based on a first feature extraction model in the network model to be trained;
extracting the light features in the sample interference image based on a second feature extraction model in the network model to be trained;
and removing the raindrop features and the light features in the sample interference image through a mixed network in the network model to be trained.
4. The image processing method of claim 3, wherein the first characteristic loss comprises a mean square error and the second characteristic loss comprises a contrast loss.
5. The image processing method according to any one of claims 1 to 4, wherein updating the network model to be trained based on the first feature loss and the second feature loss to obtain the interference-free network model includes:
generating a total feature loss based on the first feature loss and the second feature loss;
and updating the network model to be trained by reversely transmitting the total characteristic loss to the network model to be trained, so as to obtain the interference-free network model.
6. An image processing apparatus, comprising:
the image acquisition module is used for acquiring an image to be processed and the interference factor type of the image to be processed;
the model acquisition module is used for acquiring a corresponding interference elimination network model according to the interference factor type; the interference elimination network model is obtained by training a sample interference image based on a sample original image and a sample interference feature added based on the sample original image;
the model processing module is used for removing interference factor characteristics in the image to be processed based on the interference elimination network model to obtain a result image;
wherein the image processing apparatus further comprises:
the original sample acquisition module is used for acquiring the original image of the sample and the sample interference characteristics corresponding to the interference factor type;
the interference sample generation module is used for adding the sample interference characteristics to the sample original image to obtain the sample interference image;
the extraction and removal module is used for extracting interference extraction features in the sample interference image based on the network model to be trained, and removing the interference extraction features based on the sample interference image to obtain an interference removal image;
a loss calculation module for calculating a first feature loss between the sample interference feature and the interference extraction feature, and a second feature loss between the interference removal image and the sample original image;
and the model updating module is used for updating the network model to be trained according to the first characteristic loss and the second characteristic loss to obtain the interference-free network model.
7. An image processing apparatus, characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the image processing method according to any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the image processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010789107.4A CN111899199B (en) | 2020-08-07 | 2020-08-07 | Image processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010789107.4A CN111899199B (en) | 2020-08-07 | 2020-08-07 | Image processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111899199A CN111899199A (en) | 2020-11-06 |
| CN111899199B true CN111899199B (en) | 2024-03-19 |
Family
ID=73246084
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010789107.4A Active CN111899199B (en) | 2020-08-07 | 2020-08-07 | Image processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111899199B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114004757B (en) * | 2021-10-14 | 2024-04-05 | 大族激光科技产业集团股份有限公司 | Method, system, device and storage medium for removing interference in industrial image |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109118435A (en) * | 2018-06-15 | 2019-01-01 | 广东工业大学 | A kind of depth residual error convolutional neural networks image de-noising method based on PReLU |
| CN109191409A (en) * | 2018-07-25 | 2019-01-11 | 北京市商汤科技开发有限公司 | Image procossing, network training method, device, electronic equipment and storage medium |
| CN109754376A (en) * | 2018-12-28 | 2019-05-14 | 深圳美图创新科技有限公司 | Image de-noising method and device |
| CN109886210A (en) * | 2019-02-25 | 2019-06-14 | 百度在线网络技术（北京）有限公司 | A kind of traffic image recognition methods, device, computer equipment and medium |
| CN110163813A (en) * | 2019-04-16 | 2019-08-23 | 中国科学院深圳先进技术研究院 | A kind of image rain removing method, device, readable storage medium storing program for executing and terminal device |
| CN110189260A (en) * | 2019-04-15 | 2019-08-30 | 浙江大学 | A kind of image denoising method based on multiple dimensioned parallel gate neural network |
| CN110288547A (en) * | 2019-06-27 | 2019-09-27 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating image denoising model |
| CN110490118A (en) * | 2019-08-14 | 2019-11-22 | 厦门美图之家科技有限公司 | Image processing method and device |
| CN110838095A (en) * | 2019-11-06 | 2020-02-25 | 广西师范大学 | Single image rain removing method and system based on cyclic dense neural network |
| CN111192215A (en) * | 2019-12-30 | 2020-05-22 | 百度时代网络技术（北京）有限公司 | Image processing method, device, equipment and readable storage medium |
| CN111291704A (en) * | 2020-02-24 | 2020-06-16 | 上海眼控科技股份有限公司 | Interference removing method and device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10706508B2 (en) * | 2018-03-29 | 2020-07-07 | Disney Enterprises, Inc. | Adaptive sampling in Monte Carlo renderings using error-predicting neural networks |
-
2020
- 2020-08-07 CN CN202010789107.4A patent/CN111899199B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109118435A (en) * | 2018-06-15 | 2019-01-01 | 广东工业大学 | A kind of depth residual error convolutional neural networks image de-noising method based on PReLU |
| CN109191409A (en) * | 2018-07-25 | 2019-01-11 | 北京市商汤科技开发有限公司 | Image procossing, network training method, device, electronic equipment and storage medium |
| CN109754376A (en) * | 2018-12-28 | 2019-05-14 | 深圳美图创新科技有限公司 | Image de-noising method and device |
| CN109886210A (en) * | 2019-02-25 | 2019-06-14 | 百度在线网络技术（北京）有限公司 | A kind of traffic image recognition methods, device, computer equipment and medium |
| CN110189260A (en) * | 2019-04-15 | 2019-08-30 | 浙江大学 | A kind of image denoising method based on multiple dimensioned parallel gate neural network |
| CN110163813A (en) * | 2019-04-16 | 2019-08-23 | 中国科学院深圳先进技术研究院 | A kind of image rain removing method, device, readable storage medium storing program for executing and terminal device |
| CN110288547A (en) * | 2019-06-27 | 2019-09-27 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating image denoising model |
| CN110490118A (en) * | 2019-08-14 | 2019-11-22 | 厦门美图之家科技有限公司 | Image processing method and device |
| CN110838095A (en) * | 2019-11-06 | 2020-02-25 | 广西师范大学 | Single image rain removing method and system based on cyclic dense neural network |
| CN111192215A (en) * | 2019-12-30 | 2020-05-22 | 百度时代网络技术（北京）有限公司 | Image processing method, device, equipment and readable storage medium |
| CN111291704A (en) * | 2020-02-24 | 2020-06-16 | 上海眼控科技股份有限公司 | Interference removing method and device, computer equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| D3R-Net: Dynamic Routing Residue Recurrent Network for Video Rain Removal;J. Liu 等;《IEEE Transactions on Image Processing》;第28卷;699-712 * |
| Semi-supervised transfer learning for image rain removal;Wei W 等;《Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.2019》;3877-3886 * |
| 基于卷积神经网络的红外图像去噪方法研究;朱宜生 等;《环境技术》(第06期);99-103 * |
| 结合深度残差学习和感知损失的图像去噪;吴从中 等;《中国图象图形学报》(第10期);1483-1491 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111899199A (en) | 2020-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108268867B (en) | License plate positioning method and device | |
| EP3786835A1 (en) | Traffic image recognition method and apparatus, and computer device and medium | |
| CN108805016B (en) | Head and shoulder area detection method and device | |
| CN111275040B (en) | Positioning method and device, electronic equipment and computer readable storage medium | |
| CN111178272B (en) | Method, device and equipment for identifying driver behavior | |
| CN112598922B (en) | Parking space detection method, device, equipment and storage medium | |
| CN113988112B (en) | Method, device and equipment for detecting lane line and storage medium | |
| CN111340041B (en) | License plate recognition method and device based on deep learning | |
| CN112364898A (en) | Image identification automatic labeling method, device, equipment and storage medium | |
| CN111899199B (en) | Image processing method, device, equipment and storage medium | |
| CN110197501B (en) | Image processing method and apparatus | |
| CN112101246A (en) | Vehicle identification method, device, equipment and medium | |
| CN109278759B (en) | Vehicle safe driving auxiliary system | |
| CN113850151A (en) | Method, device, terminal and storage medium for identifying distraction behavior of driver | |
| CN112733864B (en) | Model training method, target detection method, device, equipment and storage medium | |
| CN116977215A (en) | Image defogging method, device, equipment and storage medium | |
| CN116311205A (en) | License plate recognition method, license plate recognition device, electronic equipment and storage medium | |
| CN115762172A (en) | Method, device, equipment and medium for identifying vehicles entering and exiting parking places | |
| CN111177878A (en) | Method, device and terminal for screening derivative simulation scenes | |
| KR101910256B1 (en) | Lane Detection Method and System for Camera-based Road Curvature Estimation | |
| CN115272284A (en) | Power transmission line defect identification method based on image quality evaluation | |
| KR102251799B1 (en) | Non-supervised-learning-based poor vehicle number image recovery learning method and device | |
| CN113609980A (en) | Lane line sensing method and device for automatic driving vehicle | |
| CN114022869B (en) | Vehicle heavy identification method and device based on cascade network | |
| CN116543365B (en) | Lane line identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |