CN113919491A - Method for assisting in training classification network by means of heatmap - Google Patents
Method for assisting in training classification network by means of heatmap Download PDFInfo
- Publication number
- CN113919491A CN113919491A CN202010662452.1A CN202010662452A CN113919491A CN 113919491 A CN113919491 A CN 113919491A CN 202010662452 A CN202010662452 A CN 202010662452A CN 113919491 A CN113919491 A CN 113919491A
- Authority
- CN
- China
- Prior art keywords
- network
- heatmap
- training
- classification
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000013507 mapping Methods 0.000 claims abstract description 7
- 238000011176 pooling Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 abstract description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for assisting in training a classification network by means of heatmap, which comprises the following steps: step 1, a network is provided with two branches in the middle, wherein the left branch uses a full-connection network to predict a classification result; step 2, designing the right branch as a network only used for training; step 3, making a heatmap in a network used for training only in the right branch, marking the position of a common characteristic region in a sample picture, and mapping the position to a heat map; step 4, making an output layer label in the left-branch network; step 5, training a network, namely weighting and adding the loss value of the heatmap and the loss value of the output layer, and then training the network, wherein the first half network before branching, the heatmap network and the classification network share the weight, so that the image characteristics learned by the heatmap can bring benefits to the classification network; and 6, removing the network only used for training when the trained network is used for reasoning. The invention provides a novel deep learning training method which can improve the generalization capability of a classification network under the condition of limited training samples.
Description
Technical Field
The invention relates to the technical field of convolutional neural networks, in particular to a method for assisting in training a classification network by means of heatmap.
Background
Today, convolutional neural network deep learning is increasingly widely applied. Existing deep learning convolutional neural networks, such as classification networks that contain fully connected layers, and full convolutional networks for image segmentation that do not contain fully connected layers.
However, the existing deep learning neural network training is highly dependent on training samples, and under the condition of less training samples, an overfitting phenomenon is easy to occur, so that the model generalization capability is poor.
The common terminology in the prior art is as follows:
1. the convolutional neural network is a feature extractor consisting of a convolutional layer and a pooling layer.
2. heatmap, heat map, here a manually calibrated picture that corresponds to the training sample.
3. And a full connection layer, wherein each neuron is fully connected with all neurons in the previous layer.
Disclosure of Invention
In order to solve the above problems, the present invention is directed to: a new deep learning training method is provided, and the generalization capability of a classification network can be improved under the condition that training samples are limited.
Specifically, the invention provides a method for assisting in training a classification network by means of heatmap, which comprises the following steps:
step 1, a network is provided with two branches in the middle, wherein the left branch uses a full-connection network to predict a classification result;
step 2, designing the right branch as a network only used for training;
step 3, making a heatmap in a network used for training only in the right branch, marking the position of a common characteristic region in a sample picture, and mapping the position to a heat map;
step 4, making an output layer label in the left-branch network;
step 5, training a network, namely weighting and adding the loss value of the heatmap and the loss value of the output layer, and then training the network, wherein the first half network before branching, the heatmap network and the classification network share the weight, so that the image characteristics learned by the heatmap can bring benefits to the classification network;
and 6, when the trained network is used for reasoning, the network only used for training is removed, and the calculated amount is saved.
The heatmap network is a right branch after branching, the classification network is a left branch after branching, the output of the first half part network is used as the input of the heatmap network and the classification network of the last two networks, the heatmap network and the classification network are parallel relations and are not dependent on each other and not influenced mutually, and the heatmap network and the classification network influence the weight data of the first half part network together in the network learning process.
The method is characterized in that a heatmap with a corresponding relation with a training sample is generated through manually judging and calibrating the characteristics of the required learning image and is used for assisting training.
The fully connected network including the left branch in step 1 further includes: an Input layer, a C1 convolutional layer, an S2 pooling layer, a C3 convolutional layer, an S4 pooling layer, a C5 convolutional layer, an F6 full link layer, and an output layer.
And (2) predicting the heatmap of the picture by using an upsampling and convolution network in the right branch of the step 2, wherein the heatmap and the training sample picture have a corresponding relation.
The heatmap and the training sample picture have a corresponding relation, and the size of the heatmap is 1:1 or equal ratio mapping according to the actual required ratio.
The step 3 further comprises:
whether a specific area exists in the picture is detected by the classification network, the edge information of the specific area is important information, and the background and other contents except the edge of the specific area are ignorable information, so that the area of the edge of the specific area is defined as a common feature area and is a pattern feature which needs to be learned by the neural network, and therefore, the edge area of the specific area in the sample picture is mapped to the heatmap, namely, the area containing the important information is highlighted, and if no specific area exists in the picture, the heatmap pattern is a full black picture.
The step 4 of manufacturing the output layer label further comprises the following steps:
whether a picture contains a specific area or not is judged, the picture containing the specific area can be classified into one type, and the picture not containing the specific area can be classified into another type.
Thus, the present application has the advantages that: and generating a heatmap with a corresponding relation between the heatmap and the training samples through manually judging and calibrating the characteristics of the required learning images, wherein the heatmap is used for assisting training, and effectively improving the generalization capability of the classification network under the condition of less training samples. And an auxiliary characteristic learning network is added in the training process to assist the network in learning the image characteristics, corresponding calculation is removed in the reasoning process, the calculated amount is saved, and the efficiency is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention.
FIG. 1 is a schematic flow diagram of the process of the present invention.
Fig. 2 is a schematic diagram of a network structure to which the method of the present invention is applied.
Fig. 3 is a picture example of one case in step 3 of the specific embodiment.
Fig. 4 is a picture example of another case in step 3 of the embodiment.
Detailed Description
In order that the technical contents and advantages of the present invention can be more clearly understood, the present invention will now be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, the present invention relates to a method for assisting in training a classification network by means of heatmap, said method comprising the steps of:
step 1, a network is provided with two branches in the middle, wherein the left branch uses a full-connection network to predict a classification result;
step 2, designing the right branch as a network only used for training;
step 3, making a heatmap in a network used for training only in the right branch, marking the position of a common characteristic region in a sample picture, and mapping the position to a heat map;
step 4, making an output layer label in the left-branch network;
step 5, training a network, namely weighting and adding the loss value of the heatmap and the loss value of the output layer, and then training the network, wherein the first half network before branching, the heatmap network and the classification network share the weight, so that the image characteristics learned by the heatmap can bring benefits to the classification network;
and 6, removing the network only used for training when the trained network is used for reasoning.
As shown in fig. 2, it can be seen from the network structure applying the method of the present invention that the network has branches in the middle, the left branch uses a fully connected network, the classification result is predicted, the right branch uses an upsampling and convolutional network, the heatmap of the picture is predicted, the heatmap and the training sample picture have a corresponding relationship, and the size can be equal ratio mapping of 1:1 or other ratios.
Further, in step 3, a heatmap is created, and the positions of the common feature areas in the sample picture are labeled and mapped to a heat map. As shown in fig. 3, for example, a classification network detects whether there is paper in the picture, the edge information of the paper is an important information, the background and the person in the paper are negligible information, the area of the paper edge is defined as a common feature area, which is a pattern feature that needs to be learned by the neural network, so that the area of the paper edge in the left sample picture in fig. 3 is mapped to the heatmap, i.e., the right half of fig. 3, and includes the important information area highlighted, if there is no paper in the pattern, the heatmap pattern is a full black picture, as shown in fig. 4.
Further, in step 4, an output layer label is made, for example, if we judge whether the picture contains a piece of paper, the picture can be classified into one type containing paper and the picture can be classified into another type containing no paper.
Further, in step 5, the network is trained, the loss value of the heatmap and the loss value of the output layer are weighted and added, and then the network is trained, because the first half, the heatmap network and the classification network share the weight, the image features learned by the heatmap can make the classification network profit, avoid the network learning other irrelevant features, and enhance the generalization ability of the network.
Finally, in step 6, when the trained network inference is used, the network used only for training in fig. 2 is removed, so that the calculation amount is saved.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A method for assisting in training a classification network with a heatmap, the method comprising the steps of:
step 1, a network is provided with two branches in the middle, wherein the left branch uses a full-connection network to predict a classification result;
step 2, designing the right branch as a network only used for training;
step 3, making a heatmap in a network used for training only in the right branch, marking the position of a common characteristic region in a sample picture, and mapping the position to a heat map;
step 4, making an output layer label in the left-branch network;
step 5, training a network, namely weighting and adding the loss value of the heatmap and the loss value of the output layer, and then training the network, wherein the first half network before branching, the heatmap network and the classification network share the weight, so that the image characteristics learned by the heatmap can bring benefits to the classification network;
and 6, removing the network only used for training when the trained network is used for reasoning.
2. The method for assisting in training classification networks by means of heatmap as claimed in claim 1, wherein the method is to generate heatmap with correspondence relationship between training samples for assisting training by manually determining and calibrating the required learning image features.
3. The method of claim 1, wherein the step 1 of the fully-connected network including the left branch further comprises: an Input layer, a C1 convolutional layer, an S2 pooling layer, a C3 convolutional layer, an S4 pooling layer, a C5 convolutional layer, an F6 full link layer, and an output layer.
4. The method for assisting in training a classification network by means of heatmap as claimed in claim 1, wherein the step 2 right branch predicts heatmap of pictures using upsampling and convolutional networks, which have correspondence with the training sample pictures.
5. The method as claimed in claim 4, wherein the heatmap and the training sample picture have a correspondence relationship, and the size is 1:1 or an equal ratio mapping according to an actual required ratio.
6. The method for assisting in training a classification network by means of heatmap as claimed in claim 1, wherein said step 3 further comprises:
whether a specific area exists in the picture is detected by the classification network, the edge information of the specific area is important information, and the background and other contents except the edge of the specific area are ignorable information, so that the area of the edge of the specific area is defined as a common feature area and is a pattern feature which needs to be learned by the neural network, and therefore, the edge area of the specific area in the sample picture is mapped to the heatmap, namely, the area containing the important information is highlighted, and if no specific area exists in the picture, the heatmap pattern is a full black picture.
7. The method of claim 1, wherein the step 4 of creating output layer labels further comprises:
whether a picture contains a specific area or not is judged, the picture containing the specific area can be classified into one type, and the picture not containing the specific area can be classified into another type.
8. The method as claimed in claim 1, wherein the heatmap network is a right branch after branching, the classification network is a left branch after branching, the output of the first half network is used as the input of the heatmap network and the classification network of the last two networks, the heatmap network and the classification network are parallel relations, and are independent of each other, and do not affect each other, and during the network learning process, the heatmap network and the classification network affect the weight data of the first half network together.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010662452.1A CN113919491A (en) | 2020-07-10 | 2020-07-10 | Method for assisting in training classification network by means of heatmap |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010662452.1A CN113919491A (en) | 2020-07-10 | 2020-07-10 | Method for assisting in training classification network by means of heatmap |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113919491A true CN113919491A (en) | 2022-01-11 |
Family
ID=79232237
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010662452.1A Pending CN113919491A (en) | 2020-07-10 | 2020-07-10 | Method for assisting in training classification network by means of heatmap |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113919491A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107610141A (en) * | 2017-09-05 | 2018-01-19 | 华南理工大学 | A kind of remote sensing images semantic segmentation method based on deep learning |
| CN110245721A (en) * | 2019-06-25 | 2019-09-17 | 深圳市腾讯计算机***有限公司 | Training method, device and the electronic equipment of neural network model |
| EP3547211A1 (en) * | 2018-03-30 | 2019-10-02 | Naver Corporation | Methods for training a cnn and classifying an action performed by a subject in an inputted video using said cnn |
-
2020
- 2020-07-10 CN CN202010662452.1A patent/CN113919491A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107610141A (en) * | 2017-09-05 | 2018-01-19 | 华南理工大学 | A kind of remote sensing images semantic segmentation method based on deep learning |
| EP3547211A1 (en) * | 2018-03-30 | 2019-10-02 | Naver Corporation | Methods for training a cnn and classifying an action performed by a subject in an inputted video using said cnn |
| CN110245721A (en) * | 2019-06-25 | 2019-09-17 | 深圳市腾讯计算机***有限公司 | Training method, device and the electronic equipment of neural network model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111950649B (en) | Attention mechanism and capsule network-based low-illumination image classification method | |
| CN112116599B (en) | Sputum smear tubercle bacillus semantic segmentation method and system based on weak supervised learning | |
| CN111507993A (en) | Image segmentation method and device based on generation countermeasure network and storage medium | |
| CN111738110A (en) | Remote sensing image vehicle target detection method based on multi-scale attention mechanism | |
| CN112381097A (en) | Scene semantic segmentation method based on deep learning | |
| CN114627360A (en) | Substation equipment defect identification method based on cascade detection model | |
| CN112183635A (en) | Method for realizing segmentation and identification of plant leaf lesions by multi-scale deconvolution network | |
| CN112488025B (en) | Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion | |
| CN113284100B (en) | Image quality evaluation method based on recovery image to mixed domain attention mechanism | |
| CN112418351B (en) | Zero sample learning image classification method based on global and local context sensing | |
| CN114419323B (en) | Cross-modal learning and domain self-adaptive RGBD image semantic segmentation method | |
| CN111008633A (en) | License plate character segmentation method based on attention mechanism | |
| CN114816997B (en) | Defect prediction method based on graph neural network and bidirectional GRU feature extraction | |
| CN111768792A (en) | Audio steganalysis method based on convolutional neural network and domain confrontation learning | |
| CN112784909B (en) | Image classification and identification method based on self-attention mechanism and self-adaptive sub-network | |
| CN112364979B (en) | GoogLeNet-based infrared image identification method | |
| Saravanarajan et al. | Improving semantic segmentation under hazy weather for autonomous vehicles using explainable artificial intelligence and adaptive dehazing approach | |
| CN111126561B (en) | Image processing method based on multi-path parallel convolution neural network | |
| CN113919491A (en) | Method for assisting in training classification network by means of heatmap | |
| CN115601744B (en) | License plate detection method for vehicle body and license plate with similar colors | |
| CN117058716A (en) | Cross-domain behavior recognition method and device based on image pre-fusion | |
| CN111798966A (en) | Scanning method for assisting in identifying digital pathological section based on artificial intelligence | |
| CN116524261A (en) | Image classification method and product based on multi-mode small sample continuous learning | |
| CN116912872A (en) | Drawing identification method, device, equipment and readable storage medium | |
| CN113537325B (en) | Deep learning method for image classification based on extracted high-low layer feature logic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |