CN114283287A - Robust field adaptive image learning method based on self-training noise label correction - Google Patents
Robust field adaptive image learning method based on self-training noise label correction Download PDFInfo
- Publication number
- CN114283287A CN114283287A CN202210221128.5A CN202210221128A CN114283287A CN 114283287 A CN114283287 A CN 114283287A CN 202210221128 A CN202210221128 A CN 202210221128A CN 114283287 A CN114283287 A CN 114283287A
- Authority
- CN
- China
- Prior art keywords
- source domain
- domain
- pseudo
- samples
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012937 correction Methods 0.000 title claims abstract description 10
- 230000003044 adaptive effect Effects 0.000 title claims description 14
- 238000009826 distribution Methods 0.000 claims abstract description 38
- 238000005070 sampling Methods 0.000 claims abstract description 15
- 230000006870 function Effects 0.000 claims abstract description 9
- 238000013136 deep learning model Methods 0.000 claims abstract description 4
- 239000000203 mixture Substances 0.000 claims description 15
- 238000002372 labelling Methods 0.000 claims description 11
- 238000000638 solvent extraction Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 230000010354 integration Effects 0.000 claims description 4
- 239000003550 marker Substances 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 238000002360 preparation method Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 claims description 2
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000013508 migration Methods 0.000 description 3
- 230000005012 migration Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 101100001669 Emericella variicolor andD gene Proteins 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a robust field self-adaptive image learning method based on self-training noise label correction, which comprises the following steps: acquiring a source domain, a target domain image set and a source domain low-quality label; initializing various parameters; building a model and a loss function; sequentially inputting the image sets of the source domain and the target domain into two mark classifiers; the two mark classifiers detect noise for the opposite side before each iterative training, then predict pseudo marks again for the noise source domain sample and the target domain sample, and perform rebalance sampling to participate in the next iterative training; inputting a target domain pseudo label set into target domain specific network training; and after the training is finished, performing a class prediction task on the target domain image by using the target domain specific classifier. Aiming at the problem that the category distribution of a source domain is inconsistent with that of a target domain, the method adopts a rebalanced sampling pseudo-mark sample mechanism to ensure that the sampling proportion of each category of the source domain and the target domain is consistent, and the accuracy of a deep learning model on the target domain is improved.
Description
Technical Field
The invention relates to a robust field self-adaptive image learning method based on self-training noise label correction.
Background
The traditional supervised learning needs a large amount of images and accurate labeling information, however, in practical situations, the collection of a large amount of accurate labels requires extremely high cost, so that the labels contain a large amount of noise. The unsupervised field self-adaptation is applied to another data set with different but similar data distribution by utilizing a training model on the data set with the accurate label in a migration mode, wherein the data set with the accurate label is a source field, and the data set without the label is a target field. Although conventional domain adaptation solves the problem of lack of supervisory information on the target domain, it ignores the problem of significant cost for source domain marker acquisition, and therefore their performance is severely degraded when there is noise in the labeled information in the source domain.
Disclosure of Invention
The invention provides a robust field adaptive image learning method based on self-training noise label correction, and aims to further improve the accuracy of the field adaptive image learning method on a target field when the problems of labeled noise and inconsistent source field and target field category distribution (category distribution offset) are faced.
In order to achieve the purpose, the invention adopts the following technical scheme:
a robust field self-adaptive image learning method based on self-training noise label correction comprises the following steps:
Wherein,D s representing source domain raw images acquired by a network platformx si And its corresponding source domain low quality markComposed source domain primitive numbersAccording to the data set, the data of the data set,N s representing a source domain raw data setD s Total number of medium samples;
D t representing the original image only by the target domainx ti The composed target domain data set is composed of,N t to representD t The number of medium target domain samples;
step 2, initializing various parameters including iteration timest=0, the secondtRound false mark thresholdγ tPre-training parametersN warm ;
And 3, building a deep learning model and a loss function, comprising the following steps: feature extractorGTwo tag classifierC 1、C 2Target domain specific classifierC t Cross entropy loss functionL ce And a consistency loss functionL sp ;
Step 4, the original data set of the source domainD s Source domain original image inx si Input feature extractorGTo extract features in an imagef si =G(x si ) Then extracting the source domain featuresf si And low quality marks corresponding to the original image of the source domainIs sent into two mark classifiersC 1、C 2In the middle ofWarm upTraining, trainingN warm A wheel;
step 5. in the detection stage, two mark classifiers are usedC 1、C 2Sequentially and respectively detecting the marking noise in the source domain, and mutually taking the original data sets of the source domain as the opposite sideD s Partitioning into clean source domain samplesAnd noise source domain samples;
Step 6, two mark classifiers are usedC 1、C 2In turn, are noise source domain samplesAnd a target domain data setD t Performing class prediction on each sample, and taking the prediction class asAndD t a pseudo-label for each sample in the set;
according toγ tFor the noise source field sample in step 6And a target domain data setD t In each pseudo-labeled sample, the sample sampling ratio of each category isγ t / KTo obtain a pseudo label setD sp ={,};
Wherein,Kthe number of categories is indicated and the number of categories,a set of pseudo-annotations of the source domain is represented,representing a target domain pseudo label set;
step 8, in the training stage, clean source domain samples are sampledAnd pseudo label setD sp Image input feature extractorGExtracting features, and inputting the extracted features and corresponding labels into two label classifiersC 1、C 2Carrying out supervision training in sequence;
for clean source domain samplesOptimizing cross entropy loss functionL ce For pseudo label setsD sp Optimizing consistency loss functionsL sp To update the feature extractorGAnd two tag classifiersC 1、C 2;
Step 9, pseudo labeling set of target domainImage input feature extractorGExtracting features, and inputting the extracted features and corresponding pseudo labels into a target domain specific classification networkC t Optimizing cross entropy loss functionL ce To updateG、C t ;
Step 10, judging the current iteration timestWhether or not the maximum number of iterations has been reachedT;
If the current number of iterationstNot reaching the maximum number of iterationsTThen, the self-training is continued by returning to the step 5,t=t+1, and updateγ t=γ 0+0.05*t(ii) a Wherein,γ 0a pseudo-mark threshold representing initialization; otherwise, go to step 11;
step 11, after the model training is finished, executing a classification prediction task, firstly using a feature extractorGExtracting features from the target domain image and inputting the extracted featuresC t And (5) performing category prediction.
The invention has the following advantages:
as mentioned above, the invention relates to a robust domain adaptive image learning method based on self-training noise label correction, which is a robust domain adaptive method based on self-training noise detection and label rebalance definition, and performs alternate training by designing a label network and a target domain specific network, wherein the label network can effectively filter label noise in a source domain through the label network, and performs rebalance sampling on pseudo label samples to solve the problem of class distribution shift (label distribution shift), and the target domain pseudo label samples are utilized to train the target domain specific network, thereby obtaining the classification capability on a target domain to realize knowledge transfer from the source domain to the target domain, and meanwhile, the invention utilizes rebalance pseudo label sampling to solve the problem of inconsistent class distribution of the source domain and the target domain, further improving under the noise condition, robustness of domain adaptive methods.
Drawings
FIG. 1 is a flow chart of a robust domain adaptive image learning method based on self-training noise label correction in an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an integral model in an embodiment of the invention;
FIG. 3 is a flow chart illustrating the filtering of source domain marker noise according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating the generation and rebalancing of sampling pseudo labels according to an embodiment of the present invention.
Detailed Description
The noun explains:
weak enhancement means that a picture is simply turned over and translated; the strong enhancement means that two kinds of strong transformation (translation, clipping, rotation, turning, image compression and the like) and disturbance (overexposure, contrast enhancement, sharpening, black and white processing, tone separation, Gaussian blur and the like) with different degrees are randomly added to one picture so as to cause the picture to be seriously distorted.
The basic concept of the invention is as follows: in the noise field self-adaptation, the labels of a source domain are obtained through a low-cost labeling platform, some labels are wrong, and in order to solve the problem of labeling noise in the source domain, the method utilizes the characteristic that a deep neural network is used for fitting a noise sample after preferentially fitting a clean sample, and a sample with small loss is considered as a clean sample through analyzing each loss so as to achieve the purpose of dividing a data set into the clean source domain sample and the noise source domain sample. In order to solve the problem of inconsistent edge distribution (covariate shift) of samples in a source domain and a target domain, the invention uses the thought of self-training for reference, firstly, a model is pre-trained by using the source domain samples with accurate marks, then, pseudo marks are gradually added into the target domain samples through a marking network to participate in the training of the whole model, in addition, a target domain specific classifier which is only trained by using the target domain pseudo mark samples is additionally trained by the invention to capture the specific distinguishing characteristics of the target domain, and finally, the transfer of knowledge from the source domain to the target domain is gradually realized. In addition, the invention considers the problem of label distribution shift (label distribution shift), namely that samples of each category of a source domain and a target domain may have different quantities, and severe category imbalance may exist in the domain, so the invention adopts the rebalance sampling to the pseudo label samples to keep the training of the source domain and the target domain on each category consistent, concretely, the invention constructs two label classifiers to filter the label noise in the source domain, utilizes the characteristic of fitting the noise samples after the deep neural network is preferentially fitted to the clean samples to cluster the sample loss, the cross entropy loss generated by the clean samples is small, the cross entropy loss generated by the noise samples is large, the sample loss can be fitted to the Gaussian mixed distribution composed of relatively high loss distribution and relatively low loss distribution, and the probability that the loss of each sample belongs to the low loss distribution is the probability that the sample is the clean sample, and in order to reduce the continuous accumulation of the errors of a single model, the two mark classifiers divide the clean and noise samples for each other, the noise source domain sample and the target domain sample are utilized together in each iteration, and a reliable pseudo label is selected for the noise source domain sample and the target domain sample by judging whether the strength enhancement is consistent or not to construct a pseudo label data set. Aiming at the problem of inconsistent distribution of categories of different domains, the method carries out rebalance sampling on the pseudo labels. By the method, the problem of noise field self-adaptation and the phenomenon of category distribution offset (the category distribution offset is embodied in that each category in each domain is unbalanced and the unbalanced effect of a source domain sample and a target domain sample on each category is different) can be effectively solved, and the robustness of the field self-adaptation method under the noise condition is further improved.
The invention is described in further detail below with reference to the following figures and detailed description:
as shown in fig. 1, the robust domain adaptive image learning method based on self-training noise label correction includes the following steps:
Wherein,D s representing source domain raw images acquired by a network platformx si And its corresponding source domain low quality markThe original data set of the source domain of the composition,N s representing a source domain raw data setD s Total number of samples in (c).
D t Representing the original image only by the target domainx ti The composed target domain data set is composed of,N t to representD t Number of medium target domain samples.
The source domain and target domain images can easily acquire a batch of high-quality images through an internet platform.
The annotation of the source domain image set can be obtained through a network public annotation platform, such as a crowdsourcing annotation platform; the markers obtained by such low cost are not completely accurate and therefore contain a large number of noisy markers.
When the real mark of the image has the problem of unbalanced category, the collected mark is also unbalanced, and therefore, the source domain category distribution and the target domain category distribution may have a non-uniform phenomenon.
In such cases, it is extremely challenging to implement knowledge migration from the source domain to the target domain.
Step 2, initializing various parameters including iteration timest=0, the secondtRound false mark thresholdγ tPre-training parametersN warm . Wherein,γ tthe hyperparameter set by people represents that the upper limit of the number of the pseudo mark samples is set in each round of iterative training.
And 3, building a deep learning model and a loss function, comprising the following steps: feature extractorGTwo tag classifierC 1、C 2Target domain specific classifierC t Cross entropy loss functionL ce And a consistency loss functionL sp 。
As shown in fig. 2, the entire frame comprises four parts: feature extractor (encoder) composed of deep convolutional networkGAnd a classifier composed of three fully-connected layers and a BN (Batch Normalization) layerC 1、C 2、C t 。
Wherein,Gis a deep convolutional neural network used for extracting the characteristics of the sample and mapping the image to a high-dimensional characteristic space.In order to be a clean source domain sample,andand respectively providing pseudo label sets of the source domain and the target domain which can be used after screening.
Two tag classifiers in this exampleC 1、C 2The effects of (A) are as follows:
1. in the detection stage, the original data set of the source domain is mainly subjected toD s The noise mark in the signal is detected and filtered, andD s partitioning into clean source domain samplesAnd noise source domain samplesAnd for detected noise source field samplesAnd target domain samplesD t Predicting pseudo-marks, respectively obtained by re-balancing the samples、And participating in the next round of self-training.
C t For target domain specific classifiers, only target domain pseudo-labeled sets are usedTraining to obtain the characteristic features of the target domain without being interfered by the source domain features, and finally obtaining the target domain with good classification performanceC t And realizing the migration of the knowledge from the source domain to the target domain.
For a single picturex,GIt is first mapped into a high-dimensional depth feature space and thenC 1、C 2AndC t predicting probability of the mapped features, mapping toKProbability vector of dimensionP model And,Kis the number of categories.
Step 4, the original data set of the source domainD s Source domain original image inx si Input feature extractorGTo extract features in an imagef si =G(x si ) Then the original image of the source domain is processedx si Extracted source domain featuresf si And low quality marks corresponding to the original image of the source domainIs sent into two mark classifiersC 1、C 2In the middle ofWarm upTraining, trainingN warm And (4) wheels.
Warm upThe training means that before formal self-training, the original data set of the source domain is utilizedD s By optimizing the cross entropy loss functionL ce Simple prediction to update a modelAnd training, namely fitting the characteristics of the noise sample after preferentially fitting the clean sample according to the deep neural network, so that the model can be used as pre-training of the following self-training (step 5-step 10) through initial training, fit the clean mark without fitting the noise mark, and play a role in initializing the network parameters of the whole model.
Step 5. in the detection stage, two mark classifiers are usedC 1、C 2Sequentially and respectively detecting the marking noise in the original data set of the source domain, and mutually taking the original data set of the source domain as the opposite sideD s Into clean source domain samples and noise source domain samples.
Step 5.1, initialize noise filtering thresholdτ=0.6。
Wherein,τthe hyper-parameter set artificially represents the boundary of whether the noise is judged to be a clean sample or not at each noise detection.
Step 5.2. willD s Original image of medium source domainx si Input feature extractorGExtracting features to obtainf si_t =G(x si )。
Wherein,f si_t is as followstIn the self-training process, the original image of the source domainx si Input feature extractorGThe features obtained later, features extracted from each round of training in the training processf si_t All are different.
Step 5.3, all source domain original images in the step 5.2 are processedx si Extracted featuresf si_t All input to two mark classifiers in turnC 1、C 2In using two tag classifiersC 1、C 2Respectively aiming at the original image of each source domain in sequencex si Extracted featuresf si_t Performing class prediction to obtain corresponding class prediction resultC 1(f si_t )、C 2(f si_t )。
Wherein,C 1(f si_t ) To useC 1For each source domain original imagex si Extracted featuresf si_t Performing a result of the category prediction;C 2(f si_t ) To useC 2For each source domain original imagex si Extracted featuresf si_t And (5) performing a category prediction result.
Using cross entropy loss functionL ce Calculating the class prediction results respectivelyC 1(f si_t )、C 2(f si_t ) And source domain original imagex si Corresponding source domain low quality markCalculating cross entropy loss to obtain cross entropy lossl i1,l i2} i Ns 。
Wherein,l i1is composed ofC 1(f si_t ) And source domain original imagex si Corresponding source domain low quality markThe cross-entropy loss of (a) is,l i2is composed ofC 2(f si_t ) And source domain original imagex si Corresponding source domain low quality markCross entropy loss of (2).
Step 5.4. Cross entropy loss for all source domain original images respectively by means of Gaussian Mixture Model (GMM)l i1,l i2} i Ns Fitting a Gaussian mixture distribution to obtain the Gaussian mixture distributionp(g|l i1)、p(g| l i2)。
Each gaussian mixture distribution consists of two gaussian distributions representing a distribution with less loss and a distribution with greater loss, respectively.
Wherein,p(g| l i1)、p(g| l i2) Respectively two mark classifiersC 1、C 2For each source domain original imagex si ∈D s Source domain low-quality mark corresponding to category predictionAnd calculating the probability of low loss after cross entropy loss.
Fitting principle of Gaussian mixture distribution:
according to the characteristics of fitting noise samples after the clean samples are fitted preferentially by the deep neural network, the sample loss is clustered, the cross entropy loss generated by the clean samples is small, the cross entropy loss generated by the noise samples is large, and the loss of all samples can be fitted into Gaussian mixture distribution consisting of relatively high loss distribution and relatively low loss distribution. Based on the property that the deep neural network preferentially fits the clean mark, we consider that the source domain raw image with less loss is more likely to belong to a clean sample.
Step 5.5, Gaussian mixture distributionp(g| l i1)、p(g| l i2) Greater than a noise filtering thresholdτThe source domain original image of (2) is used as a clean source domain sample, and the rest source domain original image is used as a noise source domain sample.
To reduce the constant accumulation of errors in the individual models themselves, two label classifiers are used before the next round of self-trainingC 1、C 2Partitioning clean source domain samples for each otherAnd noise source domain samplesI.e. mark classifierC 1To the data partitioning result ofC 2Use, mark classifierC 2To the data partitioning result ofC 1The preparation is used.
FIG. 3 shows two tag classifiersC 1、C 2How to filter the markup noise in the source domain.
Step 6, two mark classifiers are usedC 1、C 2In turn, are noise source domain samplesAnd a target domain data setD t Performing class prediction on each sample, and taking the prediction class as the classAndD t pseudo-labeling of each sample.
To overcome the problem of inconsistent distribution of source domain and target domain categories, the method is based onγ tFor in step 6AndD t is re-balanced sampled for each pseudo-labeled sample in a sample sampling ratio of each class ofγ t / KTo obtain a pseudo label setD sp ={,}。
Wherein,Kthe number of categories is indicated and the number of categories,a set of pseudo-annotations of the source domain is represented,representing a target domain pseudo label set.
FIG. 4 is a flow chart illustrating the generation and rebalancing of sampling pseudo labels in an embodiment of the present invention.
Step 6.1. noise Source Domain samplesThe original mark is no longer trusted and is sampled with the target domainD t (itself, a non-labeled sample) constituting a set of non-labeled samples x b |x b ∈∪D t }. Wherein,x b representing samples in the unlabeled sample set.
Step 6.2. samplex b Strongly enhanced version ofA(x b ) And weakly enhanced versionsα(x b ) Respectively input two tag classifiersC 1、C 2. Wherein,A(x b ) Is to the samplex b Adding transformation and disturbance of different degrees to severely distort the picture;α(x b ) Is to the samplex b Obtained through vertical turning and translation processing.
Step 6.3. two tag classifiersC 1、C 2For weak enhanced versions respectivelyα(x b ) And strongly enhanced versionsA(x b ) Carrying out classified prediction to obtain four prediction results which are respectivelyp 1 α 、p 2 α 、p 1 A 、p 2 A . Wherein,p 1 α for mark classifierC 1For weak enhanced versionα(x b ) Performing a prediction result of classification prediction;p 2 α for mark classifierC 2For weak enhanced versionα(x b ) And (5) carrying out a prediction result of classification prediction.p 1 A For mark classifierC 1For weak enhanced versionA (x b ) Performing a prediction result of classification prediction;p 2 A for mark classifierC 1For weak enhanced versionA (x b ) And (5) carrying out a prediction result of classification prediction. For the predicted resultp 1 α 、p 2 α Integration, i.e. addition of probability prediction vectors, is finally obtainedp α . For the predicted resultp 1 A 、p 2 A Integration, i.e. addition of probability prediction vectors, is finally obtainedp A . Wherein,p α 、p A representing two label classifiers separatelyC 1、C 2For weak enhanced versionα(x b ) Enhanced versionA(x b ) I.e. the confidence on each class.
Step 6.4. calculate allKClass-class maximum probability prediction class:,. Wherein,means all ofKThe maximum probability in a class predicts the class.
Representing two tag classifiersC 1、C 2For weak enhanced versionα(x b ) Comprehensive pretreatmentA measured false mark;representing two tag classifiersC 1、C 2For strong enhancement versionA(x b ) The predicted pseudo-tags are synthesized.
Will be provided with=The prediction pseudo mark is used as a preliminary reliable pseudo mark, and the prediction probability, namely the confidence degree ranking on the prediction category is ranked from high to low; according to the confidence ranking, from each category, the sampling equal proportion isγ t / KObtaining a source domain pseudo-label set by using the pseudo-label samples with the highest quantity and confidence coefficientAnd target domain pseudo label set。
For example: for all the pseudo-labeled samples labeled as class k, the method ranks from high to low according to the prediction confidence of the pseudo-labeled samples on the class k, and then the sampling proportion isγ t / KA pseudo-labeled sample on class k.
Wherein,、is the prediction probability vector of the marking network to all images with the size ofN×K。
And selecting the category with the highest prediction probability as a pseudo label for the single label-free sample.
because the first round of iterative training model is not trained to be mature and cannot predict accurate pseudo labels for label-free samples, the invention utilizes a source domain pseudo label setTraining target domain specific classifiers as a first iterationC t The pseudo-tagged data set of (1).
Step 8, in the training stage, clean source domain samples are sampledAnd pseudo label setD sp Image input feature extractorGExtracting features, and inputting the extracted features and corresponding labels into two label classifiersC 1、C 2And (5) carrying out supervision training in sequence.
Wherein for clean source domain samplesOptimizing cross entropy loss functionL ce For pseudo label setsD sp Optimizing consistency loss functionsL sp To update the feature extractorGTwo tag classifiersC 1、C 2。
Updating two tag classifiersC 1、C 2For clean source domain samples: (,)∈(ii) a Two tag classifierC 1、C 2To pairHas a prediction probability of() Cross entropy loss functionL ce The concrete form of (A) is as follows:
wherein,Brepresenting the number of clean source field samples per small batch.Representing the source domain original image in a clean source domain sample,representing the label corresponding to the source domain image in the clean source domain sample.
Each small batch of clean source domain samples refers to clean source domain samplesAll samples in the network are evenly divided into subsets with equal sizes, and each small batch of clean source domain samples are sequentially selected and sent into the network for training.
When the data size is too large, all data cannot be sent to the network for training at one time, but all samples are divided into subsets with the same size during each iteration training, and then the subsets are sequentially selected from the subsets and sent to the network for training, so that each iteration is not a loss function for calculating all data, but a small batch of loss functions.
Therefore, the training speed of the model and the convergence speed of the model can be accelerated.
Two tag classifierC 1、C 2Re-predicting pseudo-labels for noise source domain samples and target domain samplesAnd forming a pseudo-labeled sample set (x b , ) Using a consistency loss functionL sp The optimization model is specifically in the form of:
wherein,A(x b ) For each samplex b A strongly enhanced version of (a).P model (A(x b ) ) a presentation marker classifierC 1OrC 2For the samplex b Strongly enhanced version ofA(x b ) The class prediction of (1).H(,P model (A(x b ) ) is cross entropy, as follows:
step 9, pseudo labeling set of target domainIs input to the feature extractorGExtracting features, and inputting the extracted features and corresponding pseudo labels into a target domain specific classification networkC t Optimizing cross entropy loss functionL ce To updateG、C t 。
Specific classifiers in a target domainC t In the training phase, the loss function used in this embodiment is cross entropy lossL ce OptimizingL ce To update the target domain specific classifierC t 。
Updating target domain specific classifiersC t Using only target domain pseudo-labeled samples: (,)∈To train, target domain specific classifiersC t To pairHas a prediction probability ofThen it crosses the entropy loss functionL ce The concrete form is as follows:
wherein,representing a set of pseudo labels of a target domainThe original image of the medium target domain is,representing two tag classifiersC 1、C 2Is composed ofThe added pseudo-mark is added to the mark,Brepresenting the number of pseudo-labeled samples per small batch of the target domain.
Each small batch of target domain pseudo-labeled samples refers to pseudo-labeling target domainsAll samples in the target domain are uniformly divided into subsets with equal sizes, and each small batch of target domain pseudo-labeled samples are sequentially selected and sent into a network for training.
When the data size is too large, all data cannot be sent to the network for training at one time, but all samples are divided into subsets with the same size during each iteration training, and then the subsets are sequentially selected from the subsets and sent to the network for training, so that each iteration is not a loss function for calculating all data, but a small batch of loss functions.
Therefore, the training speed of the model and the convergence speed of the model can be accelerated.
Step 10, judging the current iteration timestWhether or not the maximum number of iterations has been reachedT(ii) a If it istNot reaching the maximum number of iterationsTThen, the self-training is continued by returning to the step 5,t=t+1, and updateγ t=γ 0+0.05*t(ii) a Otherwise, go to step 11;
wherein,γ 0indicating an initialized pseudo-mark threshold.
Step 11, after the model training is completed, a feature extractor capable of extracting reliable features on the target domain is obtainedGAnd target domain specific classifier capable of performing reliable classification performance on target domain samplesC t 。
Performing a final classification prediction task, the method of the invention uses a feature extractorGExtracting features from the target domain image and inputting the extracted features into a target domain specific classification networkC t And performing category prediction.
It should be understood, however, that the description herein of specific embodiments is not intended to limit the invention to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. The robust field self-adaptive image learning method based on self-training noise label correction is characterized by comprising the following steps of:
Wherein,D s representing source domain raw images acquired by a network platformx si And its corresponding source domain low quality markThe original data set of the source domain of the composition,N s representing a source domain raw data setD s Total number of medium samples;
D t representing origin only by target domainStarting imagex ti The composed target domain data set is composed of,N t to representD t The number of medium target domain samples;
step 2, initializing various parameters including iteration timest=0, the secondtRound false mark thresholdγ tPre-training parametersN warm ;
Step 3, building a deep learning model and a loss function, including a feature extractorGTwo tag classifierC 1、C 2Target domain specific classifierC t Cross entropy loss functionL ce And a consistency loss functionL sp ;
Step 4, the original data set of the source domainD s Source domain original image inx si Input feature extractorGTo extract features in an imagef si =G(x si ) Then extracting the source domain featuresf si And low quality marks corresponding to the original image of the source domainIs sent into two mark classifiersC 1、C 2In the middle ofWarm upTraining, trainingN warm A wheel;
step 5. in the detection stage, two mark classifiers are usedC 1、C 2Sequentially and respectively detecting the marking noise in the source domain, and mutually taking the original data sets of the source domain as the opposite sideD s Partitioning into clean source domain samplesAnd noise source domain samples;
Step 6, two mark classifiers are usedC 1、C 2In turn, are noise source domain samplesAnd a target domain data setD t Performing class prediction on each sample, and taking the prediction class asAndD t a pseudo-label for each sample in the set;
according toγ tFor the noise source field sample in step 6And a target domain data setD t In each pseudo-labeled sample, the sample sampling ratio of each category isγ t / KTo obtain a pseudo label setD sp ={,};
Wherein,Kthe number of categories is indicated and the number of categories,a set of pseudo-annotations of the source domain is represented,representing a target domain pseudo label set;
step 8, in the training stage, clean source domain samples are sampledAnd pseudo label setD sp Image input feature extractorGExtracting features, and inputting the extracted features and corresponding labels into two label classifiersC 1、C 2Carrying out supervision training in sequence;
for clean source domain samplesOptimizing cross entropy loss functionL ce For pseudo label setsD sp Optimizing consistency loss functionsL sp To update the feature extractorGAnd two tag classifiersC 1、C 2;
Step 9, pseudo labeling set of target domainImage input feature extractorGExtracting features, and inputting the extracted features and corresponding pseudo labels into a target domain specific classification networkC t Optimizing cross entropy loss functionL ce To updateG、C t ;
Step 10, judging the current iteration timestWhether or not the maximum number of iterations has been reachedT;
If the current number of iterationstNot reaching the maximum number of iterationsTThen, the self-training is continued by returning to the step 5,t=t+1, and updateγ t=γ 0+0.05*t(ii) a Wherein,γ 0a pseudo-mark threshold representing initialization; otherwise, go to step 11;
step 11, after the model training is finished, executing a classification prediction task, firstly using a feature extractorGExtracting features from the target domain image and inputting the extracted featuresC t And (5) performing category prediction.
2. The robust domain adaptive image learning method as recited in claim 1,
the step 5 specifically comprises the following steps:
step 5.1, initialize noise filtering thresholdτ=0.6;
Step 5.2. willD s Original image of medium source domainx si Input feature extractorGExtracting features to obtainf si_t =G(x si );
Wherein,f si_t is as followstIn the self-training process, the original image of the source domainx si Input feature extractorGThe characteristics obtained later;
step 5.3, all source domain original images in the step 5.2 are processedx si Extracted featuresf si_t All input to two mark classifiers in turnC 1、C 2In using two tag classifiersC 1、C 2Respectively aiming at the original image of each source domain in sequencex si Extracted featuresf si_t Performing class prediction to obtain corresponding class prediction resultC 1(f si_t )、C 2(f si_t );
Wherein,C 1(f si_t ) Is composed ofC 1For each source domain original imagex si Extracted featuresf si_t Performing a result of the category prediction;C 2(f si_t ) Is composed ofC 2For each source domain original imagex si Extracted featuresf si_t Performing a result of the category prediction;
using cross entropy loss functionL ce Calculating the class prediction results respectivelyC 1(f si_t )、C 2(f si_t ) And source domain original imagex si Corresponding source domain low quality markObtaining cross entropy lossl i1,l i2} i Ns ;
Wherein,l i1to representC 1(f si_t ) And source domain original imagex si Corresponding source domain low quality markThe cross-entropy loss of (a) is,l i2to representC 2(f si_t ) And source domain original imagex si Corresponding source domain low quality markCross entropy loss of (d);
step 5.4, respectively carrying out comparison on all source domain original images by means of Gaussian mixture modelsx si Is a cross entropy lossl i1,l i2} i Ns Fitting a Gaussian mixture distribution to obtain the Gaussian mixture distributionp(g| l i1)、p(g| l i2);
Each Gaussian mixture distribution consists of two Gaussian distributions which respectively represent a distribution with low loss and a distribution with high loss;
wherein,p(g| l i1)、p(g| l i2) Respectively two mark classifiersC 1、C 2For each source domain original imagex si ∈D s Source domain low-quality mark corresponding to category predictionCalculating the probability of low loss after cross entropy loss;
step 5.5, Gaussian mixture distributionp(g| l i1)、p(g| l i2) Middle greater than noise filtering thresholdτAs a clean source domain sampleThe residual source domain original image is used as a noise source domain sample;
3. The robust domain adaptive image learning method as recited in claim 2,
in step 6, the specific steps of generating the pseudo mark are as follows:
step 6.1. noise Source Domain samplesBook (I)And target domain samplesD t Forming a set of unmarked samples x b |x b ∈∪D t };
Wherein,x b representing samples in the unlabeled sample set;
step 6.2. samplex b Strongly enhanced version ofA(x b ) And weakly enhanced versionsα(x b ) Are inputted separatelyC 1、C 2;
Wherein,A(x b ) Is a samplex b Adding transformation and disturbance of different degrees to severely distort the picture;
α(x b ) Is to the samplex b Obtained by vertical turning and translation processing;
step 6.3. two tag classifiersC 1、C 2For weak enhanced versions respectivelyα(x b ) And strongly enhanced versionsA(x b ) Carrying out classified prediction to obtain four prediction results which are respectivelyp 1 α 、p 2 α 、p 1 A 、p 2 A ;
Wherein,p 1 α for mark classifierC 1For weak enhanced versionα(x b ) Performing a prediction result of classification prediction;p 2 α for mark classifierC 2For weak enhanced versionα(x b ) Performing a prediction result of classification prediction;
p 1 A for mark classifierC 1For weak enhanced versionA (x b ) Performing a prediction result of classification prediction;p 2 A for mark classifierC 1For weak enhanced versionA (x b ) Performing a prediction result of classification prediction;
for the predicted resultp 1 α 、p 2 α Integration, i.e. addition of probability prediction vectors, is finally obtainedp α ;
For the predicted resultp 1 A 、p 2 A Integration, i.e. addition of probability prediction vectors, is finally obtainedp A ;
Wherein,p α 、p A representing two label classifiers separatelyC 1、C 2For weak enhanced versionα(x b ) Enhanced versionA(x b ) The overall predicted class outcome of (a), i.e., confidence in each class;
representing two tag classifiersC 1、C 2For weak enhanced versionα(x b ) Pseudo-labeling for comprehensive prediction;
representing two tag classifiersC 1、C 2For strong enhancement versionA(x b ) Pseudo-labeling for comprehensive prediction;
will be provided with=The prediction pseudo mark is used as a preliminary reliable pseudo mark, and the prediction probability, namely the confidence degree ranking on the prediction category is ranked from high to low; according to the confidence ranking, from each category, the sampling equal proportion isγ t / KObtaining a source domain pseudo-label set by using the pseudo-label samples with the highest quantity and confidence coefficientAnd target domain pseudo label set。
4. The robust domain adaptive image learning method as recited in claim 1,
the feature extractorGIs a feature extractor consisting of a deep convolutional network; two tag classifierC 1、C 2And a target domain specific classifierC t All are classifiers composed of three fully-connected layers and a normalization layer.
5. The robust domain adaptive image learning method as recited in claim 3,
in the training phase, optimizingL ce 、L sp To update two tag classifiersC 1、C 2;
Updating two tag classifiersC 1、C 2For clean source domain samples: (,)∈Two tag classifierC 1、C 2For images in clean source domain samplesHas a prediction probability of() Cross entropy loss functionL ce The concrete form of (A) is as follows:
wherein,Brepresenting the number of clean source domain samples per small batch,Krepresenting a category total;
each small batch of clean source domain samples refers to clean source domain samplesIn which all samples are evenly divided into subsets each having equal sizeThen, sequentially selecting each small batch of clean source domain samples, and sending the small batch of clean source domain samples into a network for training;
representing the source domain original image in a clean source domain sample,a mark corresponding to the source domain image in the clean source domain sample is represented;
two tag classifierC 1、C 2Re-predicting pseudo-labels for noise source domain samples and target domain samplesAnd forming a pseudo-labeled sample set (x b , ) Using a consistency loss functionL sp The optimization model is specifically in the form of:
wherein,A(x b ) For each samplex b A strongly enhanced version of (c);
P model (A(x b ) ) a presentation marker classifierC 1OrC 2For the samplex b Strongly enhanced version ofA(x b ) Predicting the category of (1);
6. the robust domain adaptive image learning method as recited in claim 3,
in the training phase, optimizingL ce To update the target domain specific classifier;
updating target domain specific classifiersC t Using only target domain pseudo-labeled samples: (,)∈To train, target domain specific classifiersC t For imagesHas a prediction probability ofThen it crosses the entropy loss functionL ce The concrete form is as follows:
wherein,representing a set of pseudo labels of a target domainA middle target domain original image;representing two tag classifiersC 1、C 2Is composed ofThe added pseudo-mark is added to the label,Brepresenting the number of pseudo-labeled samples per small batch of the target domain,Krepresenting a category total;
each small batch of target domain pseudo-labeled samples refers to pseudo-labeling target domainsAll samples in the target domain are uniformly divided into subsets with equal sizes, and each small batch of target domain pseudo-labeled samples are sequentially selected and sent into a network for training.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210221128.5A CN114283287B (en) | 2022-03-09 | 2022-03-09 | Robust field adaptive image learning method based on self-training noise label correction |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210221128.5A CN114283287B (en) | 2022-03-09 | 2022-03-09 | Robust field adaptive image learning method based on self-training noise label correction |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114283287A true CN114283287A (en) | 2022-04-05 |
CN114283287B CN114283287B (en) | 2022-05-06 |
Family
ID=80882312
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210221128.5A Active CN114283287B (en) | 2022-03-09 | 2022-03-09 | Robust field adaptive image learning method based on self-training noise label correction |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114283287B (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114723994A (en) * | 2022-04-18 | 2022-07-08 | 中国矿业大学 | Hyperspectral image classification method based on dual-classifier confrontation enhancement network |
CN114998602A (en) * | 2022-08-08 | 2022-09-02 | 中国科学技术大学 | Domain adaptive learning method and system based on low confidence sample contrast loss |
CN115331088A (en) * | 2022-10-13 | 2022-11-11 | 南京航空航天大学 | Robust learning method based on class labels with noise and imbalance |
CN115331065A (en) * | 2022-10-13 | 2022-11-11 | 南京航空航天大学 | Robust noise multi-label image learning method based on decoder iterative screening |
CN115496972A (en) * | 2022-11-15 | 2022-12-20 | 杭州涿溪脑与智能研究所 | Industrial field self-adaption method based on data mixing |
CN117132841A (en) * | 2023-10-26 | 2023-11-28 | 之江实验室 | Domain self-adaptive image classification method and device for conservation and progression |
CN117848588A (en) * | 2024-03-07 | 2024-04-09 | 青岛天一红旗软控科技有限公司 | Mechanical balance self-adaptive test method |
CN118097319A (en) * | 2024-04-29 | 2024-05-28 | 南京航空航天大学 | Image classification method with unseen class and noise labels in online stream data |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110837850A (en) * | 2019-10-23 | 2020-02-25 | 浙江大学 | Unsupervised domain adaptation method based on counterstudy loss function |
US20200130177A1 (en) * | 2018-10-29 | 2020-04-30 | Hrl Laboratories, Llc | Systems and methods for few-shot transfer learning |
CN111368977A (en) * | 2020-02-28 | 2020-07-03 | 交叉信息核心技术研究院(西安)有限公司 | Enhanced data enhancement method for improving accuracy and robustness of convolutional neural network |
WO2020159638A1 (en) * | 2019-01-30 | 2020-08-06 | Hrl Laboratories, Llc | System and method for unsupervised domain adaptation via sliced-wasserstein distance |
CN112232241A (en) * | 2020-10-22 | 2021-01-15 | 华中科技大学 | Pedestrian re-identification method and device, electronic equipment and readable storage medium |
CN112287994A (en) * | 2020-10-26 | 2021-01-29 | 北京嘀嘀无限科技发展有限公司 | Pseudo label processing method, device, equipment and computer readable storage medium |
CN112906606A (en) * | 2021-03-05 | 2021-06-04 | 南京航空航天大学 | Domain-adaptive pedestrian re-identification method based on mutual divergence learning |
US20210217405A1 (en) * | 2020-01-10 | 2021-07-15 | International Business Machines Corporation | Implementing a domain adaptive semantic role labeler |
CN113344044A (en) * | 2021-05-21 | 2021-09-03 | 北京工业大学 | Cross-species medical image classification method based on domain self-adaptation |
CN113378981A (en) * | 2021-07-02 | 2021-09-10 | 湖南大学 | Noise scene image classification method and system based on domain adaptation |
CN113807420A (en) * | 2021-09-06 | 2021-12-17 | 湖南大学 | Domain self-adaptive target detection method and system considering category semantic matching |
-
2022
- 2022-03-09 CN CN202210221128.5A patent/CN114283287B/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20200130177A1 (en) * | 2018-10-29 | 2020-04-30 | Hrl Laboratories, Llc | Systems and methods for few-shot transfer learning |
WO2020159638A1 (en) * | 2019-01-30 | 2020-08-06 | Hrl Laboratories, Llc | System and method for unsupervised domain adaptation via sliced-wasserstein distance |
CN113316790A (en) * | 2019-01-30 | 2021-08-27 | 赫尔实验室有限公司 | System and method for unsupervised domain adaptation via SLICED-WASSERSTEIN distance |
CN110837850A (en) * | 2019-10-23 | 2020-02-25 | 浙江大学 | Unsupervised domain adaptation method based on counterstudy loss function |
US20210217405A1 (en) * | 2020-01-10 | 2021-07-15 | International Business Machines Corporation | Implementing a domain adaptive semantic role labeler |
CN111368977A (en) * | 2020-02-28 | 2020-07-03 | 交叉信息核心技术研究院(西安)有限公司 | Enhanced data enhancement method for improving accuracy and robustness of convolutional neural network |
CN112232241A (en) * | 2020-10-22 | 2021-01-15 | 华中科技大学 | Pedestrian re-identification method and device, electronic equipment and readable storage medium |
CN112287994A (en) * | 2020-10-26 | 2021-01-29 | 北京嘀嘀无限科技发展有限公司 | Pseudo label processing method, device, equipment and computer readable storage medium |
CN112906606A (en) * | 2021-03-05 | 2021-06-04 | 南京航空航天大学 | Domain-adaptive pedestrian re-identification method based on mutual divergence learning |
CN113344044A (en) * | 2021-05-21 | 2021-09-03 | 北京工业大学 | Cross-species medical image classification method based on domain self-adaptation |
CN113378981A (en) * | 2021-07-02 | 2021-09-10 | 湖南大学 | Noise scene image classification method and system based on domain adaptation |
CN113807420A (en) * | 2021-09-06 | 2021-12-17 | 湖南大学 | Domain self-adaptive target detection method and system considering category semantic matching |
Non-Patent Citations (8)
Title |
---|
CHEN WEI等: "CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning", 《COMPUTER VISION AND PATTERN RECOGNITION》 * |
QIANGGUO JIN等: "Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images", 《ARXIV》 * |
XIYU YU等: "Label-Noise Robust Domain Adaptation", 《PROCEEDINGS OF THE 37TH INTERNATIONAL CONFERENCE ON MACHINE LEARNING》 * |
YITING CHENG等: "Dual Path Learning for Domain Adaptation of Semantic Segmentation", 《COMPUTER VISION FOUNDATION》 * |
付家慧: "深度迁移学习算法及其应用研究", 《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》 * |
吴涛: "领域自适应方法在图像分类上的研究与应用", 《万方在线》 * |
杨艺等: "一种基于自训练的众包标记噪声纠正算法", 《自动化学报》 * |
程康明等: "一种自训练框架下的三优选半监督回归算法", 《智能***学报》 * |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114723994A (en) * | 2022-04-18 | 2022-07-08 | 中国矿业大学 | Hyperspectral image classification method based on dual-classifier confrontation enhancement network |
CN114998602A (en) * | 2022-08-08 | 2022-09-02 | 中国科学技术大学 | Domain adaptive learning method and system based on low confidence sample contrast loss |
CN114998602B (en) * | 2022-08-08 | 2022-12-30 | 中国科学技术大学 | Domain adaptive learning method and system based on low confidence sample contrast loss |
CN115331088A (en) * | 2022-10-13 | 2022-11-11 | 南京航空航天大学 | Robust learning method based on class labels with noise and imbalance |
CN115331065A (en) * | 2022-10-13 | 2022-11-11 | 南京航空航天大学 | Robust noise multi-label image learning method based on decoder iterative screening |
CN115496972A (en) * | 2022-11-15 | 2022-12-20 | 杭州涿溪脑与智能研究所 | Industrial field self-adaption method based on data mixing |
CN115496972B (en) * | 2022-11-15 | 2023-04-07 | 杭州涿溪脑与智能研究所 | Industrial field self-adaption method based on data mixing |
CN117132841A (en) * | 2023-10-26 | 2023-11-28 | 之江实验室 | Domain self-adaptive image classification method and device for conservation and progression |
CN117132841B (en) * | 2023-10-26 | 2024-03-29 | 之江实验室 | Domain self-adaptive image classification method and device for conservation and progression |
CN117848588A (en) * | 2024-03-07 | 2024-04-09 | 青岛天一红旗软控科技有限公司 | Mechanical balance self-adaptive test method |
CN117848588B (en) * | 2024-03-07 | 2024-06-04 | 青岛天一红旗软控科技有限公司 | Mechanical balance self-adaptive test method |
CN118097319A (en) * | 2024-04-29 | 2024-05-28 | 南京航空航天大学 | Image classification method with unseen class and noise labels in online stream data |
Also Published As
Publication number | Publication date |
---|---|
CN114283287B (en) | 2022-05-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114283287B (en) | Robust field adaptive image learning method based on self-training noise label correction | |
WO2021134871A1 (en) | Forensics method for synthesized face image based on local binary pattern and deep learning | |
CN110443143B (en) | Multi-branch convolutional neural network fused remote sensing image scene classification method | |
CN108648191B (en) | Pest image recognition method based on Bayesian width residual error neural network | |
CN109345508B (en) | Bone age evaluation method based on two-stage neural network | |
CN111160553B (en) | Novel field self-adaptive learning method | |
CN108399406A (en) | The method and system of Weakly supervised conspicuousness object detection based on deep learning | |
CN113361566B (en) | Method for migrating generative confrontation network by using confrontation learning and discriminant learning | |
CN108416382B (en) | Web image training convolutional neural network method based on iterative sampling and one-to-many label correction | |
CN110781921A (en) | Depth residual error network and transfer learning-based muscarinic image identification method and device | |
CN115331088B (en) | Robust learning method based on class labels with noise and imbalance | |
CN112115967B (en) | Image increment learning method based on data protection | |
CN112733965A (en) | Label-free image classification method based on small sample learning | |
CN115331065B (en) | Robust noise multi-label image learning method based on decoder iterative screening | |
CN113780242A (en) | Cross-scene underwater sound target classification method based on model transfer learning | |
CN116206185A (en) | Lightweight small target detection method based on improved YOLOv7 | |
CN113297988A (en) | Object attitude estimation method based on domain migration and depth completion | |
CN114139616A (en) | Unsupervised domain adaptive target detection method based on uncertainty perception | |
CN111028203A (en) | CNN blind image quality evaluation method based on significance | |
CN112883931A (en) | Real-time true and false motion judgment method based on long and short term memory network | |
CN113869463B (en) | Long tail noise learning method based on cross enhancement matching | |
CN113989256A (en) | Detection model optimization method, detection method and detection device for remote sensing image building | |
CN114049503A (en) | Saliency region detection method based on non-end-to-end deep learning network | |
CN107392242B (en) | Cross-domain picture classification method based on homomorphic neural network | |
CN115984653B (en) | Construction method of dynamic intelligent container commodity identification model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |