CN113420786A - Semi-supervised classification method for feature mixed image - Google Patents

Semi-supervised classification method for feature mixed image Download PDF

Info

Publication number
CN113420786A
CN113420786A CN202110599001.2A CN202110599001A CN113420786A CN 113420786 A CN113420786 A CN 113420786A CN 202110599001 A CN202110599001 A CN 202110599001A CN 113420786 A CN113420786 A CN 113420786A
Authority
CN
China
Prior art keywords
image
mixed
feature
semi
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110599001.2A
Other languages
Chinese (zh)
Inventor
姚博
文成林
徐晓滨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110599001.2A priority Critical patent/CN113420786A/en
Publication of CN113420786A publication Critical patent/CN113420786A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a semi-supervised classification method of feature mixed images, and relates to the field of computer image processing. The invention comprises the following steps: manufacturing a semi-supervised data set; performing feature fusion augmentation processing on the label-free data; adding a consistency measurement method for the network, and designing a new objective function; and finally training the network and testing the classification effect of the network on the feature mixed image. According to the method, a multi-image mixed sample capacity increasing technology is integrated into an expansion framework of a semi-supervised model, the diversity of training data is increased, various types of noise interference is introduced, the generalization capability of the model is improved, the robustness of the model is enhanced, and the accurate classification of the characteristic mixed images is realized.

Description

Semi-supervised classification method for feature mixed image
Technical Field
The invention belongs to the field of computer image processing, and particularly relates to a semi-supervised classification method of feature mixed images
Background
Semi-supervised learning is an important research direction in the field of current machine learning, in the field of semi-supervised learning, training data usually contains a large amount of label-free data, the training requirements of a model cannot be met only by augmenting labeled data, and particularly when the number of labeled data is small, the model is easy to be over-fitted. In order to solve the problem, the main stream semi-supervised learning model depends on a consistency regularization term in an objective function to carry out consistency measurement on data distribution before and after augmentation, effective information is extracted from non-label data, and the generalization performance and the robustness of the model can be effectively improved due to the diversity and the complexity of augmentation types.
In the field of semi-supervised classification, a consistency regularization method mainly uses a RandAugment augmentation framework, a strategy space with only a single image augmentation method is defined, and each image randomly selects the augmentation method to generate corresponding augmentation data. In addition to single image augmentation, the Mixup mixed data augmentation method realizes data augmentation by modeling different types of data, and achieves remarkable performance improvement in the image classification field under supervised learning. Mixup, as a hybrid augmentation method, typically generates a hybrid image containing multiple target features, which is often found in artificially modified images or images of hybrid species. Because the mixed image does not belong to any determined known class, the strategy space of RandAugment cannot be directly added for consistency measurement, which brings difficulty to the classification of the mixed image under semi-supervision.
Disclosure of Invention
The invention aims to provide a semi-supervised classification method of a feature mixed image, which realizes the rapid and accurate classification of the feature mixed image when only a small amount of labeled data exists.
The method comprises the following specific steps:
step 1, collecting and sorting images of objects to be classified as a data set, and only carrying out partial labeling on the images of each category.
And 2, performing multiple feature mixing augmentation on each unmarked image in the data set.
And 3, designing a consistency measurement method related to the unmarked image and the augmented image for the convolutional network, and improving the objective function of the network.
And 4, inputting the data set containing the marked image, the unmarked image and the augmented image into the improved convolution network, and training until the network is converged.
And 5, taking the mixed image with the multi-target characteristics as a test set to obtain an image classification result.
Further, in step 1, in order to ensure that the class probabilities in the network training are equal, the number of labeled images of each class is consistent, that is, the class labels are balanced.
Further, in step 2, in the case of the non-labeled image enhancement, a Mixup mixed data enhancement method is used in addition to the usual single image enhancement such as stretching, inversion, and cropping, and a mixed image having a plurality of kinds of features is generated.
Further, step 3 includes the following sub-steps:
3.1 select KL divergence as the consistency measure representing the deviation of the model prediction output before and after input augmentation.
3.2 the improved network objective function not only includes the cross entropy of the traditional marked image, but also includes the consistency measurement of the unmarked image.
Further, in step 4, in each step of network training, the annotated image, the unlabeled image and the augmented image are simultaneously input into the network, the annotated image is subjected to feature extraction, then cross entropy is calculated according to the annotation, consistency measurement of network output is carried out on the unlabeled image and the augmented image, and the unlabeled image and the augmented image participate in gradient calculation together.
Further, in step 5, the network outputs class probability to the multi-feature mixed image which is difficult to classify, which indicates the class source of the mixed feature, and obtains the classification result of the mixed image.
The invention has the beneficial effects that:
(1) and the generalization capability of the model is improved. The multi-image mixed sample capacity increasing technology is integrated into an expansion framework of a semi-supervised model, the diversity of training data is increased, and the generalization capability of the model is improved.
(2) And the robustness of the model is enhanced. The mixed image introduces various types of noise interference on the augmented various data samples, and the sensitivity of the model to interfered sample data is reduced.
(3) And the classification capability of the mixed image is improved. The characteristic mixed sample generated in multiple modes establishes an evaluation method for the consistency measurement with the original image, and the evaluation method is incorporated into the semi-supervised learning model, so that the generalization capability of the model is further improved, and the classification capability of the mixed image is improved.
Drawings
FIG. 1 is a flow chart of classification of feature-mixed images under semi-supervised learning;
fig. 2 is a schematic diagram of information storage of a Mixup mixed image in the present invention.
Detailed Description
The invention provides a semi-supervised classification method of feature mixed images, which is further explained by combining the attached drawings.
As shown in fig. 1, the present invention provides a semi-supervised classification method for feature-mixed images, comprising the following steps:
step 1, manufacturing a semi-supervised data set:
a CIFAR-10 data set is selected as a classification object, 50000 training pictures are divided into 10 classes, and each class is provided with 5000 labeled images. Randomly selecting N in each categoryulAnd (4) keeping the labels for each picture, deleting the labels for all the rest training data, and only keeping the images as a semi-supervised training set.
Step 2, feature mixing and augmentation:
and performing multiple feature mixing augmentation on each unmarked image in the data set. The Mixup mixed data augmentation method achieves the purpose of data augmentation by interpolating images and labels, wherein x represents the images, and y represents the labels, and the method is specifically as follows:
Figure BDA0003092123560000031
where λ represents the mixing coefficient, obtained by sampling the Beta distribution with parameters α, β. (x)1,y1) And (x)2,y2) Are two samples drawn at random.
The label-free data in the method has no label for mixing, and the image can only be interpolated to generate mixed tensor mixedxTherefore, the labeling process of the feature-blended image is put into the consistency regularization of step 3.
Another difference between Mixup multi-image mixing and single-image augmentation is that two or more images need to be enumerated and recorded for further fusion and consistency measurement. In order to ensure the randomness of the method, double-ended queues (Queue) are adopted to record data participating in multi-image mixing.
As shown in fig. 2, the Queue (Queue) stores the original image for blending internally, if at the previous time, the original image x1With the original image x2Generating a mixed image A, and storing the mixed image A as mixed information A; the original image x will be at the current moment1Pop-up queue, original image x2Adding, generating a mixed image B, and storing as mixed information B; the next moment will be the original image x2Pop-up queue, original image x3Enqueuing mixes and stores the information. And continuing until all input images needing hybrid enhancement are traversed. The mixed information records the mixed image and the two original image information before mixing.
Step 3 consistency measurement:
the consistency measurement method aims to add a regularization term to an objective function of the network and prevent the occurrence of network overfitting under the condition of a small amount of labeled data. The main idea of consistency regularization is that for an input, even if slightly disturbed, its prediction should be consistent, i.e. the network prediction output is unchanged before and after the same object is augmented. This is consistent with the goal of most objective functions, i.e., it is desirable that predictions be consistent with labels. Consistency measurement and KL divergence measurement mode D are carried out on data before and after augmentationKLMay be represented by the following formula:
Figure BDA0003092123560000041
wherein
Figure BDA0003092123560000042
Is the augmented data of sample x, fθ(.) represents the model under training, with the model parameters being θ. Mixing tensor mixedxPassed to model f in trainingθ(.) can derive an output tensor mixedout. Is trainedIn the process, the output tensor mixed of the modeloutRespectively carrying out consistency measurement on the original image before mixing, carrying out weighted summation on a plurality of measurement values according to a mixing coefficient lambda of the image, and adding the sum as a regularization item into a target function, wherein the concrete expression is as follows:
dkl＝λDKL(x1||mixedout)+(1-λ)DKL(x2||mixedout)(3)
the objective function L of the final network is as follows:
L＝H(Nl)+dkl(Nul)(4)
wherein N islRepresents the annotation data, H (N)l) Cross entropy, N, representing annotation dataulRepresenting unlabelled data, dkl(Nul) A consistency regularization term representing unlabeled data.
And 4, step 4: network training
The network adopts WRN-28-10 realized based on TensorFlow, and compared with a deep residual error network (ResNet), the WRN increases the width of convolution cores in a residual error block (Resblock), namely the characteristic number of convolution layers is increased, so that the WRN has a better effect than the traditional ResNet under the same depth, and meanwhile, the network depth is reduced, and the network convergence speed is accelerated.
And 5: classifying feature-blended images
And taking the mixed image with the multi-target characteristics as a test set, outputting the class probability to the multi-characteristic mixed image which is difficult to classify by the network, indicating the class source of the mixed characteristics, and obtaining the classification result of the mixed image.
According to the method, the multi-image mixing and amplification technology is integrated into the semi-supervised field, so that the diversity of training data is increased, the generalization capability of the model is improved, meanwhile, various noises are introduced in the training stage, and the robustness of the model is enhanced. The characteristic mixed sample generated in multiple modes establishes an evaluation method for the consistency measurement with the original image, and the evaluation method is incorporated into the semi-supervised learning model, so that the generalization capability of the model is further improved, and the classification capability of the mixed image is improved. The method and the device realize rapid and accurate classification of the feature mixed image when only a small amount of labeled data exists.

Claims (7)

1. A semi-supervised classification method of feature-mixed images is characterized by comprising the following steps:
step 1, collecting and sorting images of objects to be classified as a data set, and only carrying out partial labeling on the images of each category;
step 2, performing multiple feature mixing augmentation on each unmarked image in the data set;
step 3, designing a consistency measurement method related to the unmarked image and the augmented image for the convolution network, and improving the objective function of the network;
step 4, inputting a data set containing an annotated image, a non-annotated image and an augmented image into the improved convolution network, and training until the network is converged;
and 5, taking the mixed image with the multi-target characteristics as a test set to obtain an image classification result.
2. The semi-supervised classification method for feature mixed images as claimed in claim 1, wherein in step 1, the number of labeled images in each category is consistent, i.e. category labeling is balanced.
3. The semi-supervised classification method for feature-mixed images as claimed in claim 1, wherein in step 2, in addition to the commonly used single image expansion of stretching, inversion and cropping, a Mixup data expansion method is used to generate mixed images with various types of features.
4. The semi-supervised classification method for feature-mixed images according to claim 1, wherein in step 3, KL divergence is selected as a consistency measure representing a deviation of model prediction outputs before and after input augmentation.
5. The semi-supervised classification method for feature mixed images as claimed in claim 1, wherein in step 3, the improved network objective function includes not only cross entropy of traditional labeled images but also consistency measure of unlabeled images.
6. The semi-supervised classification method for the feature mixed image as recited in claim 1, wherein in step 4, in each step of network training, the annotated image, the unlabeled image and the augmented image are simultaneously input into the network, the annotated image is subjected to feature extraction, then cross entropy is calculated according to the annotation, and consistency measurement of network output is performed on the unlabeled image and the augmented image, and the unlabeled image and the augmented image participate in gradient calculation.
7. The semi-supervised classification method for the feature-mixed image as claimed in claim 1, wherein in step 5, the network outputs class probability indicating the class source of the mixed features to the multi-feature mixed image which is difficult to classify, and obtains the classification result of the mixed image.
CN202110599001.2A 2021-05-31 2021-05-31 Semi-supervised classification method for feature mixed image Pending CN113420786A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110599001.2A CN113420786A (en) 2021-05-31 2021-05-31 Semi-supervised classification method for feature mixed image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110599001.2A CN113420786A (en) 2021-05-31 2021-05-31 Semi-supervised classification method for feature mixed image

Publications (1)

Publication Number Publication Date
CN113420786A true CN113420786A (en) 2021-09-21

Family

ID=77713336

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110599001.2A Pending CN113420786A (en) 2021-05-31 2021-05-31 Semi-supervised classification method for feature mixed image

Country Status (1)

Country Link
CN (1) CN113420786A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272777A (en) * 2022-09-26 2022-11-01 山东大学 Semi-supervised image analysis method for power transmission scene

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111222648A (en) * 2020-01-15 2020-06-02 深圳前海微众银行股份有限公司 Semi-supervised machine learning optimization method, device, equipment and storage medium
CN111695466A (en) * 2020-06-01 2020-09-22 西安电子科技大学 Semi-supervised polarization SAR terrain classification method based on feature mixup
CN111723209A (en) * 2020-06-28 2020-09-29 上海携旅信息技术有限公司 Semi-supervised text classification model training method, text classification method, system, device and medium
CN112381148A (en) * 2020-11-17 2021-02-19 华南理工大学 Semi-supervised image classification method based on random regional interpolation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111222648A (en) * 2020-01-15 2020-06-02 深圳前海微众银行股份有限公司 Semi-supervised machine learning optimization method, device, equipment and storage medium
CN111695466A (en) * 2020-06-01 2020-09-22 西安电子科技大学 Semi-supervised polarization SAR terrain classification method based on feature mixup
CN111723209A (en) * 2020-06-28 2020-09-29 上海携旅信息技术有限公司 Semi-supervised text classification model training method, text classification method, system, device and medium
CN112381148A (en) * 2020-11-17 2021-02-19 华南理工大学 Semi-supervised image classification method based on random regional interpolation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
VIKAS VERMA,ALEX LAMB,ET AL.: "Interpolation Consistency Training for Semi-Supervised Learning", 《ARXIV》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272777A (en) * 2022-09-26 2022-11-01 山东大学 Semi-supervised image analysis method for power transmission scene
CN115272777B (en) * 2022-09-26 2022-12-23 山东大学 Semi-supervised image analysis method for power transmission scene

Similar Documents

Publication Publication Date Title
Noroozi et al. Representation learning by learning to count
CN110119753B (en) Lithology recognition method by reconstructed texture
CN113487629B (en) Image attribute editing method based on structured scene and text description
CN111369003B (en) Method and device for determining fidelity of quantum bit reading signal
CN112528058B (en) Fine-grained image classification method based on image attribute active learning
CN113762138A (en) Method and device for identifying forged face picture, computer equipment and storage medium
CN115587985A (en) Method for dividing cell nucleus of histopathology image and normalizing dyeing style
CN114724218A (en) Video detection method, device, equipment and medium
CN113989556A (en) Small sample medical image classification method and system
CN112163106A (en) Second-order similarity perception image Hash code extraction model establishing method and application thereof
CN116823793A (en) Device defect detection method, device, electronic device and readable storage medium
CN115249313A (en) Image classification method based on meta-module fusion incremental learning
CN113420786A (en) Semi-supervised classification method for feature mixed image
CN116912674A (en) Target detection method and system based on improved YOLOv5s network model under complex water environment
CN116958809A (en) Remote sensing small sample target detection method for feature library migration
Volokitin et al. Efficiently detecting plausible locations for object placement using masked convolutions
Shan et al. Semantic segmentation based on deep convolution neural network
CN114155388B (en) Image recognition method and device, computer equipment and storage medium
Kundu et al. Optimal Machine Learning Based Automated Malaria Parasite Detection and Classification Model Using Blood Smear Images.
CN113435389B (en) Chlorella and golden algae classification and identification method based on image feature deep learning
CN115223033A (en) Synthetic aperture sonar image target classification method and system
CN110489584B (en) Image classification method and system based on dense connection MobileNet model
CN113724261A (en) Fast image composition method based on convolutional neural network
CN114359786A (en) Lip language identification method based on improved space-time convolutional network
CN111340111B (en) Method for recognizing face image set based on wavelet kernel extreme learning machine

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210921

WD01 Invention patent application deemed withdrawn after publication