CN113298184B - Sample extraction and expansion method and storage medium for small sample image recognition - Google Patents
Sample extraction and expansion method and storage medium for small sample image recognition Download PDFInfo
- Publication number
- CN113298184B CN113298184B CN202110687034.2A CN202110687034A CN113298184B CN 113298184 B CN113298184 B CN 113298184B CN 202110687034 A CN202110687034 A CN 202110687034A CN 113298184 B CN113298184 B CN 113298184B
- Authority
- CN
- China
- Prior art keywords
- sample
- cluster
- category
- samples
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
A sample extraction and expansion method and a storage medium for small sample image recognition belong to the technical field of image processing. The method aims to solve the problem that an error sample is generated possibly caused in a mode of generating a new sample in a small sample image identification process. The invention firstly provides a sample extraction method based on feature reconstruction to solve the problem of feature loss of a small sample data set, and realizes extraction of a typical small sample data set from a large sample data set from the aspect of data features. According to the method, the mass center of the large sample data is used as the standard of extraction measurement, so that the extracted typical small sample data set has more comprehensive characteristics and more stable effect. The invention also provides a sample expansion method based on the deformation information, and the extracted typical small sample data set is expanded into a new large sample data set by using the inter-data deformation information of the same type heterogeneous clusters in the optimal division. The method is mainly used for sample extraction and expansion of small sample image recognition.
Description
Technical Field
The invention relates to an image sample extraction and expansion method and a storage medium. Belongs to the technical field of image processing.
Background
Neural networks often require large amounts of data to accomplish effective training. In the low data area, the training effect and generalization capability of the network may not perform well.
In order to ensure the identification effect of the network, a small sample learning method based on data expansion is basically adopted at present, and most of the data expansion utilizes the thought of generating countermeasures to generate data or directly uses difference information among similar data sets to realize data enhancement. However, this method does not consider the problem of whether the features of the small sample data set are complete before expansion, which may result in the expanded data still lacking important features. The problem that whether the use of the difference information is reasonable is not considered, and if irrelevant difference information is used on small sample data, a new sample is generated, and an error sample is generated. The training by using the newly generated wrong sample can affect the training effect of the network, which can not improve the recognition effect, and even can lead to the deterioration of the recognition effect.
Disclosure of Invention
The invention aims to solve the problem that a new sample generation mode is adopted in a small sample image recognition process, which may cause the generation of wrong samples, and further influences the training effect.
A sample extraction method for small sample image recognition comprises the following steps:
s1, calculating the central support point C of each category of the image large sample data set k ；
S2, dividing the sample data of the same kind into clusters with dynamic quantity from the angle of the sample characteristics, and calculating the centroid of each cluster according to each division condition;
calculating a new central point of the category under the dividing condition according to the mass center of each cluster; calculating the sum of the same cluster error and the centroid error to be used as the total error of the category under the dividing condition; selecting the condition of the minimum error as the optimal division mode of the category;
and S3, uniformly and quantitatively extracting samples from each optimally divided cluster according to the sample distribution, and taking the samples as a small sample data set for characteristic reconstruction.
Further, calculating the central support point C of each category of the image large sample data set k Comprises the following steps:
aiming at the image large sample data set, calculating the average vector of all the feature vectors in each category in the large sample data set, and taking the average vector as the central support of the categoryPoint C k ：
S k Sample set, S, representing the Kth category k L represents the number of samples in the sample set of the kth category; x is the number of i Denotes belonging to S k Sample data of a sample set, y i Is a sample x i A corresponding label;
is an embedding function.
Further, step S2 includes the steps of:
2.1, clustering the characteristic vectors of the samples in each category in the large sample data set into a dynamic number of clusters, wherein m clusters are arranged;
2.2 calculating the centroid C 'of each new cluster' k_m_n All samples under the cluster are expressed as X ∈ S k_m_ ；C′ k_m_n Representing the centroid of the nth cluster after the data of the class k is divided into m clusters; s k_m_n Representing a set of all samples in the nth cluster after the samples in the category k are divided into m clusters;
2.3, through centroid C 'of each new cluster' k_m_n Calculating the new centroid C 'of the category under the division' k_m ；
Wherein, C' k_m Representing a new central support point of the category formed by the mass center of each cluster after dividing the samples of the Kth category into m clusters;
2.4 by calculating the cluster error L sce And error of center of mass L ce As the total error L of the class in the case of this division s (ii) a And selecting the condition with the minimum total error as the optimization division of the category data.
Further, the air conditioner is provided with a fan,the total error L s ＝L sve +L ce (ii) a Wherein
The cluster error is the sum of the distances between all samples in each cluster and the centroid of the cluster:
the centroid error is the distance between the new centroid of the class constructed by each cluster of centroids and the centroid of the class in the large sample:
L ce ＝C′ k_m -C k 。
a storage medium having stored therein at least one instruction which is loaded and executed by a processor to implement a method of sample extraction for small sample image recognition.
The sample expansion method for small sample image recognition is characterized in that a sample extraction method for small sample image recognition is utilized to determine a small sample data set, expansion is carried out based on the small sample data set, and the sample expansion comprises a training stage and a generation stage; a training stage:
typical small sample datasets are divided into the form of sample pairs (x' m ，x′ n ) Wherein x' m And x' n Sample data belonging to different clusters in the same category; namely x' m ∈Q k_i_s ，x′ n ∈Q k_i_t S ≠ t, where Q k_i_s And Q k_i_t Respectively representing the sample data of the s-th cluster and the t-th cluster after the data of the class k are optimally divided into the i clusters;
the sample expansion process is completed by using a sample expansion model, wherein the sample expansion model is a network structure comprising an inference model, a generation model, a discrimination model and a classifier;
inferring model pairs of samples (x ') for different clusters of the same type' m ，x′ n ) Coding and learning samples x 'of different clusters of the same type' m And x' n Deformation information Z ═ E (x' m ，x′ n )；
Generation model utilizes deformation information Z and input samples x 'in potential space' n To generate
Is shown as
The discriminant model is trained to distinguish between true sample pairs (x) c ，x′ m ) Or reconstructing the sample pair
The network can reconstruct the sample more accurately through antagonism training;
the classifier is used for reconstructing samples
Classifying;
a generation stage:
sample pairs (x ') of different clusters of the same kind' m ，x′ n ) As input to the model, the deformation information Z and the sample x 'are then' u ∈Q k_i_t As an input of the generation model, finally generating a new sample for the category; by constantly changing sample pairs (x ') of inference model inputs' m ，x′ n ) Or sample x 'of the input of the generative model is changed' u To generate more new samples for that category.
Further, the change infers a sample pair (x ') of model inputs' m ，x′ n ) Need to guarantee the sample pair (x' m ，x′ n ) Are pairs of samples trained in the training process.
Further, the changing generates a sample x 'of an input of a model' u Is guaranteed to be x' u And sample x 'of the generative model input' n And (5) the same kind and the same cluster.
Further, the total loss of the sample expansion model in the training phase is as follows:
L＝L mse +L D +L cls
wherein L is mse To generate losses, L D To identify loss, L cls Is a classification loss.
A storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement a sample augmentation method for small sample image recognition.
Has the advantages that:
the invention firstly provides a sample extraction method based on feature reconstruction to solve the problem of feature loss of a small sample data set, and realizes the extraction of a typical small sample data set from a large sample data set from the aspect of data features. According to the method, the mass center of the large sample data is used as the standard of extraction measurement, so that an extracted typical small sample data set has more comprehensive characteristics and more stable effect; meanwhile, a typical small sample data set with complete characteristics is provided for data expansion. The invention further provides a sample expansion method based on deformation information, and the extracted typical small sample data set is expanded into a new large sample data set by using the inter-data deformation information of the same type of heterogeneous clusters in the optimal division. The method extracts and uses the deformation information more reasonably, so that the generated data is more accurate. The invention ensures that the generated image data is more accurate, so that the network identification effect can be effectively improved when the image identification network is further trained.
Drawings
FIG. 1 is an overview of a sample extraction and expansion method;
FIG. 2 is a model of antagonism data enhancement based on feature reconstruction and deformation information;
FIG. 3 is a graph of data volume versus model accuracy;
fig. 4 is a sample labeled 5 in the MNIST dataset as an example to visualize the centroids of the clusters in the optimal partition cluster;
fig. 5 is a new sample generated on the MNIST digital data set (labeled 3) and the emist character data set (labeled 46) using the network structure in the SECIDI method.
Detailed Description
The first embodiment is as follows:
the embodiment is a sample extraction method for identifying small sample images, which is essentially a sampling method based on feature reconstruction, and the main idea is to optimally divide large sample data by an unsupervised fuzzy clustering method and then reconstruct a typical small sample data set. Specifically, the method comprises the following steps:
firstly, calculating the central support point C of each category of the image large sample dataset k 。
Secondly, the similar sample data is divided into clusters with dynamic quantity from the angle of sample characteristics, and the mass center of each cluster is calculated according to each division condition.
Then, a new center point of the category under the division condition is calculated according to the mass center of each cluster. And the sum of the cluster error and the centroid error is calculated as the total error of the class in the division case. And selecting the condition of the minimum error as the optimal division mode of the category.
Finally, from the optimally partitioned cluster for each class, typical small sample data is reconstructed.
The specific reconstruction process comprises the following steps:
1. class mean (centroid) calculation:
for a large sample data set of an image, calculating an average vector of all feature vectors in each category in the large sample data set, and taking the average vector as a central support point C of the category k . The categories are classified according to the labels of the images, that is, the categories refer to data sets classified according to the labels, such as handwritten number recognition, where all labels are 0 are of one category, labels are 1 are of one category, and so on.
S k Sample set, S, representing the Kth category k L represents the number of samples in the sample set of the kth category; x is the number of i Denotes belonging to S k Sample data of a sample set, y i Is a sample x i A corresponding label;
to embed a function, an embedding function refers in some embodiments to a convolutional layer of a convolutional neural network.
2. Optimal division of homogeneous data:
2.1, clustering the feature vectors of the samples in each category in the large sample dataset into a dynamic number of clusters (assuming m clusters).
2.2, calculating the centroid C 'of each new cluster' k_m_n (representing the centroid of the nth cluster after dividing the data of class k into m clusters), all samples under that cluster are represented as X ∈ S k_m_n (S k_m_n Representing the set of all samples in the nth cluster after the samples in the category k are divided into m clusters).
2.3, center of mass C 'through each new cluster' k_m_n Calculating the new centroid C 'of the category under the division' k_m 。
Wherein, C' k_m Indicating a new central support point of the class consisting of the centroid of each cluster after the samples of the kth class are divided into m clusters.
2.4 by calculating the error L of the same cluster sce And error of center of mass L ce As the total error L of the class under such partitioning s . And selecting the condition with the minimum total error as the optimization division of the category data.
The cluster error refers to the sum of the distances between all samples in each cluster and the centroid of the cluster:
the centroid error refers to the distance between a new centroid of the class constructed by the centroids of the clusters and the centroid of the class in the large sample:
L ce ＝C′ k_m -C k (4)
total error of
L s ＝L sce +L ce (5)
3. Reconstructing a small sample dataset:
and uniformly and quantitatively extracting samples from each optimally divided cluster according to the sample distribution of the clusters to serve as a small sample data set for characteristic reconstruction.
The second embodiment is as follows:
the embodiment is a storage medium, and at least one instruction is stored in the storage medium and loaded and executed by a processor to implement a sample extraction method for small sample image recognition.
The third concrete implementation mode:
the embodiment is a sample expansion method for identifying small sample images, and essentially is a sample expansion method based on intra-class deformation information. The main idea is to generate and expand a new data set of the same type by learning deformation information among data of the same type and then applying the deformation information to other samples of the same type. In order to better learn deformation information among the same-class data, through a sample extraction method of feature reconstruction, the first embodiment provides a typical small sample data set with complete features for data expansion, each category in the typical small sample data set is divided into multiple clusters with feature differences, and then expansion is performed on the basis of the small sample data set determined by the first embodiment. Fig. 1 is an overview diagram of a sample extraction and expansion method based on feature reconstruction and deformation information, where (a) represents a large sample data set, (b) represents a division of the large sample data set by a co-clustering error and a centroid error, where dotted blank dots represent the centroids of clusters, and solid blank dots represent the centroids of the classes, (c) represents an optimal division of the large sample data set, (d) represents a typical small sample data set extracted by the feature reconstruction method, (e) represents arrows represent deformation information among clusters of the small sample data set, and (f) represents a new large sample data set after expansion based on the deformation information, where double-colored dots represent newly generated samples.
Specifically, the expansion of the sample includes two stages: a training phase and a generation phase.
A training stage:
typical small sample datasets are divided into the form of sample pairs (x' m ，x′ n ) Wherein x' m And x' n Sample data belonging to different clusters in the same category. Namely x' m ∈Q k_i_s ，x′ n ∈Q k_i_t S ≠ t, where Q k_i_s And Q k_i_t Respectively showing the sample data of the s-th cluster and the t-th cluster after the data of the class k is optimally divided into the i clusters. Therefore, deformation information between the s-th cluster sample and the t-th cluster sample is guaranteed to be learned under the condition of reasonable division. This has the advantage that the learned deformation information must be relevant for the category and can be used in this category. The core of the sample expansion method is to train a network structure consisting of an inference model, a generation model, a discrimination model and a classifier, as shown in fig. 5, the inference model is as follows: for sample pairs (x ') of different clusters of the same kind' m ，x′ n ) Coding, forming deformation information Z in the hidden space, generating a model passing through the deformation information Z and a sample x' n To reconstruct the sample
Generating a model: utilizing deformation information Z and input sample x 'in potential space' n To reconstruct
Is shown as
Judging the model: for real sample pair (x) c ，x′ m ) And reconstructing the sample pairs
And (6) judging. A classifier: for the reconstructed sample
And (6) classifying.
Inference model learns samples x 'of different clusters of the same type' m And x' n Deformation information in between. The deformation information is learned by a self-encoder: x 'is coded by a self-encoder' m And x' n Performing dimensionality reduction processing, reserving key information of the dimensionality reduction processing as much as possible, and reusing the reserved key information and one sample x' n To train the network to generate another sample x' m It is also the process of this training generation that makes the information retained after dimensionality reduction deformation information. In other words, the deformation information is x' n Conversion to x' m The required additional information (the additional information is a low-dimensional vector compressed by the encoder) exists in the potential space, and is denoted as Z ═ E (x ″). m ，x′ n )。
The generative model utilizes deformation information Z and input samples x 'within the potential space' n To generate
Is shown as
The generative model is the generative network in the countermeasure network.
Wherein the content of the first and second substances,
in order to generate a loss of energy,
representing newly generated samples
And real sample x' m Mean square error between;
the discriminant model is trained to distinguish between true sample pairs (x) c ，x′ m ) Or reconstructing the sample pair
The network can reconstruct the sample more accurately through the antagonism training.
Wherein L is D To discriminate the loss, D (-) is the discriminator and G (-) is the generator.
The classifier is used for reconstructing samples
And (6) classifying.
Wherein L is cls Is a classification loss;
total loss:
L＝L mse +L D +L cls (9)
a generation stage:
the invention still uses the sample pairs (x ') of different clusters of the same kind' m ，x′ n ) As input to the model, then deformation information Z and sample x' u ∈Q k_i_t (and x' n Homogeneous cluster) as input to the generative model, and finally generates a for the classA new sample is taken. By constantly changing sample pairs (x ') of inference model inputs' m ，x′ n ) (but should be guaranteed to be a sample pair trained during training) or sample x 'that changes the input to the generative model' u To generate more new samples for that category (but to ensure that sample x 'of the model input is generated)' n Like clusters) so that maximum utilization of the learned deformation information is ensured. The purpose of using countertraining in the training process is to allow the network to generate a new sample from the existing samples, which is enough to be a new sample for the class as long as it looks different from the input sample.
FIG. 2 is a resistance data enhancement model based on feature reconstruction and deformation information.
The pseudo code is as follows:
the fourth concrete implementation mode:
the present embodiment is a storage medium having at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement a sample augmentation method for small sample image recognition.
Examples
The invention performed three experiments on two standard reference datasets (MNIST and emist), respectively a quantification experiment of large and small sample data, an extraction experiment of a typical small sample dataset, and an expansion experiment of a typical small sample dataset. To evaluate the SEFI method and the SECIDI method.
For the MNIST data set, 10 categories exist, the training set has 60000 pieces of data, the testing set has 10000 pieces of data, and the first 2000 pieces of data in the training set are selected as a large sample data set. 30% (total 600 pieces of data) extracted therefrom were taken as small sample data. In a small sample dataset, the sample size for each category is around 60.
For the balanced dataset within the EMNIST dataset, there were 47 categories (of which there were 10 number categories, 37 character categories), 112800 pieces of data for the training set, and 18800 pieces of data for the test set. For comparison with other experiments, the present invention selects only categories 42-47 (6 categories total) as the character recognition task. Likewise, the present invention selects 200 pieces of data per category (1200 total) as a large sample dataset.
In the experimental process, the invention takes the extracted data as a small sample data set, and the Rest un-extracted data as a Rest data set and is also used for testing.
Sample data of 10% (200 samples) to 90% (1800 samples) are respectively extracted from a large sample data set by using a Bootstrapping sampling method. And training the extracted data by using the same CNNs model structure. And testing by using the same test set to obtain the relationship between the data volume and the model accuracy, and showing the conditions of optimal, average and worst model accuracy under the condition of the same data volume, as shown in fig. 3.
And (3) analyzing an experimental result:
1. from fig. 3, it is apparent that as the amount of data decreases, the accuracy of the model decreases. And the accuracy of the model drops even more when the sample drawn is less than 30% of the large sample.
2. For the small sample data with the same quantity extracted from the large sample data set, the models trained by different sample selection have obvious difference in testing. And as the amount of extraction decreases, the difference increases significantly. The importance of the invention to provide a typical small sample data set for a model is also illustrated.
3. Through quantification experiments of large and small sample data, the present invention determines 30% of the large sample data volume as the number of small sample datasets (i.e., fifty-six samples per class). Because when the present invention samples using the Bootstrapping method, there is an effect close to the training of large sample data when extracting 30% of the large sample data amount. If the number is reduced, the optimal situation is difficult to achieve the training effect of large sample data.
The method optimally divides the data of each category of the large sample data set, and gradually increases from dividing into two clusters in the dividing process when L is n+1 ＞αL n When (wherein L) n Representing the total loss when dividing into n clusters, alpha is a hyperparameter, and the invention sets alpha to 0.95 in the experimental process). And after the optimal division number is determined, dividing each category of data. The partitioning results of the MNIST dataset are shown in table 1 below.
Table 1: the MNIST data set is optimally divided:
second, in the optimal partitioning for each class, uniformly quantitative decimated samples constitute a typical small sample dataset. The remaining unextracted data is used as the Rest data set. Training the model by using the extracted typical small sample data set, and then testing the model by using the test set and the Rest data set respectively.
Finally, the invention respectively compares 4 methods such as a large sample data set, a Bootstrapping sampling method, a uniform sampling method, an SEFR sampling method (SEFR is the sampling method of the invention, and the test set and the corresponding Rest data set are respectively used for testing), and the like. The experiments were performed 50 times each and the best, average and worst cases are summarized in table 2 below:
table 2: experimental results on digital data sets
And (3) analysis of experimental results:
1. the SEFR method is significantly superior to the Bootstrapping sampling method and the uniform sampling method in terms of image. First, the average classification accuracy of the SEFR method is higher than the boosting sampling method and the uniform sampling method. Secondly, the floating range of the SEFR method is obviously smaller than that of the Bootstrapping sampling method, and the effect is more stable. And in the worst case, is significantly better than the Bootstrapping sampling method. The SEFR method can avoid the extreme case of the extracted sample set, and the worst case can be higher than the average level of the boosting sampling method.
2. The use of the SEFR method reduces the average accuracy by 4.5% compared with the average accuracy of the large sample data training under the condition that the data volume is reduced by 70%. Because the present invention uses CNNs to learn the characteristics of the samples, from a characteristic point of view, the small sample dataset extracted by the SEFR method can cover most of the characteristics of the large sample dataset.
3. The optimal case of a typical small sample dataset when tested with the SEFR extraction method and with the Rest dataset replaces 94.5% of the data of the original large sample.
4. In the experiment of the SEFR method, the precision of a plurality of experiments can reach more than 90 percent, and even can be close to the average precision of a model trained by a large sample data set.
The data of the same category is divided into a plurality of clusters, and through a visualization method, the invention takes the sample of the label 5 in the MNIST as an example, and the sample is divided into four clusters. The centroid of each cluster is shown as shown in fig. 4. Fig. 4 visualizes the centroids of the clusters optimally divided by the sample labeled with the MNIST dataset as 5, fig. 4(a) to 4(d) respectively show the visualized pictures of the centroids of different clusters, and the centroids of each cluster have different shapes, and the deformation information to be searched by the present invention is the conversion information between any two clusters (the combination of fig. 4(a) and 4(b) or fig. 4(c) and 4 (d)). And this difference information (between fig. 4(a) and fig. 4(b)) is used on other samples of one of the clusters (fig. 4(b)), and this difference information (between fig. 4(c) and fig. 4(d)) can also be used on other samples of one of the clusters (fig. 4 (d)).
The typical small sample data set extracted from MNIST digital data set and EMNIST character data set is subjected to multiple data expansion experiments based on SECIDI method (the expansion method of the invention), each time, the data is expanded by one time based on the original data volume, and the expanded new large sample data set is compared with the typical small sample data before expansion by using DAGAN and other expansion methods, as shown in the following tables 3 and 4.
Table 3: MNIST SECIDI Classification
Table 4: EMNIST SECIDI Classification
And (3) analyzing an experimental result:
the classification accuracy of SECIDI on the MNIST numeric data set and emist character data set is shown in table 3 and table 4, which show the average accuracy of all experiments. Table 3 shows that after typical small sample datasets in MNIST are extended using SECIDI method, the average accuracy is improved by more than 2.5% compared to before the extension. When there are 100 samples per class for a typical small sample dataset, the extended average accuracy is close to the average accuracy of the original large sample dataset. The experimental result shows that the generalization of the extended large sample data training model is better, the deformation information can be correctly used for generating new data, and the newly generated data is different from the original data, so that the training effect of the network is improved.
Table 4 shows the results after expansion of the representative small sample dataset in emist using the SECIDI method, which is significantly better than the results of the representative small sample dataset and DAGAN method before expansion. In two experiments, the SECIDI method is improved by about 1.8% compared with the DAGAN method. The method and the device perform optimal cluster division on the typical small sample data set, so that the network can learn the in-class deformation information more easily and correctly. And when judging the newly generated sample, the invention provides the centroid of the corresponding cluster to the newly generated sample, so as to make the newly generated sample more consistent with the characteristics and distribution condition of the cluster.
Fig. 5 shows new samples generated on the MNIST digital data set (labeled 3) and the emist character data set (labeled 46) using the network structure in the SECIDI method, and fig. 5(a) and 5(b) correspond to the MNIST digital data set and the emist character data set, respectively.
Visualization experiment: the lower left pictures in fig. 5(a) and 5(b) are both newly generated samples. Taking fig. 5(b) as an example, the deformation information between the two samples at the top left and top right and the sample at the bottom right are used to generate a new sample of the category. The aim of the invention is firstly to ensure that the new sample generated must belong to the category and that the new sample generated is different from the several samples provided, thus ensuring the availability of the new sample generated. The generalization of the network is improved by the expanded large sample data set.
The present invention is capable of other embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the present invention.
Claims (7)
1. A sample extraction method for small sample image recognition is characterized by comprising the following steps:
s1, calculating the center support point C of each category of the image large sample data set k ；
S2, dividing the sample data of the same kind into clusters with dynamic quantity from the angle of sample characteristics, and calculating the mass center of each cluster according to each division condition;
calculating a new central point of the category under the dividing condition according to the mass center of each cluster; calculating the sum of the same cluster error and the centroid error to be used as the total error of the category under the dividing condition; selecting the condition of the minimum error as the optimal dividing mode of the category;
s3, uniformly and quantitatively extracting samples from each optimally divided cluster according to the sample distribution of the clusters, and using the samples as a small sample data set for feature reconstruction;
calculating the central support point C of each category of the image large sample dataset k Comprises the following steps:
for the image large sample data set, calculating the average vector of all the characteristic vectors in each category in the large sample data set, and taking the average vector as the central support point C of the category k ：
S k Sample set representing the Kth category, | S k L represents the number of samples in the sample set of the kth category; x is the number of i Denotes belonging to S k Sample data of a sample set, y i Is a sample x i A corresponding label;
is an embedding function;
step S2 includes the following steps:
2.1, clustering the characteristic vectors of the samples in each category in the large sample data set into a dynamic number of clusters, wherein m clusters are arranged;
2.2 calculating the centroid C 'of each new cluster' k_m_n Denote all samples under the cluster as X ∈ S k_m_n ；C′ k_m_n Representing the centroid of the nth cluster after the data of the class k is divided into m clusters; s. the k_m_n Representing a set of all samples in the nth cluster after the samples in the category k are divided into m clusters;
2.3, center of mass C 'through each new cluster' k_m_n Calculate the new centroid C 'of the class under the partition' k_m ；
Wherein, C' k_m Representing a new central support point of the category formed by the mass center of each cluster after dividing the samples of the Kth category into m clusters;
2.4 by calculating the cluster error L sce And error of center of mass L ce As the total error L of the class in the case of this division s (ii) a Selecting the condition with the minimum total error as the optimal division of the data of the category;
the total error L s ＝L sce +L ce (ii) a Wherein
The cluster error is the sum of the distances between all samples in each cluster and the centroid of the cluster:
the centroid error is the distance between the new centroid of the class constructed by the centroids of the clusters and the centroid of the class in the large sample:
L ce ＝C′ k_m -C k 。
2. the sample expansion method for small sample image recognition is characterized in that a small sample data set is determined by the sample extraction method for small sample image recognition in claim 1, and expansion is carried out based on the small sample data set, wherein the sample expansion comprises a training phase and a generation phase;
a training stage:
typical small sample datasets are divided into the form of sample pairs (x' m ，x′ n ) Wherein x' m And x' n Sample data belonging to different clusters in the same category; namely x' m ∈Q k_i_s ，x′ n ∈Q k_i_t S ≠ t, where Q k_i_s And Q k_i_t Respectively representing the sample data of the s-th cluster and the t-th cluster after the data of the class k are optimally divided into the i clusters;
the sample expansion process is completed by using a sample expansion model, wherein the sample expansion model is a network structure comprising an inference model, a generation model, a discrimination model and a classifier;
inference model versus sample pair (x ') of different clusters of the same type' m ，x′ n ) Coding and learning samples x 'of different clusters of the same type' m And x' n Deformation information Z ═ E (x' m ，x′ n )；
Generation model utilizes deformation information Z and input samples x 'in potential space' n To generate
Is shown as
The discriminant model is trained to distinguish between true sample pairs (x) c ，x′ m ) Or reconstructing the sample pair
The network can reconstruct the sample more accurately through antagonism training;
the classifier is used for reconstructing samples
Classifying;
a generation stage:
sample pairs (x ') of different clusters of the same kind' m ，x′ n ) As input to the model, then deformation information Z and sample x' u ∈Q k_i_t As an input of the generation model, a new sample is finally generated for the category; by continually changing inferencesModel-input sample pair (x' m ，x′ n ) Or sample x 'of the input of the generative model is changed' u To generate more new samples for that category.
3. The sample augmentation method for small sample image recognition of claim 2, wherein the changing infers a sample pair (x ') of model inputs' m ，x′ n ) Need to guarantee the sample pair (x' m ，x′ n ) Are pairs of samples trained in the training process.
4. The sample augmentation method for small sample image recognition of claim 3, wherein the changing generates samples x 'of an input of a model' u Is guaranteed to be x' u And sample x 'of the generative model input' n And (5) the same kind and the same cluster.
5. The sample expansion method for small sample image recognition according to claim 4, wherein the total loss of the sample expansion model in the training phase is as follows:
L＝L mse +L D +L cls
wherein L is mse To produce losses, L D To discriminate loss, L cls Is a classification loss.
6. A storage medium having stored therein at least one instruction which is loaded and executed by a processor to implement a method for sample extraction for small sample image recognition as claimed in claim 1.
7. A storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement a sample expansion method for small sample image recognition as claimed in any one of claims 2 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110687034.2A CN113298184B (en) | 2021-06-21 | 2021-06-21 | Sample extraction and expansion method and storage medium for small sample image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110687034.2A CN113298184B (en) | 2021-06-21 | 2021-06-21 | Sample extraction and expansion method and storage medium for small sample image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298184A CN113298184A (en) | 2021-08-24 |
| CN113298184B true CN113298184B (en) | 2022-09-02 |
Family
ID=77329008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110687034.2A Active CN113298184B (en) | 2021-06-21 | 2021-06-21 | Sample extraction and expansion method and storage medium for small sample image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298184B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114998956B (en) * | 2022-05-07 | 2023-03-28 | 北京科技大学 | Small sample image data expansion method and device based on intra-class difference |
| CN117932457B (en) * | 2024-03-22 | 2024-05-28 | 南京信息工程大学 | Model fingerprint identification method and system based on error classification |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109948728A (en) * | 2019-03-28 | 2019-06-28 | 第四范式（北京）技术有限公司 | The method and apparatus of the training of abnormal transaction detection model and abnormal transaction detection |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108959305A (en) * | 2017-05-22 | 2018-12-07 | 北京国信宏数科技有限公司 | A kind of event extraction method and system based on internet big data |
| CN108446738A (en) * | 2018-03-23 | 2018-08-24 | 联想(北京)有限公司 | A kind of clustering method, device and electronic equipment |
| CN109325532A (en) * | 2018-09-18 | 2019-02-12 | 成都网阔信息技术股份有限公司 | The image processing method of EDS extended data set under a kind of small sample |
| US11615264B2 (en) * | 2019-03-05 | 2023-03-28 | Memorial Sloan Kettering Cancer Center | Systems and methods for image classification using visual dictionaries |
| CN111461179B (en) * | 2020-03-11 | 2023-06-30 | 清华大学 | Base class selection method and device for small sample learning |
| CN112200211B (en) * | 2020-07-17 | 2024-04-05 | 南京农业大学 | Small sample fish identification method and system based on residual network and transfer learning |
| CN112231583B (en) * | 2020-11-11 | 2022-06-28 | 重庆邮电大学 | E-commerce recommendation method based on dynamic interest group identification and generation of confrontation network |
| CN112966740B (en) * | 2021-03-05 | 2024-03-29 | 西安邮电大学 | Small sample hyperspectral image classification method based on core sample self-adaptive expansion |
| CN112784929B (en) * | 2021-03-14 | 2023-03-28 | 西北工业大学 | Small sample image classification method and device based on double-element group expansion |
-
2021
- 2021-06-21 CN CN202110687034.2A patent/CN113298184B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109948728A (en) * | 2019-03-28 | 2019-06-28 | 第四范式（北京）技术有限公司 | The method and apparatus of the training of abnormal transaction detection model and abnormal transaction detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298184A (en) | 2021-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110837836B (en) | Semi-supervised semantic segmentation method based on maximized confidence | |
| CN113378632B (en) | Pseudo-label optimization-based unsupervised domain adaptive pedestrian re-identification method | |
| Sinha et al. | Class-wise difficulty-balanced loss for solving class-imbalance | |
| CN113298184B (en) | Sample extraction and expansion method and storage medium for small sample image recognition | |
| CN105023006B (en) | Face identification method based on enhanced nonparametric maximal margin criterion | |
| CN110852288A (en) | Cell image classification method based on two-stage convolutional neural network | |
| CN103258210A (en) | High-definition image classification method based on dictionary learning | |
| CN110413791A (en) | File classification method based on CNN-SVM-KNN built-up pattern | |
| CN110008899B (en) | Method for extracting and classifying candidate targets of visible light remote sensing image | |
| CN111325264A (en) | Multi-label data classification method based on entropy | |
| Naqvi et al. | Feature quality-based dynamic feature selection for improving salient object detection | |
| CN109086794B (en) | Driving behavior pattern recognition method based on T-LDA topic model | |
| CN112784921A (en) | Task attention guided small sample image complementary learning classification algorithm | |
| Li et al. | Co-saliency detection based on hierarchical consistency | |
| Li et al. | HEp-2 specimen classification via deep CNNs and pattern histogram | |
| CN117153268A (en) | Cell category determining method and system | |
| CN110197213A (en) | Image matching method, device and equipment neural network based | |
| CN113436115A (en) | Image shadow detection method based on depth unsupervised learning | |
| Lim et al. | A fuzzy qualitative approach for scene classification | |
| Kalibhat et al. | Measuring self-supervised representation quality for downstream classification using discriminative features | |
| CN117441196A (en) | Method for determining image descriptors, encoding pipeline, and visual location recognition method | |
| Ju et al. | A novel neutrosophic logic svm (n-svm) and its application to image categorization | |
| Altintakan et al. | An improved BOW approach using fuzzy feature encoding and visual-word weighting | |
| CN112598056A (en) | Software identification method based on screen monitoring | |
| CN112308151A (en) | Weighting-based classification method for hyperspectral images of rotating forest |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |