CN108520282B - Triple-GAN-based classification method - Google Patents

Triple-GAN-based classification method Download PDF

Info

Publication number
CN108520282B
CN108520282B CN201810330974.4A CN201810330974A CN108520282B CN 108520282 B CN108520282 B CN 108520282B CN 201810330974 A CN201810330974 A CN 201810330974A CN 108520282 B CN108520282 B CN 108520282B
Authority
CN
China
Prior art keywords
generated data
label
generator
data
distribution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810330974.4A
Other languages
Chinese (zh)
Other versions
CN108520282A (en
Inventor
欧阳建权
方昆
唐欢容
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiangtan University
Original Assignee
Xiangtan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiangtan University filed Critical Xiangtan University
Priority to CN201810330974.4A priority Critical patent/CN108520282B/en
Publication of CN108520282A publication Critical patent/CN108520282A/en
Application granted granted Critical
Publication of CN108520282B publication Critical patent/CN108520282B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/24323—Tree-organised classifiers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The generation of a countermeasure network (GAN) shows great development prospect in image generation and semi-supervised learning, and is developed into a Triple-countermeasure network (Triple-GAN). However, the Triple-GAN based classification method still has two problems to be solved: based on KL divergence distribution structure, gradient disappearance is easy to generate, and unstable training condition appears; due to the fact that Triple-GAN manually marks and labels the samples, manual marking workload is too large, marking is not uniform, and the like. Based on the method, real samples are classified by random forests (RandomForests), leaf nodes are automatically marked with labels, and a thought construction loss function of a countermeasure network (LSGAN) is generated by least squares to avoid gradient disappearance.

Description

Triple-GAN-based classification method
Technical Field
The invention relates to a classification method of images, in particular to a classification method based on images
Figure BDA0001627939650000011
A method for quickly classifying images by combining distribution, Triple-GAN three-player games and random forests belongs to the field of image processing.
Background
Image mining is an emerging area in data mining. Image classification is the basis of data mining, and becomes more and more important in the face of a large amount of image data. Many classification techniques, such as decision trees, minimum distance methods, neural networks, fuzzy classification, support vector machines, k-means, etc., are currently available for image classification. The proposal of the GAN model brings a new height to the image classification field and also promotes the development of the image mining technology. And the research based on the GAN model also becomes a research hotspot.
The generation of the countermeasure network (GAN) has shown great development prospects in the fields of image generation and semi-supervised learning, and GAN has become a hot research direction in the fields of image and visual computation, voice and language processing, information security and chess games, particularly artificial intelligence. GAN has been developed by two-player games into Triple-play networks (Triple-GAN), the Triple-GAN body comprising: a classifier (C), a generator (G) and a discriminator (D). Triple-GAN ensures that the classifier and the builder achieve the optimal solution for classification from the point of view of game theory and enables the builder to sample data in a particular class.
Triple-GAN is based on the target loss function of the KL divergence profile construction, there are still cases where the gradient disappears when the profiles do not cross. Therefore, it is proposed to use a least squares penalty function for the discriminator using a least squares generation countermeasure network (LSGAN). Experiments prove that the objective function of the LSGAN can be minimized
Figure BDA0001627939650000012
And (4) distribution. Compared to conventional GAN, LSGAN has two advantages: LSGAN can generate higher quality images than conventional GAN; LSGAN behaves more stably during learning.
However, the Triple-GAN still has the following problems in the application process: (1) due to the utilization of KL divergence distributions, gradient disappearance is easily generated under the condition that the distributions are not crossed, and the training is stopped when the training does not reach an ideal result, so that the model training is unstable. (2) The problem of excessive data marking workload exists because the manual marking of the sample can cause uneven marking of the label. Therefore, the existing Triple-GAN image classification technology has the defects and needs to be improved.
Disclosure of Invention
Aiming at the problems, the classification algorithm of Triple-GAN is optimized by random forest (RandomForests) and LSGAN is utilized at the same time
Figure BDA0001627939650000013
Constructing a distribution loss function based on the minimum chi-square, and performing the following work:
(1) improving the classifier, simultaneously fusing random forests into the classifier, and constructing the classifier for automatically marking the labels;
(2) the inference process of the LSGAN loss function is utilized, and a Triple-GAN construction model is combined, so that the advantages of inheriting the stability of the LSGAN and inheriting the Triple-GAN are formed.
According to the embodiment provided by the invention, a Triple-GAN based classification method is provided.
A Triple-GAN based classification method, comprising the steps of:
1) acquiring a real sample; constructing a three-player game model based on Triple-GAN, wherein the three-player game model consists of a classifier (C), a generator (G) and a discriminator (D); serializing the real samples and inputting the serialized real samples into a classifier (C); the discriminator (D) is formed by a random forest based on binary decision tree predictive analysis, wherein the random forest is formed by m trees,
Figure BDA0001627939650000021
wherein: n is the number of real samples, h is the height of the binary decision tree, and h is an integer greater than or equal to 3; formation 2h-1M leaf nodes, using the leaf nodes as real sample labels;
2) Triple-GAN-based construction
Figure BDA0001627939650000022
An objective function of the distributed classifier (C), generator (G) and discriminator (D);
3) inputting the training samples into a generator (G) to form generated data; and distinguishing the difference between the real sample label and the generated data according to the objective function, marking the label of the generated data when the objective function reaches balance, and classifying the image (namely the generated data) through the marked label.
In the invention, the step 1) is specifically as follows:
101) downloading one or more images of mini, svhn and cifar10 as real samples, serializing and inputting the real samples into a classifier (C);
102) the three-player game model constructs a binary decision tree according to a random forest algorithm, and distributes n real samples into m decision trees;
103) and marking the leaf nodes of the m decision trees with labels to form class labels y.
In the invention, the step 3) is specifically as follows:
301) downloading one or more images of mini, svhn and cifar10 as training samples;
302) serializing the training samples to form generated data x, and inputting the generated data x into a generator (G) as training set data;
303) combining the class label y and the generated data x to form combined distribution;
304) and distinguishing the difference between the generated data x and the class label y according to the objective function, marking the label on the generated data x when the objective function reaches balance, and classifying the image (namely the generated data) through the marked label.
In the invention, the class label y and the generated data x are combined to form a combined distribution, and the specific steps are as follows:
(1) generating data x and class labels y to form generator joint distribution Pg (x, y) in a generator (G);
(2) constructing binary decision tree logic according to a random forest algorithm to judge the generator joint distribution Pg (x, y):
when the content of the generator combined distribution Pg (x, y) is consistent with the decision tree, entering the next branch for judgment until all leaf nodes corresponding to the generated data x are found, and constructing a classifier combined distribution Pc (x, y) of the leaf nodes corresponding to the generated data x and the class label y;
and (3) when the content of the Pg (x, y) of the generator joint distribution is inconsistent with the decision tree, entering the next forest decision tree, and iterating the process in the step (2) until all leaf nodes corresponding to the generated data x are found.
In the invention, the step 2) is specifically as follows: inputting the generator combined distribution Pg (x, y) and the classifier combined distribution Pc (x, y) into a discriminator (D), and generating the classifier combined distribution Pc (x, y) and Pc (x, y) of the leaf node and the class label y corresponding to the data x
Figure BDA0001627939650000031
Calculating an optimized objective function by distribution; wherein, the target functions of the classifier (C), the generator (G) and the discriminator (D) are set as follows:
Figure BDA0001627939650000032
wherein: x represents generated data, y represents a class label, and z represents a leaf node corresponding to the generated data x; a represents that the generator (G) encodes the generated data, a belongs to { -1,0,1 }; b represents that the classifier (C) encodes the class label, b belongs to { -1,0,1 }; c represents that the generator (G) judges the optimal deception sample data through an objective function for the generated data, and c belongs to { -1,0,1 }; in this equation: e(x,y)～p(x,y)Representing the expected value of the joint distribution p (x, y) of the label pair (x, y) in the discriminator (D); d (x, y) represents the probability of a true sample label pair false label of the label pair (x, y) in the discriminator;
Figure BDA0001627939650000033
represents the expected value of the joint distribution Pc (x, y) of the label pairs (x, y) in the classifier (C);
Figure BDA0001627939650000034
an expectation value representing the joint distribution Pg (x, y) of the label pair (x, y) in the generator (G); g (y, z) represents the mapping of the label pair (y, z) in the generator in the sample space, and D (G (y, z), y) represents the probability of the generated sample of the label pair (y, z) in the discriminator to the real sample; rcRepresenting a balance function, α is a tuning parameter, α ∈ {0,1 }.
Preferably, a is-1, b is 1, and c is 0, i.e., b-c ═ 1 and b-a ═ 2; balance function
Figure BDA0001627939650000035
Objective function satisfaction minimization
Figure BDA0001627939650000036
Divergence distribution; wherein: p is the probabilistic joint distribution of the discriminators, and Pc is the probabilistic joint distribution of the classifiers.
In the present invention, the objective function reaching balance means: when the discriminator reaches the optimum, namely the objective function reaches 0.5, marking a label on the generated data x to form sample data, inserting the sample data into the output data in an interpolation mode, and finishing the classification of the generated data x;
when the arbiter does not reach the optimum, the balance function Rc is adjusted until the objective function reaches the optimum, i.e. the objective function reaches 0.5.
Preferably, the method further comprises: and continuously training the Triple-GAN optimization model until all the generated data x are labeled with labels, and classifying and outputting sample data formed by the generated data x according to the sequence of the class labels y, namely finishing the classification of the generated data x.
Preferably, the number of images in step 101) exceeds 1 ten thousand; the serialization in step 101) takes the list form.
Preferably, the number of images in step 301) exceeds 1 ten thousand; the serialization in step 302) takes the list form.
In the invention, the training set contained in each tree in the forest is classified by adopting a self-help selection method (bootstrap) and a resampling mode. And has the following advantages: (1) the random forest has excellent anti-noise capability and is not easy to over-fit through the characteristics of random sample selection and random feature selection; (2) the random forest can process high-dimensional data under the condition of not reducing the dimension; (3) the random forest can process both continuous and discrete data sets, does not need to make a normalized data set, and has strong adaptability; (4) the random forest measures the similarity of samples through a similarity matrix generated in an efficient training process.
In the prior art, the Triple-GAN is based on a target loss function constructed by KL divergence distributions, and the situation that the gradient disappears when the distributions are not crossed still exists. The invention proposes to use a least squares penalty function for the discriminator using a least squares generation countermeasure network (LSGAN). Experiments prove that the objective function of the LSGAN can be minimized
Figure BDA0001627939650000041
And (4) distribution. Compared to conventional GAN, LSGAN has two advantages: LSGAN can generate higher quality images than conventional GAN; LSGAN behaves more stably during learning.
In the invention, a random forest is used as a prediction classification algorithm, the random forest is a history table of a plurality of samples generated from a history data table in a random sampling mode, and a decision tree is generated for each history table of the samples. Because the data is put back into the summary table after each generation of the sampling table, each decision tree is independent and has no relation. And forming a random forest by the plurality of decision trees. And when a new piece of data is generated, each decision tree in the forest is judged respectively, and the result of voting most is taken as the final judgment result. Thereby increasing the probability of correctness. The random forest is combined with semi-supervised learning, and based on the characteristics of the random forest, the random forest is constructed according to the following steps:
firstly, a three-player game model is constructed based on Triple-GAN, the three-player game is respectively composed of a classifier, a discriminator and a discriminator, wherein a generator is composed of a random forest based on binary decision tree prediction analysis, wherein the random forest is composed of m trees (
Figure BDA0001627939650000042
Where n total number of true samples and h is the binary decision tree height), form 2h-1M leaf nodes, will
Figure BDA0001627939650000043
And marking labels for leaf node samples of the real samples. The newly formed classifier iterates the data generated by the generator, generates generator data and random forest decision prediction data, finds the most similar leaf nodes from the generator data, marks labels for the data generated by the generator, and inputs the labels into the discriminator.
Then, the construction is based on
Figure BDA0001627939650000051
A distribution classifier (C), a generator (G) and a discriminator (D) loss function. The classifier (C), the generator (G) and the discriminator (D) are mainly distinguished from the Triple-GAN by the following steps: the Triple-GAN loss function is based on a KL divergence distribution, which for the case of distribution disjointed, there is a gradient vanishing, whereas
Figure BDA0001627939650000052
Controlling through confidence spaceThe case where the gradient disappears.
In the present invention, an a-b encoding scheme is used for the discriminator using the LSGAN objective function principle, where b is to represent true data, a is to represent false data, i.e., generated data, and c is to represent spoofed data. In the formula, C represents a classifier, G represents a generator, and D represents a discriminator (the same applies below). For any fixed C and G, the optimum discriminator D of the game is defined by the objective function U (C, G, D), as shown in equation (1):
Figure BDA0001627939650000053
wherein P isa(x，y)＝(1-α)Pg(x，y)+αPc(x，y)α∈(0，1)。
Given the classifier C and generator G, in combination with the LSGAN objective function, where C represents the value of G, C hoped D to believe false data, the objective function can be rewritten as equation (2):
Figure BDA0001627939650000054
when P (x, y) ═ Pa (x, y), a global minimum function V (C, G) is constructed, and equation (2) is converted to equation (3):
Figure BDA0001627939650000055
when b-c is 1 and b-a is 2, from the derivation of LSGAN, equation (3) can be derived as:
Figure BDA0001627939650000056
from equation (4) it can be derived that the global minimum function V (C, G) is based on
Figure BDA0001627939650000057
And (4) distribution.
Wherein P isa(x，y)＝(1-α)Pg(x，y)+αPc(x, y) α ∈ (0, 1), and integration on both sides yields the following equation (5):
∫P(x，y)＝(1-α)∫Pg(x，y)dx+α∫Pc(x，y)dx (5)；
according to the formula (2), the target function pair V (C, G) takes the minimum value, and the maximum value judgment is carried out on D, so that the target function is obtained as shown in the formula (6):
Figure BDA0001627939650000061
when b-C is 1, b-a is 2, and P (x, y) is Pa (x, y), V (G, C) is 0, resulting in an imbalance in training, which makes the results less than optimal. To solve this problem, an Rc function is constructed, utilizing Rc
Figure BDA0001627939650000062
The distribution confidence control and the distribution principle make a distribution balance structure for the discriminator and the classification, namely:
Figure BDA0001627939650000063
and (4) reasoning by combining formulas (6) and (7) to obtain a final objective function formula (8):
Figure BDA0001627939650000064
when b-c-1 and b-a-2,
Figure BDA0001627939650000065
namely, when the data of the a, b and c coding marks respectively take-1, 1 and 0, the loss function is proved to satisfy the minimization
Figure BDA0001627939650000066
Divergence distribution.
Wherein: x represents generated data, y represents a class label, and z represents a leaf node corresponding to the generated data x; a represents that the generator (G) encodes the generated data, a belongs to { -1,0,1 }; b represents that the classifier (C) encodes the class label, b belongs to { -1,0,1 }; c represents a deception sample for which the generator (G) discriminates the generated data optimally by means of an objective functionData, c ∈ { -1,0,1 }; in this equation: e(x,y)～p(x,y)Representing the expected value of the joint distribution p (x, y) of the label pair (x, y) in the discriminator (D); d (x, y) represents the probability of a true sample label pair false label of the label pair (x, y) in the discriminator;
Figure BDA0001627939650000067
represents the expected value of the joint distribution Pc (x, y) of the label pairs (x, y) in the classifier (C);
Figure BDA0001627939650000068
an expectation value representing the joint distribution Pg (x, y) of the label pair (x, y) in the generator (G); g (y, z) represents the mapping of the label pair (y, z) in the generator in the sample space, and D (G (y, z), y) represents the probability of the generated sample of the label pair (y, z) in the discriminator to the real sample; rcRepresenting a balance function, α is a tuning parameter, α ∈ {0,1 }.
In the present invention, the loss function is common to the objective function.
Compared with the prior art, the technical scheme of the invention has the following beneficial technical effects:
1. based on
Figure BDA0001627939650000071
The distribution makes the model more stable, avoids the disappearance of the gradient and makes the convergence of the model faster;
2. the manual label marking is changed into automatic label marking, and meanwhile, the labels are marked through random forest leaf nodes, so that the labels are marked more uniformly;
3. and combining random forest prediction analysis and leaf node marking labels, and iterating the data generated by the prediction generator more quickly.
Drawings
FIG. 1 is a general block diagram of the present invention;
FIG. 2 is a graph showing the distribution of random forest and KL divergence in the embodiment of the present invention
Figure BDA0001627939650000072
Error rate comparison of the distribution;
FIG. 3 is a Triple-GAN optimization model loss function cross-loss profile according to an embodiment of the present invention;
FIG. 4 is an original image of minist;
FIG. 5 is a classified image of minist after the method of the present invention is applied.
Detailed Description
According to the embodiment provided by the invention, a Triple-GAN based classification method is provided.
A Triple-GAN based classification method, comprising the steps of:
1) acquiring a real sample; constructing a three-player game model based on Triple-GAN, wherein the three-player game model consists of a classifier (C), a generator (G) and a discriminator (D); serializing the real samples and inputting the serialized real samples into a classifier (C); the discriminator (D) is formed by a random forest based on binary decision tree predictive analysis, wherein the random forest is formed by m trees,
Figure BDA0001627939650000073
wherein: n is the number of real samples, h is the height of the binary decision tree, and h is an integer greater than or equal to 3; formation 2h-1M leaf nodes, using the leaf nodes as real sample labels;
2) Triple-GAN-based construction
Figure BDA0001627939650000074
An objective function of the distributed classifier (C), generator (G) and discriminator (D);
3) inputting the training samples into a generator (G) to form generated data; and distinguishing the difference between the real sample label and the generated data according to the objective function, marking the label of the generated data when the objective function reaches balance, and classifying the image (namely the generated data) through the marked label.
In the invention, the step 1) is specifically as follows:
101) downloading one or more images of mini, svhn and cifar10 as real samples, serializing and inputting the real samples into a classifier (C);
102) the three-player game model constructs a binary decision tree according to a random forest algorithm, and distributes n real samples into m decision trees;
103) and marking the leaf nodes of the m decision trees with labels to form class labels y.
In the invention, the step 3) is specifically as follows:
301) downloading one or more images of mini, svhn and cifar10 as training samples;
302) serializing the training samples to form generated data x, and inputting the generated data x into a generator (G) as training set data;
303) combining the class label y and the generated data x to form combined distribution;
304) and distinguishing the difference between the generated data x and the class label y according to the objective function, marking the label on the generated data x when the objective function reaches balance, and classifying the image (namely the generated data) through the marked label.
In the invention, the class label y and the generated data x are combined to form a combined distribution, and the specific steps are as follows:
(1) generating data x and class labels y to form generator joint distribution Pg (x, y) in a generator (G);
(2) constructing binary decision tree logic according to a random forest algorithm to judge the generator joint distribution Pg (x, y):
when the content of the generator combined distribution Pg (x, y) is consistent with the decision tree, entering the next branch for judgment until all leaf nodes corresponding to the generated data x are found, and constructing a classifier combined distribution Pc (x, y) of the leaf nodes corresponding to the generated data x and the class label y;
and (3) when the content of the Pg (x, y) of the generator joint distribution is inconsistent with the decision tree, entering the next forest decision tree, and iterating the process in the step (2) until all leaf nodes corresponding to the generated data x are found.
In the invention, the step 2) is specifically as follows: inputting the generator combined distribution Pg (x, y) and the classifier combined distribution Pc (x, y) into a discriminator (D), and generating the classifier combined distribution Pc (x, y) and Pc (x, y) of the leaf node and the class label y corresponding to the data x
Figure BDA0001627939650000081
Calculating an optimized objective function by distribution; wherein, the target functions of the classifier (C), the generator (G) and the discriminator (D) are set as follows:
Figure BDA0001627939650000082
wherein: x represents generated data, y represents a class label, and z represents a leaf node corresponding to the generated data x; a represents that the generator (G) encodes the generated data, a belongs to { -1,0,1 }; b represents that the classifier (C) encodes the class label, b belongs to { -1,0,1 }; c represents that the generator (G) judges the optimal deception sample data through an objective function for the generated data, and c belongs to { -1,0,1 }; in this equation: e(x,y)～p(x,y)Representing the expected value of the joint distribution p (x, y) of the label pair (x, y) in the discriminator (D); d (x, y) represents the probability of a true sample label pair false label of the label pair (x, y) in the discriminator;
Figure BDA0001627939650000091
represents the expected value of the joint distribution Pc (x, y) of the label pairs (x, y) in the classifier (C);
Figure BDA0001627939650000092
an expectation value representing the joint distribution Pg (x, y) of the label pair (x, y) in the generator (G); g (y, z) represents the mapping of the label pair (y, z) in the generator in the sample space, and D (G (y, z), y) represents the probability of the generated sample of the label pair (y, z) in the discriminator to the real sample; rcRepresenting a balance function, α is a tuning parameter, α ∈ {0,1 }.
Preferably, a is-1, b is 1, and c is 0, i.e., b-c ═ 1 and b-a ═ 2; balance function
Figure BDA0001627939650000093
Objective function satisfaction minimization
Figure BDA0001627939650000094
Divergence distribution; wherein: p is the probability joint distribution of the discriminators, Pc is the probability joint score of the classifiersAnd (3) cloth.
In the present invention, the objective function reaching balance means: when the discriminator reaches the optimum, namely the objective function reaches 0.5, marking a label on the generated data x to form sample data, inserting the sample data into the output data in an interpolation mode, and finishing the classification of the generated data x;
when the arbiter does not reach the optimum, the balance function Rc is adjusted until the objective function reaches the optimum, i.e. the objective function reaches 0.5.
Preferably, the method further comprises: and continuously training the Triple-GAN optimization model until all the generated data x are labeled with labels, and classifying and outputting sample data formed by the generated data x according to the sequence of the class labels y, namely finishing the classification of the generated data x.
Preferably, the number of images in step 101) exceeds 1 ten thousand; the serialization in step 101) takes the list form.
Preferably, the number of images in step 301) exceeds 1 ten thousand; the serialization in step 302) takes the list form.
Examples
The data set MNIST is now widely used. Where the MNIST includes 60,000 training samples, 10,000 validation samples and 10,000 test samples with 28 x 28 handwritten digit size.
(1) The MNIST dataset is available on Kaggle. Csv is downloaded to the data/folders, and then files are loaded into the training data.
(2) And the generator generates sample data marked with the pseudo label according to the training data.
Inputting 10,000 verification sample data sets of MNIST into a classifier (C), establishing a decision tree with the depth of M according to a random forest algorithm based on a binary decision tree, setting the height of the decision tree to be 5, establishing 304 binary decision trees, and marking labels of the established binary decision trees respectively.
(3) The classifier predicts the joint distribution of the sample and the class label:
and (3) inputting 60,000 MNIST training data sets into a generator, and forming a joint distribution Pg (x, y) by using the generated data x and the class labels y. And the arbiter finds the leaf nodes of the y labels through the prediction analysis of the random forest decision tree to form the leaf node data x and the joint distribution Pc (x, y) of the y labels.
(4) Inputting the Pg (x, y) of the generator and the Pc (x, y) of the classifier into a discriminator, and judging whether the Pg (x, y) and the Pc (x, y) reach the optimal solution or not through an objective function, namely the objective function reaches 50%. Jointly distribute Pc (x, y) and
Figure BDA0001627939650000101
the distribution calculates an optimized objective function, and the objective functions of the classifier (C), the generator (G) and the discriminator (D) are as follows:
Figure BDA0001627939650000102
the data of the coding marks with the values of a, b and c are respectively-1, 1 and 0; namely b-c-1, b-a-2;
Figure BDA0001627939650000103
at this time, the loss function (i.e., objective function) satisfies the minimization
Figure BDA0001627939650000104
Divergence distribution.
The objective function is based on a kl divergence distribution, unlike the distribution on which the Triple-GAN is based, which is based mainly on the construction of the loss function
Figure BDA0001627939650000105
And (4) distribution.
And when the discriminator reaches the optimum, namely the target function reaches 50%, outputting the sample data with the label, and inserting the data with the label marked by the generated data x into the output data of the y label in an interpolation mode. And continuously training the Triple-GAN optimization model, and completely marking the unmarked generated data x with the B coding label to achieve the effect of classifying by the same codes.
(5) The 60,000 training sample MNIST data sets were classified as shown in the classification effect of FIG. 4.
As can be seen from fig. 2: random forest,
Figure BDA0001627939650000106
The error rate is significantly lower than the KL divergence distribution as the number of iterations increases.
As can be seen in fig. 3: the cross loss is distributed between 2.5 percent and 5 percent, the error rate is maintained in a relatively low interval, and the Triple-GAN optimization model is relatively stable.

Claims (10)

1. A Triple-GAN based classification method, comprising the steps of:
1) acquiring a real sample; constructing a three-player game model based on Triple-GAN, wherein the three-player game model consists of a classifier (C), a generator (G) and a discriminator (D); serializing the real samples and inputting the serialized real samples into a classifier (C); the discriminator (D) is formed by a random forest based on binary decision tree predictive analysis, wherein the random forest is formed by m trees,
Figure FDA0002354616320000011
wherein: n is the number of real samples, h is the height of the binary decision tree, and h is an integer greater than or equal to 3; formation 2h-1M leaf nodes, using the leaf nodes as real sample labels;
2) Triple-GAN-based construction
Figure FDA0002354616320000012
An objective function of the distributed classifier (C), generator (G) and discriminator (D); the method specifically comprises the following steps: inputting the generator combined distribution Pg (x, y) and the classifier combined distribution Pc (x, y) into a discriminator (D), and generating the classifier combined distribution Pc (x, y) and Pc (x, y) of the leaf node and the class label y corresponding to the data x
Figure FDA0002354616320000013
Calculating an optimized objective function by distribution; wherein, the target functions of the classifier (C), the generator (G) and the discriminator (D) are set as follows:
Figure FDA0002354616320000014
wherein: x represents generated data, y represents a class label, and z represents a leaf node corresponding to the generated data x; a represents that the generator (G) encodes the generated data, a belongs to { -1,0,1 }; b represents that the classifier (C) encodes the class label, b belongs to { -1,0,1 }; c represents that the generator (G) judges the optimal deception sample data through an objective function for the generated data, and c belongs to { -1,0,1 }; in this equation: e(x,y)～p(x,y)Representing the expected value of the joint distribution p (x, y) of the label pair (x, y) in the discriminator (D); d (x, y) represents the probability of a true sample label pair false label of the label pair (x, y) in the discriminator;
Figure FDA0002354616320000015
represents the expected value of the joint distribution Pc (x, y) of the label pairs (x, y) in the classifier (C);
Figure FDA0002354616320000016
an expectation value representing the joint distribution Pg (x, y) of the label pair (x, y) in the generator (G); g (y, z) represents the mapping of the label pair (y, z) in the generator in the sample space, and D (G (y, z), y) represents the probability of the generated sample of the label pair (y, z) in the discriminator to the real sample; rcα is a regulating parameter, α belongs to {0,1 };
3) inputting the training samples into a generator (G) to form generated data; and distinguishing the difference between the real sample label and the generated data according to the objective function, marking the label of the generated data when the objective function reaches balance, and classifying the image (namely the generated data) through the marked label.
2. The classification method according to claim 1, characterized in that: the step 1) is specifically as follows:
101) downloading one or more images of mini, svhn and cifar10 as real samples, serializing and inputting the real samples into a classifier (C);
102) the three-player game model constructs a binary decision tree according to a random forest algorithm, and distributes n real samples into m decision trees;
103) and marking the leaf nodes of the m decision trees with labels to form class labels y.
3. The classification method according to claim 1, characterized in that: the step 3) is specifically as follows:
301) downloading one or more images of mini, svhn and cifar10 as training samples;
302) serializing the training samples to form generated data x, and inputting the generated data x into a generator (G) as training set data;
303) combining the class label y and the generated data x to form combined distribution;
304) and distinguishing the difference between the generated data x and the class label y according to the objective function, marking the label on the generated data x when the objective function reaches balance, and classifying the image (namely the generated data) through the marked label.
4. A classification method according to claim 3, characterized in that: the method comprises the following steps of forming combined distribution by utilizing class labels y and generated data x:
(1) generating data x and class labels y to form generator joint distribution Pg (x, y) in a generator (G);
(2) constructing binary decision tree logic according to a random forest algorithm to judge the generator joint distribution Pg (x, y):
when the content of the generator combined distribution Pg (x, y) is consistent with the decision tree, entering the next branch for judgment until all leaf nodes corresponding to the generated data x are found, and constructing a classifier combined distribution Pc (x, y) of the leaf nodes corresponding to the generated data x and the class label y;
and (3) when the content of the Pg (x, y) of the generator joint distribution is inconsistent with the decision tree, entering the next forest decision tree, and iterating the process in the step (2) until all leaf nodes corresponding to the generated data x are found.
5. The classification method according to any one of claims 1 to 4, characterized in that: a takes on a value of-1, b is 1 and c is 0, i.e. b-c-1, b-a-2; balance function
Figure FDA0002354616320000021
Objective function satisfaction minimization
Figure FDA0002354616320000022
Divergence distribution; wherein: p is the probabilistic joint distribution of the discriminators, and Pc is the probabilistic joint distribution of the classifiers.
6. The classification method according to claim 3 or 4, characterized in that: the target function is balanced by: when the discriminator reaches the optimum, namely the objective function reaches 0.5, marking a label on the generated data x to form sample data, inserting the sample data into the output data in an interpolation mode, and finishing the classification of the generated data x;
when the arbiter does not reach the optimum, the balance function Rc is adjusted until the objective function reaches the optimum, i.e. the objective function reaches 0.5.
7. The classification method according to claim 6, characterized in that: the method further comprises the following steps: and continuously training the Triple-GAN optimization model until all the generated data x are labeled with labels, and classifying and outputting sample data formed by the generated data x according to the sequence of the class labels y, namely finishing the classification of the generated data x.
8. The classification method according to claim 2, characterized in that: the number of the images in the step 101) exceeds 1 ten thousand; the serialization in step 101) takes the list form.
9. The classification method according to claim 3 or 4, characterized in that: the number of the images in the step 301) exceeds 1 ten thousand; the serialization in step 302) takes the list form.
10. The classification method according to claim 6, characterized in that: the number of the images in the step 301) exceeds 1 ten thousand; the serialization in step 302) takes the list form.
CN201810330974.4A 2018-04-13 2018-04-13 Triple-GAN-based classification method Active CN108520282B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810330974.4A CN108520282B (en) 2018-04-13 2018-04-13 Triple-GAN-based classification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810330974.4A CN108520282B (en) 2018-04-13 2018-04-13 Triple-GAN-based classification method

Publications (2)

Publication Number Publication Date
CN108520282A CN108520282A (en) 2018-09-11
CN108520282B true CN108520282B (en) 2020-04-03

Family

ID=63432568

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810330974.4A Active CN108520282B (en) 2018-04-13 2018-04-13 Triple-GAN-based classification method

Country Status (1)

Country Link
CN (1) CN108520282B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109214103B (en) * 2018-09-25 2023-01-20 重庆青山工业有限责任公司 Gear reliability analysis system for generating countermeasure network based on boundary constraint
CN109389080B (en) * 2018-09-30 2022-04-19 西安电子科技大学 Hyperspectral image classification method based on semi-supervised WGAN-GP
CN109815978A (en) * 2018-12-18 2019-05-28 全球能源互联网研究院有限公司 Sample classification mark and model training method, device and electronic equipment
CN109522973A (en) * 2019-01-17 2019-03-26 云南大学 Medical big data classification method and system based on production confrontation network and semi-supervised learning
CN110097103A (en) * 2019-04-22 2019-08-06 西安电子科技大学 Based on the semi-supervision image classification method for generating confrontation network
CN110133643B (en) * 2019-05-22 2021-08-20 北京林业大学 Plant root system detection method and device
CN110909509B (en) * 2019-11-28 2022-08-05 哈尔滨理工大学 Bearing life prediction method based on InfoLSGAN and AC algorithm
CN111337929A (en) * 2020-03-26 2020-06-26 上海眼控科技股份有限公司 Meteorological cloud picture prediction method and device, computer equipment and storage medium
CN111241291B (en) * 2020-04-24 2023-01-03 支付宝(杭州)信息技术有限公司 Method and device for generating countermeasure sample by utilizing countermeasure generation network
CN112598125B (en) * 2020-11-25 2024-04-30 西安科技大学 Handwriting digital generation method based on dual-discriminant weighting generation countermeasure network
CN117313748B (en) * 2023-11-24 2024-03-12 中电科大数据研究院有限公司 Multi-feature fusion semantic understanding method and device for government affair question and answer

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107220600A (en) * 2017-05-17 2017-09-29 清华大学深圳研究生院 A kind of Picture Generation Method and generation confrontation network based on deep learning
CN107330444A (en) * 2017-05-27 2017-11-07 苏州科技大学 A kind of image autotext mask method based on generation confrontation network
CN107578017A (en) * 2017-09-08 2018-01-12 百度在线网络技术（北京）有限公司 Method and apparatus for generating image

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8744172B2 (en) * 2011-06-15 2014-06-03 Siemens Aktiengesellschaft Image processing using random forest classifiers

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107220600A (en) * 2017-05-17 2017-09-29 清华大学深圳研究生院 A kind of Picture Generation Method and generation confrontation network based on deep learning
CN107330444A (en) * 2017-05-27 2017-11-07 苏州科技大学 A kind of image autotext mask method based on generation confrontation network
CN107578017A (en) * 2017-09-08 2018-01-12 百度在线网络技术（北京）有限公司 Method and apparatus for generating image

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN;Weiwei Hu 等;《arXiv》;20170220;1-7 *
Least Squares Generative Adversarial Networks;Xudong Mao 等;《ICCV》;20170831;2794-2802 *
Triple Generative Adversarial Nets;Chongxuan Li 等;《31st Conference on Neural Information Processing Systems (NIPS 2017)》;20171209;4091-4101 *
洪洋 等.深度卷积对抗生成网络综述.《第18届中国***仿真技术及其应用学术年会论文集》.2017, *

Also Published As

Publication number Publication date
CN108520282A (en) 2018-09-11

Similar Documents

Publication Publication Date Title
CN108520282B (en) Triple-GAN-based classification method
Zhao et al. Federated learning with non-iid data
De Amorim Feature relevance in ward’s hierarchical clustering using the L p norm
Lysiak et al. Optimal selection of ensemble classifiers using measures of competence and diversity of base classifiers
CN111461226A (en) Countermeasure sample generation method, device, terminal and readable storage medium
CN116635866A (en) Method and system for mining minority class data samples to train a neural network
Wu et al. Graphmixup: Improving class-imbalanced node classification on graphs by self-supervised context prediction
CN107392919A (en) Gray threshold acquisition methods, image partition method based on self-adapted genetic algorithm
CN107578101B (en) Data stream load prediction method
Johansson et al. Evolved decision trees as conformal predictors
CN111210023A (en) Automatic selection system and method for data set classification learning algorithm
Antonelli et al. Multi-objective evolutionary learning of granularity, membership function parameters and rules of Mamdani fuzzy systems
Pilat et al. Incorporating user preferences in MOEA/D through the coevolution of weights
CN113505855A (en) Training method for anti-attack model
Sublime et al. Analysis of the influence of diversity in collaborative and multi-view clustering
Liang et al. FS-MOEA: A novel feature selection algorithm for IDSs in vehicular networks
CN108960486A (en) Interactive set evolvement method based on grey support vector regression prediction adaptive value
Barazandeh et al. A decentralized adaptive momentum method for solving a class of min-max optimization problems
Singhal et al. Greedy Shapley Client Selection for Communication-Efficient Federated Learning
D’Ambrosio et al. Optimizing cellular automata through a meta-model assisted memetic algorithm
CN116010832A (en) Federal clustering method, federal clustering device, central server, federal clustering system and electronic equipment
CN113537313B (en) Unbalanced data set analysis method based on WGAN training convergence
Hwang et al. An efficient domain-adaptation method using GAN for fraud detection
Manoju et al. Conductivity based agglomerative spectral clustering for community detection
Toledo et al. Managing natural noise in collaborative recommender systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant