CN113918716A - Method and device for constructing generation confrontation topic model based on spectrum norm normalization - Google Patents
Method and device for constructing generation confrontation topic model based on spectrum norm normalization Download PDFInfo
- Publication number
- CN113918716A CN113918716A CN202111199957.XA CN202111199957A CN113918716A CN 113918716 A CN113918716 A CN 113918716A CN 202111199957 A CN202111199957 A CN 202111199957A CN 113918716 A CN113918716 A CN 113918716A
- Authority
- CN
- China
- Prior art keywords
- topic
- vector
- confrontation
- supervised
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010606 normalization Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000001228 spectrum Methods 0.000 title claims abstract description 21
- 239000013598 vector Substances 0.000 claims abstract description 82
- 238000009826 distribution Methods 0.000 claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 42
- 230000003595 spectral effect Effects 0.000 claims abstract description 23
- 230000003042 antagnostic effect Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 5
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000002457 bidirectional effect Effects 0.000 abstract description 4
- 239000011159 matrix material Substances 0.000 description 13
- 230000006870 function Effects 0.000 description 12
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000008451 emotion Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Abstract
The invention provides a method and a device for constructing a generated confrontation topic model based on spectrum norm normalization, wherein the method comprises the following steps: acquiring a text data set, and processing to obtain a tf-idf vector set of the text; a spectral norm normalization method is introduced to carry out countermeasure training on the generated countermeasure topic model to obtain a trained generator, an encoder and a supervised discriminator with a classification function; after the subject model is subjected to countermeasure training, the generator outputs subject-word distribution in the text data set; the encoder outputs a document-topic distribution in the text dataset, inputs the topic-word distribution and the document-topic distribution into a supervised discriminator, and classifies the text. The invention constructs a supervised bidirectional generation confrontation network and introduces a method of spectrum norm normalization, so that a topic model can be stably trained; the supervised discriminator can effectively extract and utilize the guidance of the label information to improve the quality of the generated theme while ensuring that the generated countermeasure network carries out countermeasure training.
Description
Technical Field
The invention relates to the technical field of nature, in particular to a method and a device for constructing a confrontation topic generation model based on spectrum norm normalization.
Background
The Chinese text is an important carrier for network information dissemination, and how to efficiently mine valuable contents from massive text data is a very worthy of research. Topic models are an important technology for text mining, which can extract potential topics or keywords from a large amount of text data to be applied to downstream tasks such as emotion analysis, text classification and question and answer systems. The more topic models applied today include implicit Dirichlet generation confrontation topic models, confrontation generation confrontation topic models, two-way confrontation generation confrontation topic models, and the like.
The publication number is CN109918510A (publication date is 2019-06-21), and a cross-domain keyword extraction method is provided, and comprises the following steps: constructing a topic-based antagonistic neural network, encoding texts in a source field and a target field based on topics by using a topic-based encoder, introducing antagonistic learning to ensure that characteristics learned by the topic-based encoder are independent of the fields and the private characteristics of the target field are reserved by using an antagonistic learning and a bidirectional self-encoder, and finally completing keyword extraction by combining a keyword labeler in the topic-based antagonistic neural network with the output of the topic-based encoder; continuously optimizing each part parameter in the confrontation neural network based on the theme in the training stage; in the testing stage, the text of the target field is input into the trained subject-based confrontation neural network, so that the keywords are extracted.
The method utilizes the antagonistic neural network topic model to extract keywords, and the existing topic model based on the antagonistic neural network still has the following two problems: firstly, the models have larger instability in the training process; secondly, the model cannot effectively utilize the label information.
Disclosure of Invention
The invention provides a method and a device for constructing a confrontation topic model based on spectrum norm normalization, aiming at overcoming the defects that the topic model in the prior art is unstable in the training process and label information cannot be effectively utilized.
In order to solve the technical problems, the technical scheme of the invention is as follows:
in a first aspect, the invention provides a method for generating a confrontation topic model based on spectrum norm normalization, which comprises the following steps:
s1: and acquiring a text data set, preprocessing the text data set, and performing feature vectorization on the text to obtain a text feature vector set, namely a tf-idf vector set.
S2: a generative confrontational model is constructed, wherein the generative confrontational model comprises a generator, an encoder, and a supervised evaluator.
S3: inputting the tf-idf vector into the generated confrontation topic model and performing confrontation training on the generated confrontation topic model by using a spectral norm normalization method to obtain a trained generated confrontation topic model; wherein the generator outputs a topic-word distribution in the text dataset; the encoder outputs a document-topic distribution in the text dataset; and inputting the document-subject distribution and the subject-word distribution into a supervised discriminator with a classification function to classify the text.
Preferably, the calculation formula of the tf-idf vector in step S1 is as follows:
tf-idfi，j＝tfi，j×idfi
where | D | represents the total number of documents in the text dataset, | { j: t is ti∈djDenotes the number of documents containing subject words, k denotes the number of preset subjects, ni，jNumber of occurrences of jth word, n, representing ith documentk，jNumber of occurrences of jth word, tf, representing kth documenti，jFrequency, idf, of occurrence of jth word representing ith document in whole text data setijThe ratio of the number of documents representing the jth word in which the ith document appears to the total number of documents, tf-idfi，jAnd expressing the finally obtained word frequency-inverse document frequency.
Preferably, the preprocessing of the text data set in step S1 includes word segmentation, stop word removal, punctuation removal, digit removal, high frequency word removal, and low frequency word removal.
Preferably, the step S2 specifically includes the following steps:
s3.1: initializing and generating parameters of a confrontation topic model;
s3.2: training an encoder using the tf-idf vector;
s3.3: training a generator by using preset Dirichlet distribution sampling data;
s3.4: the output of the coder and the generator is utilized, and the supervised discriminator is trained by using a spectral norm normalization method, so that the aim of confrontation training is fulfilled.
Preferably, the step S3.2 specifically includes the following steps:
s3.2.1: tf-idf vectors of S documents are selected from the tf-idf vector set obtained in step S1
S3.2.2: vector tf-idf
The Yu is input into the encoder to obtain the theme vector
Preferably, the encoder includes a linear connection layer, a BatchNormal layer, a leakyreu layer, a linear connection layer, and a Softmax layer, which are connected in this order.
Preferably, the step S3.3 specifically includes the following steps:
s3.3.1: dirichlet distribution-compliant vectors sampled from a Dirichlet distribution prior distribution
S3.3.2: vector to be subjected to Dirichlet distribution
Inputting the data into a generator to obtain a generated document vector
Preferably, the generator includes a linear connection layer, a BatchNormal layer, a leakyreu layer, a linear connection layer, and a Softmax layer, which are connected in sequence.
Preferably, the step S3.4 specifically includes the following steps:
s3.4.1: vector the subject
And tf-idf vector
Are spliced to obtain
Will be provided with
As a true input vector to the supervised discriminator, r denotes that the vector is from a true sample.
S3.4.2: will generate a document vector
And topic vector
Is spliced to obtain
Will be provided with
As a generating input vector for the supervised discriminator; f denotes that the vector is from the generated samples.
S3.4.3: inputting a true input vector and a generated input vector into a computerSupervising the discriminator to obtain a classification output probability
And supervised discriminator output probability
C represents the number of categories in the text data set, and C belongs to {1,2.. C } and represents the probability that the corresponding sample output by the supervised discriminator is a real sample; if the probability of identifying y-c is the greatest, then the representative supervised discriminator considers the data to be from a true sample.
S3.4.4: the Wasserstein distance is calculated as the total loss function L:
wherein ,
is a prior parameter, lambda, of the Dirichlet distribution determined from the dimensions of the subject distribution1 and λ2Is a parameter that sets the importance of training between two distributions for tuning the model, LadvRepresenting the function of the penalty of confrontation, LclsA classification loss function is represented.
S3.4.5: and (3) calculating an arithmetic mean of all samples, carrying out back propagation by using an RMSprop method in a gradient descent method to minimize a loss function L, and simultaneously limiting propagation of an overlarge gradient by using a spectral norm normalization method to ensure the stability of training until the model converges.
In a second aspect, the present invention further provides a device for constructing a generative confrontation topic model based on spectrum norm normalization, which is applied to the method for constructing a generative confrontation topic model based on spectrum norm normalization in any of the above schemes, and includes:
the data acquisition module is used for acquiring a text data set, preprocessing the text data set, and then performing feature vectorization on the text through a word bag model to obtain a text feature vector set, namely a tf-idf vector set;
a model building module for building a generative confrontational model, wherein the generative confrontational model comprises a generator, an encoder, and a supervised discriminator;
the training module is used for inputting the tf-idf vector into the generated confrontation topic model and carrying out confrontation training on the generated confrontation topic model by using a spectral norm normalization method to obtain a trained generated confrontation topic model; wherein the generator outputs a topic-word distribution in the text dataset; the encoder outputs a document-topic distribution in the text dataset; a supervised discriminator with classification function classifies texts based on an input document-topic distribution and topic-word distribution.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that: by adopting the method and the device for constructing the generated confrontation topic model based on the spectrum norm normalization, a supervised bidirectional generated confrontation network is constructed, and the method of the spectrum norm normalization is introduced to relieve the problem that the generated confrontation network cannot be stably trained and converged, so that the topic model can be stably trained; in addition, the supervised discriminator can effectively extract and utilize the guidance of the label information to improve the quality of the generated theme while ensuring that the generated countermeasure network carries out countermeasure training.
Drawings
FIG. 1 is a flow chart of a method for generating a confrontation topic model construction based on spectral norm normalization.
Fig. 2 is a network architecture diagram for generating a countermeasure topic model.
Fig. 3 is a network configuration diagram of an encoder.
Fig. 4 is a network architecture diagram of the generator.
Fig. 5 is a diagram of a network architecture with a supervised discriminator.
FIG. 6 is a schematic diagram of a device for generating a confrontation topic model based on spectral norm normalization.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
the technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
Referring to fig. 1 to 4, the present embodiment provides a method for generating a confrontation topic model based on spectrum norm normalization, including the following steps:
s1: acquiring a text data set comprising an AGnews news data set and a DBpedia encyclopedia data set, preprocessing the text data set including word segmentation, stop word removal, punctuation mark removal, digit removal, high-frequency word removal and low-frequency word removal, and then performing feature vectorization on the text through a word bag model to obtain a text feature vector set, namely a tf-idf vector set; the tf-idf vector is calculated as follows:
tf-idfi，j＝tfi，j×idfi
where | D | represents the total number of documents in the text dataset, | { j: t is ti∈djDenotes the number of documents containing subject words, k denotes the number of preset subjects, ni，jNumber of occurrences of jth word, n, representing ith documentk，jNumber of occurrences of jth word, tf, representing kth documenti，jFrequency, idf, of occurrence of jth word representing ith document in whole text data setijThe ratio of the number of documents representing the jth word in which the ith document appears to the total number of documents, tf-idfi，jAnd expressing the finally obtained word frequency-inverse document frequency.
S2: constructing a generating confrontation topic model, wherein the generating confrontation topic model comprises a generator, an encoder and a supervised discriminator;
s3: inputting the tf-idf vector into the generated confrontation topic model and performing confrontation training on the generated confrontation topic model by using a spectral norm normalization method to obtain a trained generated confrontation topic model; wherein the generator outputs a topic-word distribution in the text dataset; the encoder outputs a document-topic distribution in the text dataset, specifically comprising the steps of:
s3.1: initializing and generating parameters of a confrontation topic model; the parameters to be initialized comprise a priori parameter alpha of Dirichlet distribution, and preset topic number k, tf-idf vector
Vector sampled from Dirichlet distribution
n is the step ratio of the training encoder to the generator, the size m in the training process. In this embodiment, let α be 10-4，n＝5，m＝512，λ1＝1，λ2＝5。
S3.2: as shown in fig. 3, the encoder is trained using tf-idf vectors; the method specifically comprises the following steps:
s3.2.1: tf-idf vectors of m documents are selected from the tf-idf vector set obtained in step S1
S3.2.2: vector tf-idf
The ChochNormal layer, the LeakyReLU layer, the linear connection layer and the Softmax layer of the encoder are introduced to obtain a theme vector
S3.3: training a generator by using preset Dirichlet distribution sampling data; the method specifically comprises the following steps:
s3.3.1: dirichlet distribution-compliant vectors sampled from a Dirichlet distribution prior distribution
S3.3.2: vectors that would obey the Dirichlet distribution, as shown in FIG. 4
The linear connection layer, the BatchNormal layer, the LeakyReLU layer, the linear connection layer and the Softmax layer of the generator are subjected to Yu to obtain a generated document vector
S3.4: the output of the encoder and the generator is utilized, and a spectral norm normalization method is used for training the supervised discriminator at the same time, so that the aim of confrontation training is fulfilled; the method specifically comprises the following steps:
s3.4.1: vector the subject
And tf-idf vector
Is spliced to obtain
Will be provided with
Fight of steps on the eastern side of the hall where the host stood to welcome the guests is the true input vector to the supervised discriminator, r indicates that the vector is from a true sample.
S3.4.2: will generate a document vector
And topic vector
Is spliced to obtain
Will be provided with
Fight of steps on the eastern side of the hall where the host stood to welcome the guests is the input vector generated by the supervised discriminator, f indicates that the vector is from the generated samples.
S3.4.3: inputting the true input vector and the generated input vector into a supervised discriminator to obtain a classification output probability
And supervised discriminator output probability
C represents the number of categories in the text data set, and C belongs to {1,2.. C } and represents the probability that the corresponding sample output by the supervised discriminator is a real sample; if the probability of identifying y-c is the greatest, then the representative supervised discriminator considers the data to be from a true sample.
S3.4.4: the Wasserstein distance is calculated as the total loss function L:
wherein ,
is a prior parameter, lambda, of the Dirichlet distribution determined from the dimensions of the subject distribution1 and λ2Is a parameter that sets the importance of training between two distributions for tuning the model, LadvRepresenting the function of the penalty of confrontation, LclsA classification loss function is represented.
S3.4.5: an arithmetic mean is calculated for all samples, a RMSprop method in a gradient descent method is used for back propagation to minimize a loss function L, meanwhile, a spectrum norm normalization method is used, a weight matrix is normalized in the training process, namely divided by the spectrum norm of the weight matrix, and propagation of overlarge gradients is limited to ensure the stability of training until the model converges.
The multiplication of the matrix is a linear mapping, for which it is K-Lipschitz (K-Lipschitz continuous) over the entire domain of definition if it is K-Lipschitz at zero; k is a Lipschitz constant, in the embodiment, K is required to be ensured to be 1, so that the parameter matrix A of the supervised discriminator meets 1-Lipschitz continuity, propagation of overlarge gradient is limited, and the stability of training is ensured by ensuring the stability of the parameter change of the matrix A.
For the parameter matrix a of the supervised discriminator, the principle of normalization of the spectral norm is as follows:
let the vector base formed by the eigenvectors of the matrix A be { v }nIs the vector basis with corresponding eigenvalues of { λ }nIs a characteristic value, then there is
That is to say that the first and second electrodes,
if λ1For maximum eigenvalue, only the requirement of satisfying
At this time
That is, the spectrum norm σ (a) of the matrix a, and if the matrix a satisfies 1-Lipschitz continuity, it is only necessary to divide all elements of the matrix a by σ (a) at the same time.
S4: and inputting the document-subject distribution and the subject-word distribution into a supervised discriminator with a classification function to classify the text.
In this embodiment, the fully trained generator and encoder obtained after training can accept document input in the form of tf-idf vectors and output topic distributions corresponding to the documents, i.e., document-topic distributions; if the identity matrix is input to the generator, the generator outputs a matrix of k x v, where k represents the number of topics and v represents the size of the vocabulary, which is also referred to as a document-word matrix, which is typically used to extract the distribution of words in each topic and measure the quality of the topic model in terms of the relevance of the topic words in each topic.
By adopting the method and the device for constructing the generated confrontation topic model based on the spectrum norm normalization, a supervised bidirectional generated confrontation network is constructed, and the method of the spectrum norm normalization is introduced to relieve the problem that the generated confrontation network cannot be stably trained and converged, so that the topic model can be stably trained; in addition, the supervised discriminator can effectively extract and utilize the guidance of the label information to improve the quality of the generated theme while ensuring that the generated countermeasure network carries out countermeasure training. The subject term generated by the embodiment can be more effectively applied to the application of downstream tasks due to the introduced external tag information, and has a stronger application value. Compared with the prior art, the method has the advantages of convenient training, high training speed, good training effect and strong expansibility, and can be widely applied to the fields of emotion analysis, text classification, text clustering, question-answering system, machine translation and the like except for the topic model.
Example 2
Referring to fig. 6, the present embodiment provides a device for constructing a confrontation topic model based on spectrum norm normalization, which is applied to the method for constructing a confrontation topic model based on spectrum norm normalization, provided in embodiment 1, and includes: the device comprises a data acquisition module, a model construction module, a training module and a test module.
In a specific implementation process, a data acquisition module acquires a text data set, preprocesses the text data set, and then performs feature vectorization on a text through a bag-of-words model to obtain a text feature vector set, namely a tf-idf vector set; the model building module builds a generation confrontation topic model, wherein the generation confrontation topic model comprises a generator, an encoder and a supervised discriminator; the training module is used for inputting the tf-idf vector into the generated confrontation topic model and performing confrontation training on the generated confrontation topic model by using a spectral norm normalization method to obtain a trained generated confrontation topic model, wherein the generator outputs topic-word distribution in a text data set; the encoder outputs a document-topic distribution in the text dataset; the test module inputs the document-subject distribution and the subject-word distribution into a supervised discriminator with a classification function to classify the text. .
The terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. A method for generating a confrontation topic model based on spectrum norm normalization is characterized by comprising the following steps:
s1: acquiring a text data set, preprocessing the text data set, and performing feature vectorization on the text to obtain a text feature vector set, namely a tf-idf vector set;
s2: constructing a generating confrontation topic model, wherein the generating confrontation topic model comprises a generator, an encoder and a supervised discriminator;
s3: inputting the tf-idf vector into the generated confrontation topic model and performing confrontation training on the generated confrontation topic model by using a spectral norm normalization method to obtain a trained generated confrontation topic model; wherein the generator outputs a topic-word distribution in the text dataset; the encoder outputs a document-topic distribution in the text dataset; a supervised discriminator with classification function classifies texts based on an input document-topic distribution and topic-word distribution.
2. The method for constructing a confrontational subject model based on spectral norm normalization according to claim 1, wherein the calculation formula of tf-idf vector in step S1 is as follows:
tf﹣idfi,j＝tfi,j×idfi
where | D | represents the total number of documents in the text dataset, | { j: t |, wherei∈djDenotes the number of documents containing subject words, k denotes the number of preset subjects, ni,jNumber of occurrences of jth word, n, representing ith documentk,jNumber of occurrences of jth word, tf, representing kth documenti,jFrequency, idf, of occurrence of jth word representing ith document in whole text data setijThe ratio of the number of documents representing the jth word in which the ith document appears to the total number of documents, tf-idfi,jAnd expressing the finally obtained word frequency-inverse document frequency.
3. The method for constructing a confrontational model based on spectral norm normalization of claim 1, wherein the preprocessing performed on the text data set in step S1 includes word segmentation, stop word removal, punctuation removal, digit removal, high frequency word removal and low frequency word removal.
4. The method for constructing a confrontational subject model based on spectral norm normalization according to claim 1, wherein the step S2 specifically includes the following steps:
s3.1: initializing and generating parameters of a confrontation topic model;
s3.2: training an encoder using the tf-idf vector;
s3.3: training a generator by using preset Dirichlet distribution sampling data;
s3.4: the output of the coder and the generator is utilized, and the supervised discriminator is trained by using a spectral norm normalization method, so that the aim of confrontation training is fulfilled.
5. The method for constructing a generative confrontational topic model based on spectral norm normalization according to claim 4, wherein the step S3.2 specifically comprises the following steps:
s3.2.1: tf-idf vectors of S documents are selected from the tf-idf vector set obtained in step S1
S3.2.2: vector tf-idf
Inputting the theme vector into an encoder to obtain a theme vector
6. The method for constructing the antagonistic topic model based on the spectral norm normalization according to claim 4, wherein the encoder comprises a linear connection layer, a BatchNormal layer, a LeakyReLU layer, a linear connection layer and a Softmax layer which are connected in sequence.
7. The method for constructing a generative confrontational topic model based on spectral norm normalization according to claim 4, wherein the step S2.3 specifically comprises the following steps:
s3.3.1: dirichlet distribution-compliant vectors sampled from a Dirichlet distribution prior distribution
S3.3.2: vector to be subjected to Dirichlet distribution
Inputting the data into a generator to obtain a generated document vector
8. The method for constructing the confrontation topic model based on the spectral norm normalization of claim 7, wherein the generator comprises a linear connection layer, a BatchNormal layer, a LeakyReLU layer, a linear connection layer and a Softmax layer which are connected in sequence.
9. The method for constructing a generative confrontational topic model based on spectral norm normalization according to claim 7, wherein the step S3.4 specifically comprises the following steps:
s3.4.1: vector the subject
And tf-idf vector
Is spliced to obtain
Will be provided with
True input vector as supervised discriminatorAnd r denotes that the vector is from a real sample;
s3.4.2: will generate a document vector
And topic vector
Is spliced to obtain
Will be provided with
As a generating input vector for the supervised discriminator; f denotes that the vector is from the generated samples;
s3.4.3: inputting the true input vector and the generated input vector into a supervised discriminator to obtain a classification output probability
And supervised discriminator output probability
C represents the number of categories in the text data set, and C belongs to {1,2.. C } and represents the probability that the corresponding sample output by the supervised discriminator is a real sample; if the probability of identifying y as c is the maximum, the supervised discriminator considers that the data is from the real sample;
s3.4.4: the Wasserstein distance is calculated as the total loss function L:
L＝λ1Ladv+λ2Lcls
wherein ,
is a prior parameter, lambda, of the Dirichlet distribution determined from the dimensions of the subject distribution1 and λ2Is a parameter that sets the importance of training between two distributions for tuning the model, LadvRepresenting the function of the penalty of confrontation, LclsRepresenting a classification loss function;
s3.4.5: and (3) calculating an arithmetic mean of all samples, carrying out back propagation by using an RMSprop method in a gradient descent method to minimize a loss function L, and simultaneously limiting propagation of an overlarge gradient by using a spectral norm normalization method to ensure the stability of training until the model converges.
10. A device for generating a confrontation topic model based on spectrum norm normalization is characterized by comprising the following steps:
the data acquisition module is used for acquiring a text data set, preprocessing the text data set, and then performing feature vectorization on the text through a word bag model to obtain a text feature vector set, namely a tf-idf vector set;
a model building module for building a generative confrontational model, wherein the generative confrontational model comprises a generator, an encoder, and a supervised discriminator;
the training module is used for inputting the tf-idf vector into the generated confrontation topic model and carrying out confrontation training on the generated confrontation topic model by using a spectral norm normalization method to obtain a trained generated confrontation topic model;
the test module is used for testing the generated confrontation topic model after being trained by the model training module, wherein the generator outputs topic-word distribution in the text data set; the encoder outputs a document-topic distribution in the text dataset; and inputting the document-subject distribution and the subject-word distribution into a supervised discriminator with a classification function to classify the text.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111199957.XA CN113918716B (en) | 2021-10-14 | 2021-10-14 | Method and device for constructing generated countermeasure topic model based on spectrum norm normalization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111199957.XA CN113918716B (en) | 2021-10-14 | 2021-10-14 | Method and device for constructing generated countermeasure topic model based on spectrum norm normalization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113918716A true CN113918716A (en) | 2022-01-11 |
| CN113918716B CN113918716B (en) | 2023-06-02 |
Family
ID=79240433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111199957.XA Active CN113918716B (en) | 2021-10-14 | 2021-10-14 | Method and device for constructing generated countermeasure topic model based on spectrum norm normalization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113918716B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503671A (en) * | 2023-06-25 | 2023-07-28 | 电子科技大学 | Image classification method based on residual network compression of effective rank tensor approximation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084121A (en) * | 2019-03-27 | 2019-08-02 | 南京邮电大学 | Implementation method based on the human face expression migration for composing normalized circulation production confrontation network |
| CN110827213A (en) * | 2019-10-11 | 2020-02-21 | 西安工程大学 | Super-resolution image restoration method based on generation type countermeasure network |
| CN111553587A (en) * | 2020-04-26 | 2020-08-18 | 中国电力科学研究院有限公司 | New energy scene generation method and system based on confrontation learning model |
| CN112597769A (en) * | 2020-12-15 | 2021-04-02 | 中山大学 | Short text topic identification method based on Dirichlet variational self-encoder |
| CN112884856A (en) * | 2021-01-25 | 2021-06-01 | 浙江师范大学 | Text image generation method for generating confrontation network based on spectrum normalization hierarchical level |
-
2021
- 2021-10-14 CN CN202111199957.XA patent/CN113918716B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084121A (en) * | 2019-03-27 | 2019-08-02 | 南京邮电大学 | Implementation method based on the human face expression migration for composing normalized circulation production confrontation network |
| CN110827213A (en) * | 2019-10-11 | 2020-02-21 | 西安工程大学 | Super-resolution image restoration method based on generation type countermeasure network |
| CN111553587A (en) * | 2020-04-26 | 2020-08-18 | 中国电力科学研究院有限公司 | New energy scene generation method and system based on confrontation learning model |
| CN112597769A (en) * | 2020-12-15 | 2021-04-02 | 中山大学 | Short text topic identification method based on Dirichlet variational self-encoder |
| CN112884856A (en) * | 2021-01-25 | 2021-06-01 | 浙江师范大学 | Text image generation method for generating confrontation network based on spectrum normalization hierarchical level |
Non-Patent Citations (2)
| Title |
|---|
| TAKERU MIYATO等: "Spectral normalization for genrative adversarial networks", pages 1 - 26 * |
| 尹春勇;章荪;: "面向短文本情感分类的端到端对抗变分贝叶斯方法", no. 09, pages 64 - 70 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503671A (en) * | 2023-06-25 | 2023-07-28 | 电子科技大学 | Image classification method based on residual network compression of effective rank tensor approximation |
| CN116503671B (en) * | 2023-06-25 | 2023-08-29 | 电子科技大学 | Image classification method based on residual network compression of effective rank tensor approximation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113918716B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rani et al. | Deep learning based sentiment analysis using convolution neural network | |
| Kumar et al. | Aspect-based sentiment analysis using deep networks and stochastic optimization | |
| CN108537257B (en) | Zero sample image classification method based on discriminant dictionary matrix pair | |
| CN112836051B (en) | Online self-learning court electronic file text classification method | |
| Huang et al. | Siamese network-based supervised topic modeling | |
| CN113392179A (en) | Text labeling method and device, electronic equipment and storage medium | |
| Ahmed et al. | Natural language processing and machine learning based cyberbullying detection for Bangla and Romanized Bangla texts | |
| Bikku et al. | Deep learning approaches for classifying data: a review | |
| Li et al. | Semi-supervised learning for text classification by layer partitioning | |
| Qi et al. | Transfer learning of distance metrics by cross-domain metric sampling across heterogeneous spaces | |
| CN117494051A (en) | Classification processing method, model training method and related device | |
| CN115017879A (en) | Text comparison method, computer device and computer storage medium | |
| Ghosh | Sentiment analysis of IMDb movie reviews: A comparative study on performance of hyperparameter-tuned classification algorithms | |
| Jing et al. | BERT for aviation text classification | |
| CN113918716A (en) | Method and device for constructing generation confrontation topic model based on spectrum norm normalization | |
| Meng et al. | Unsupervised word embedding learning by incorporating local and global contexts | |
| CN112528653A (en) | Short text entity identification method and system | |
| CN110198291B (en) | Webpage backdoor detection method, device, terminal and storage medium | |
| Nazarizadeh et al. | Using Group Deep Learning and Data Augmentation in Persian Sentiment Analysis | |
| Noor et al. | Optimization of Sentiment Analysis of Program Sembako (BPNT) Based on Twitter | |
| Lei et al. | Hierarchical recurrent and convolutional neural network based on attention for Chinese document classification | |
| Kurt et al. | Web page classification with deep learning methods | |
| Singh et al. | Twitter data in Emotional Analysis-A study | |
| Zheng et al. | A dictionary-based convolutional recurrent neural network model for sentiment analysis | |
| Nagesh et al. | An exploration of three lightly-supervised representation learning approaches for named entity classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |