CN115169449A - Attribute-level emotion analysis method, system and storage medium based on contrast learning and continuous learning - Google Patents
Attribute-level emotion analysis method, system and storage medium based on contrast learning and continuous learning Download PDFInfo
- Publication number
- CN115169449A CN115169449A CN202210748554.4A CN202210748554A CN115169449A CN 115169449 A CN115169449 A CN 115169449A CN 202210748554 A CN202210748554 A CN 202210748554A CN 115169449 A CN115169449 A CN 115169449A
- Authority
- CN
- China
- Prior art keywords
- bert
- learning
- model
- adapter
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to the field of emotion analysis, in particular to an attribute level emotion analysis method, system and storage medium based on contrast learning and continuous learning, wherein the method comprises the following steps: acquiring text data of product comment; preprocessing the text data; training a plurality of BERT-adapter models by using a transfer learning method, and forming a model set by the trained plurality of BERT-adapter models; wherein, the single BERT-adapter model is obtained by adding an adaptation layer behind a forward propagation network layer in each decoder layer of the BERT model; inputting the preprocessed text data into a model set to obtain a plurality of logic value classification results; and obtaining attribute level emotion analysis results of the text data according to the multiple logic value classification results. The emotion analysis method and the emotion analysis system have the advantages that contrast learning is used in the emotion analysis model, and therefore the model achieves better classification effect. Meanwhile, the model constructed based on the transfer learning idea has the knowledge transfer capability and the knowledge enhancement capability, and the catastrophic forgetting problem in deep learning can be relieved.
Description
Technical Field
The invention relates to the field of emotion analysis in artificial intelligence and natural language processing, in particular to an attribute level emotion analysis method, system and storage medium based on contrast learning and continuous learning.
Background
The attribute-based Sentiment Analysis (ABSA) is a fine-grained Sentiment Analysis task and an important task in the field of natural language processing. One of the tasks of the ABSA is attribute-level sentiment classification. The purpose of the task is that when a given sentence input and a certain attribute word, the model needs to identify and classify the emotion category in the sentence for the attribute word, such as positive, negative or neutral. For example, the text input is: "restaurants are very good, and food is common. ", the attributes are: at present, fine-grained emotion analysis is mainly applied to application scenes such as product comment and public opinion analysis of an internet platform.
BERT (Bidirectional Encoder Representation from transformations) is a pre-trained language Representation model. The BERT model is a deep bi-directional, unsupervised language representation, and is a model that is pre-trained using only a corpus of plain text, and has the ability to address multiple natural language processing tasks. BERT is a stack of multiple decoding layers, where a decoding layer is a decoder based on a multi-headed attention mechanism. The output of BERT is corresponding to the input character one by one, and the fine adjustment is carried out by adding a full connection layer at the last layer, thus being applicable to various downstream tasks of natural language processing and simultaneously achieving good performance improvement. The fine tuning is to load the model parameters of the pre-training model as initial parameters, then to access the downstream task, and to train the downstream task and the pre-training model together. However, since each step of training requires updating all parameters of the model at the same time, this results in a very unfriendly application scenario, especially for low-resource and multi-tasking applications.
Meanwhile, the BERT deep learning model has the following problems:
(1) While more knowledge can be obtained from a data set when the data set sample size is large, it is relatively difficult for a model to learn knowledge of the data in situations where the data set is small or unsupervised.
(2) Even though the current data set may be trained with good results, testing the previous task test set with the current model is not as effective because training the current task training set alters the model parameters from the previous training by back-propagation of the loss function, i.e., catastrophic forgetfulness.
Disclosure of Invention
In view of this, a first objective of the present invention is to provide an attribute level emotion analysis method based on contrast learning and continuous learning, including obtaining a text, preprocessing the text, training a plurality of BERT-adapter models by using a transfer learning method, forming a BERT-adapter model set by the trained plurality of BERT-adapter models, and inputting the preprocessed text into the BERT-adapter model set to obtain an emotion analysis result.
Based on the same inventive concept, the second purpose of the invention is to provide an attribute level emotion analysis system based on comparative learning and continuous learning.
Based on the same inventive concept, a third object of the present invention is to provide a computer-readable storage medium.
The first purpose of the invention can be achieved by adopting the following technical scheme:
an attribute level emotion analysis method based on comparative learning and continuous learning is characterized by comprising the following steps of:
acquiring text data of product comment;
preprocessing the text data;
training a plurality of BERT-adapter models by using a transfer learning method, and forming a BERT-adapter model set by the trained plurality of BERT-adapter models; wherein, the single BERT-adapter model is obtained by adding an adaptation layer behind a forward propagation network layer in each decoder layer of the BERT model;
inputting the preprocessed text data into a BERT-adapter model set to obtain a plurality of logic value classification results;
and obtaining attribute level emotion analysis results of the text data according to the multiple logic value classification results.
Further, the method for preprocessing the text data comprises the following steps:
screening out text data containing sentences and attribute words in the text data;
converting the text data into the following form:
[ CLS ] + Tab set to be classified + original sentence corpus
Where [ CLS ] is the identifier of the beginning of the sentence.
Further, according to the classification result of the plurality of logic values, obtaining an attribute level emotion analysis result of the text data, specifically comprising the following steps:
each model in the BERT-adapter model set obtains a logic value corresponding to each classification result according to the input sample, and selects the classification result with the highest logic value as the output of a single model;
counting the classification results of all models in the BERT-adapter model set to obtain the classification result with the largest statistical total number;
if the classification result with the largest statistical total number is unique, taking the classification result as an attribute level emotion analysis result of the text data;
if the classification result with the most statistical total number is not unique, outputting the result according to the following method:
respectively calculating the difference between the logic value of the classification result and the logic values of other classification results according to the logic value of each BERT-adapter model outputting the classification result to obtain a logic difference value;
calculating the sum of the logic difference values of all models outputting the same classification result to obtain the statistical logic difference value of the classification result;
and comparing the statistical logic difference value of the classification result with the maximum statistical total number, and taking the classification result with the maximum statistical logic difference value as an attribute-level emotion analysis result of the text data.
Further, a plurality of BERT-adapter models are trained by using a transfer learning method, wherein a single BERT-adapter model is obtained by adding an adaptation layer behind a forward propagation network layer in each decoder layer of the BERT model, and the method specifically comprises the following steps:
constructing a transfer learning framework, comprising:
the input representation module is used for processing input information, converting the input information into a mode adaptive to the BERT-adapter model module, generating a task sequence and inputting the input information into the BERT-adapter model module according to the task sequence;
the BERT-adapter model module comprises a plurality of decoder layers, wherein each decoder layer comprises an adaptation layer and a normalization layer, wherein only the adaptation layer and the normalization layer are parameter layers which can be adjusted, and the parameters of the rest network layers are loaded by a pre-training model and cannot be modified;
the comparison learning module is used for calculating a loss function;
the adaptation layer memory module is used for loading and storing parameters of the adaptation layer in the BERT-adapter model module;
and the reverse test voting module is used for generating a BERT-adapter model set and evaluating the performance of the BERT-adapter model set.
Further, when each BERT-adapter model module starts to train, the adaptation layer memory module loads parameters of the adaptation layer in the BERT-adapter model module trained by the last task in the task sequence into the BERT-adapter model module of the current task as initial parameters; and after each BERT-adapter model module is trained, storing parameters of an adaptation layer in the current BERT-adapter model module.
Further, the comparison learning module is configured to calculate a loss function, and specifically includes:
constructing a deformation for the existing data to obtain a sample with the same type but different coding results;
setting that for a certain sample, only the corresponding deformation is the positive sample in the comparative learning, and other samples are all negative samples, wherein the specific calculation formula is as follows:
wherein τ is temperature coefficient (temperature), h i Output an implicit vector for the model of sample i, h j Output an implicit vector, h, for the model of the positive sample j k Outputting an implicit vector for a model of a negative sample k, wherein N is the number of samples; 1 k≠j To indicate a function, k is 1 when k is not equal to j, k = j, and 0 when k is equal to j.
Obtaining h output by BERT-adapter model module [CLS] And h [label_feature] Vector quantity;
according to h [CLS] And h [label_feature] And vector calculation contrast learning loss functions are used as loss functions trained by the BERT-adapter model module.
Further, the comparison learning module also passes h [CLS] And h [label_feature] Vector output is subjected to matrix multiplication to obtain a model prediction result, a cross entropy loss function of the model prediction result and a real label is calculated, a joint loss function is further obtained according to the cross entropy loss function and a comparison learning loss function, and the joint loss function is used as a loss function for BerT-adapter model module training:
l＝α 1 l ce +α 2 l cl
wherein alpha is 1 、α 2 As a constant parameter,/ ce For cross entropy loss function versus learning loss function, l cl A loss function is learned for comparison.
Further, after all training tasks are completed, the reverse test voting module correspondingly combines parameters of the adaptation layer stored in the adaptation layer memory module to generate a BERT-adapter model, and obtains a BERT-adapter model set.
The second purpose of the invention can be achieved by adopting the following technical scheme:
an attribute-level emotion analysis system based on contrast learning and continuous learning is characterized by being realized based on the attribute-level emotion analysis method, and comprising the following steps of:
the text acquisition module is used for acquiring text data of the product comment;
the preprocessing module is used for preprocessing the text data;
the model set training module is used for training a plurality of BERT-adapter models by using a transfer learning method and forming a BERT-adapter model set by the trained plurality of BERT-adapter models;
the emotion analysis module is used for inputting the preprocessed text data into the BERT-adapter model set to obtain a plurality of logic value classification results;
and the result output module is used for obtaining attribute level emotion analysis results of the text data according to the multiple logic value classification results.
The third purpose of the invention can be achieved by adopting the following technical scheme:
a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned attribute-level emotion analysis method based on contrast learning and persistent learning.
Compared with the prior art, the invention has the following beneficial effects:
(1) Based on the label set comparison learning module, the coding feature spaces of the same type of samples of the model are close to each other, and the coding feature spaces of different types of samples are repellent to each other, so that the model has an output space for better expressing the input samples, and the model achieves a better classification effect.
(2) The continuous learning module of the memory pool based on the adaptation layer realizes a memory pool module by storing adaptation layer parameters (adapter) with small parameter characteristics, and then realizes knowledge migration by utilizing the parameters of the previous model so as to help the model to better train the current task data set.
(3) Through the efficient parameter characteristic of the adaptation layer, the model has the knowledge migration capability and also has the knowledge enhancement capability, so that the catastrophic forgetting problem in deep learning can be relieved.
Drawings
Fig. 1 is a schematic diagram of a BERT-adapter training model in embodiment 1 of the present invention.
Fig. 2 is a schematic diagram of a reverse voting module according to embodiment 1 of the present invention.
Fig. 3 is a flowchart of generating a BERT-adapter model set according to embodiment 1 of the present invention.
FIG. 4 is a schematic diagram of a comparative learning module according to embodiment 1 of the present invention.
Fig. 5 is a schematic structural diagram of the BERT-adapter model in embodiment 2 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Example 1:
the embodiment provides an attribute level emotion analysis method based on comparative learning and continuous learning, which comprises the following steps of:
s100, acquiring text data of product comment;
s200, preprocessing the text data, and specifically comprises the following steps:
s201, screening out text data containing sentences and attribute words in the text data;
s202, converting the text data into the following form:
[ CLS ] + Tab set to be classified + original sentence corpus
Wherein [ CLS ] is an identifier of a first part of a sentence, the tag set to be classified may be two classification tag sets { positive, negative } (i.e., { positive, negative }), or three classification tag sets { positive, negative, neutral } (i.e., { positive, negative, neutral }), and in this embodiment, the tag set to be classified is a three classification tag set.
S300, training a plurality of BERT-adapter models by using a transfer learning method, wherein a single BERT-adapter model is obtained by adding an adaptation layer behind a forward propagation network layer in each decoder layer of the BERT model, and the specific training process comprises the following steps:
s301, constructing a transfer learning framework, as shown in fig. 1 and fig. 2, including an input representation module, a BERT-adapter model module, a contrast learning module, an adaptation layer memory module, and a reverse test voting module, specifically:
the input representation module is used for processing input information, converting the input information into a mode adaptive to the BERT-adapter model module, generating a task sequence and inputting the input information into the BERT-adapter model module according to the task sequence;
the BERT-adapter model module comprises a plurality of decoder layers, wherein each decoder layer comprises an adaptation layer and a normalization layer, wherein only the adaptation layer and the normalization layer are parameter layers which can be adjusted, and the parameters of the rest network layers are loaded by a pre-training model and cannot be modified;
the comparison learning module is used for calculating a loss function;
the adaptation layer memory module is used for loading and storing parameters of the adaptation layer in the BERT-adapter model module;
and the reverse test voting module is used for generating a BERT-adapter model set and evaluating the performance of the BERT-adapter model set.
S302, a plurality of data sets belonging to different fields or product critiques are collected through an input representation module, and random sequence sequences of the task data sets are obtained. The data set is divided according to the ratio of 8: 1 to obtain a training set, a testing set and a verification set, and the training set, the testing set and the verification set are further divided according to the expected number of models of the BERT-adapter model set to obtain the training set, the testing set and the verification set of each training. The data set file is saved by a json file, and each sample contains a sentence sample (content), an attribute word (aspect), and an emotion polarity category (polarity).
S303, mapping the emotion to be represented by numbers through an input representation module, namely (positive: 0, negative: 1, neutral: 2). The sentences and the attribute words of the samples are character string objects, and the sentence and the attribute words are segmented by utilizing a segmentation tool carried by a BERT model method, so that segmentation sequences of the sentences and the attribute words, namely character sequences, can be obtained respectively. And splicing the sentence character sequence and the attribute word character sequence to obtain an input character sequence. Embedding the category characteristic characters in front of the input character sequence to obtain a final input character sequence. And constructing a uniform length pair sequence by setting the upper limit of the length of the character sequence.
And S304, obtaining the current training task of the training model according to the sequence of the given task data set. If the current task is the 0 th task, initializing parameters in the BERT-adapter model in a parameter random initialization mode; otherwise, loading the parameters of the adaptation layer of the previous task from the adaptation layer network memory.
S305, inputting the sample based on the label information constructed in the step S303 into a BERT-adapter model to obtain a hidden vector output by the model, namely h [CLS] And h [label_feature] And (5) vector quantity.
S306, inputting the stealth quantity output by the model into a comparison learning module based on h [CLS] And h [label_feature] Calculating matrix multiplication, i.e. h [CLS] ·h [label_feature] And = output, the calculated logic value of the model for each class of the sample is obtained, and the subscript of the maximum value is the classification result. Performing cross loss entropy calculation based on the classification result and the real label; and will h [CLS] And h [label_feature] And (5) performing comparative learning loss function calculation.
As shown in fig. 4, in this embodiment, the calculating the loss function specifically includes:
constructing a deformation for the existing data to obtain a sample with the same type but different coding results;
setting that for a certain sample, only the corresponding deformation is the positive sample in the comparative learning, and other samples are all negative samples, wherein the specific calculation formula is as follows:
wherein τ is temperature coefficient (temperature), h i Output an implicit vector for the model of sample i, h j Output an implicit vector, h, for the model of the positive sample j k Outputting an implicit vector for a model of a negative sample k, wherein N is the number of samples; 1 k≠j To indicate a function, k is 1 when k is not equal to j, k = j, and 0 when k is equal to j.
According to h [CLS] And h [label_feature] Vector calculation contrasts with a learning loss function; through h [CLS] And h [label_feature] Vector output is subjected to matrix multiplication to obtain a model prediction result, a cross entropy loss function of the model prediction result and a real label is calculated, a joint loss function is further obtained according to the cross entropy loss function and a comparison learning loss function, and the joint loss function is used as a loss function for BerT-adapter model module training:
l＝α 1 l ce +α 2 l cl
wherein alpha is 1 、α 2 As a constant parameter,/ ce For cross entropy loss function versus learning loss function, l cl A loss function is learned for comparison.
S307, calculating a cross loss entropy function and a combined loss function of comparison learning, reducing the gradient to minimize the combined loss function, completing single training, and storing adaptation layer parameters of the BERT-adapter model into an adaptation layer memory module.
And S308, repeating the steps S302-S307 until the training of the training set sample of the task is finished.
S309, constructing a BERT-adapter model set from the adaptation layer memory pool by using a reverse test voting module, and evaluating the performance of the BERT-adapter model set by using a reverse test index.
In this embodiment, the reverse test index is constructed as follows: when the sequence of a certain task is given and the model trains the kth task, the current existing model is used for testing a test set of the previous k-1 task data sets, and the accuracy (accuracy) and macro-f1 of the prediction result are calculated, which are called the reverse accuracy and the reverse macro-f1. The indexes of the forward test are as follows: when the model finishes the kth task, testing the test set of the current task data set by using the current model, and calculating the accuracy (accuracycacy) and macro-f1 of the prediction result, namely the accuracy of the forward direction and the macro-f1 of the forward direction.
In this embodiment, when the model has finished training the kth task, the adapter and BERT models in the adaptation layer memory pool are constructed into k BERT-adapter models. Secondly, when testing the testing set of the ith (i is more than or equal to 0 and less than k), the task testing set can obtain the prediction logic value result of each model through calculation of the BERT-adapter model set. Subsequently, a set of test sets output by the set of models is voted: traversing the sample of each test set, and if the model set has a majority (such as positive) for a certain category of the predicted result, determining that the sample finally predicts the certain category (such as positive); and if the prediction results of each category are the same ticket number, calculating the accumulated logic value difference of each category to the rest categories, and calculating to obtain the category with the largest accumulated logic value difference as the final prediction result of the sample.
S400, inputting the preprocessed text data into a BERT-adapter model set to obtain a plurality of logic value classification results;
s500, obtaining attribute level emotion analysis results of the text data according to the classification results of the plurality of logic values, and specifically comprising the following steps:
s510, each model in the BERT-adapter model set obtains a logic value corresponding to each classification result according to the input sample, and selects the classification result with the highest logic value as the output of a single model;
s520, counting the classification results of all models in the BERT-adapter model set to obtain the classification result with the largest statistical total number;
s530, if the classification result with the largest statistical sum is unique, taking the classification result as an attribute level emotion analysis result of the text data;
s540, if the classification result with the most statistical total number is not unique, outputting the result according to the following method:
s541, respectively calculating the difference between the logic value of the classification result and the logic values of other classification results according to the logic value of each BERT-adapter model outputting the classification result to obtain a logic difference value;
s542, calculating the sum of the logic difference values of all models outputting the same classification result to obtain the statistical logic difference value of the classification result;
and S543, comparing the statistical logic difference value of the classification result with the largest statistical sum, and taking the classification result with the largest statistical logic difference value as an attribute-level emotion analysis result of the text data.
In this embodiment, the tag set-based comparison learning module can make the coding feature spaces of the same type of samples of the model similar to each other, and make the coding feature spaces of different types of samples repel each other, so that the model has an output space for better expressing the input samples, and the model achieves a better classification effect; meanwhile, the continuous learning module of the memory pool based on the adaptation layer realizes a memory pool module by storing adaptation layer parameters (adapter) with small parameter characteristics, and further realizes knowledge migration by utilizing the parameters of the previous model so as to help the model to better train the current task data set; moreover, through the efficient parameter characteristic of the adaptation layer, the model has the knowledge migration capability and also has the knowledge enhancement capability, so that the catastrophic forgetting problem existing in deep learning can be relieved.
Example 2:
the embodiment provides an attribute level emotion analysis system based on contrast learning and continuous learning, which comprises a text acquisition module, a preprocessing module, a model set training module, an emotion analysis module and a result output module, and realizes the attribute level emotion analysis method based on the contrast learning and the continuous learning in the embodiment 1 of the invention. The details of each part of this embodiment are as follows:
the text acquisition module is used for acquiring text data of the product comment;
the preprocessing module is used for preprocessing the text data;
the model set training module is used for training a plurality of BERT-adapter models by using a transfer learning method and forming a BERT-adapter model set by the trained plurality of BERT-adapter models;
the emotion analysis module is used for inputting the preprocessed text data into the BERT-adapter model set to obtain a plurality of logic value classification results;
and the result output module is used for obtaining attribute level emotion analysis results of the text data according to the multiple logic value classification results.
In this embodiment, the structure of the BERT-adapter model is shown in fig. 5. Wherein, only the adaptation layer and the normalization layer in the BERT layer are parameter layers which can be adjusted and are initialized randomly, and the network layers of the rest BERT models are loaded by a pre-training model and can not be modified. The network structure of the adaptation layer in the decoder layer is formed by overlapping two modules including a linear full connection layer and a nonlinear connection layer. And the hidden vector of the input adaptation layer in the model and the output of the adaptation layer are added and input to the normalization layer. Wherein the input embedding is the sum of word embedding, position embedding and segment embedding. Input into BERT self-attention layer to make multi-head attention-based calculation. The output of a certain decoder layer is obtained through calculation of a full connection layer, a random inactivation layer, an adaptation layer, a normalization layer and the like, and the output is continuously input to a decoder layer of the next layer to perform calculation of the same operation. And calculating to the last decoder layer to obtain the final model output.
Example 3:
the present embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program, and when the computer program is executed by a processor, the method for attribute-level emotion analysis based on contrast learning and persistent learning of embodiment 1 is implemented, specifically as follows:
acquiring text data of product comment;
preprocessing the text data;
training a plurality of BERT-adapter models by using a transfer learning method, wherein a single BERT-adapter model is obtained by adding an adaptation layer behind a forward propagation network layer in each decoder layer of the BERT model;
forming a BERT-adapter model set by the trained plurality of BERT-adapter models;
inputting the preprocessed text data into a BERT-adapter model set to obtain a plurality of logic value classification results; and obtaining attribute level emotion analysis results of the text data according to the multiple logic value classification results.
It should be noted that the computer readable storage medium of the present embodiment may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In the present embodiment, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this embodiment, however, a computer readable signal medium may include a propagated data signal with a computer readable program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer-readable storage medium may be written with a computer program for performing the present embodiments in one or more programming languages, including an object oriented programming language such as Java, python, C + +, and conventional procedural programming languages, such as C, or similar programming languages, or combinations thereof. The program may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be understood that the above-described embodiments are only a few embodiments of the present invention, and the present invention is not limited to the details of the above-described embodiments, and that any suitable changes or modifications thereof by one of ordinary skill in the art may be made without departing from the scope of the present invention.
Claims (10)
1. An attribute level emotion analysis method based on comparative learning and continuous learning is characterized by comprising the following steps of:
acquiring text data of product comment;
preprocessing the text data;
training a plurality of BERT-adapter models by using a transfer learning method, and forming a BERT-adapter model set by the trained plurality of BERT-adapter models; wherein, the single BERT-adapter model is obtained by adding an adaptation layer behind a forward propagation network layer in each decoder layer of the BERT model;
inputting the preprocessed text data into a BERT-adapter model set to obtain a plurality of logic value classification results; and obtaining an attribute level emotion analysis result of the text data according to the classification result of the plurality of logic values.
2. The method for analyzing emotion of attribute level based on comparative learning and continuous learning as claimed in claim 1, wherein the preprocessing is performed on text data, comprising the steps of:
screening out text data containing sentences and attribute words in the text data;
converting the text data into the following form:
[ CLS ] + Tab set to be classified + original sentence corpus
Where [ CLS ] is the identifier of the beginning of the sentence.
3. The method for analyzing attribute-level emotion based on comparative learning and continuous learning as claimed in claim 1, wherein the result of attribute-level emotion analysis of text data is obtained according to the result of classification of a plurality of logical values, and the specific steps are as follows:
each model in the BERT-adapter model set obtains a logic value corresponding to each classification result according to the input sample, and selects the classification result with the highest logic value as the output of a single model;
counting the classification results of all models in the BERT-adapter model set to obtain the classification result with the largest statistical total number;
if the classification result with the largest statistical total number is unique, taking the classification result as an attribute level emotion analysis result of the text data;
if the classification result with the most statistical total number is not unique, outputting the result according to the following method:
respectively calculating the difference between the logic value of the classification result and the logic values of other classification results according to the logic value of each BERT-adapter model outputting the classification result to obtain a logic difference value;
calculating the sum of the logic difference values of all models outputting the same classification result to obtain the statistical logic difference value of the classification result;
and comparing the statistical logic difference value of the classification result with the maximum statistical total number, and taking the classification result with the maximum statistical logic difference value as the attribute level emotion analysis result of the text data.
4. The method of analyzing emotion of attribute level based on releative learning and persistent learning as claimed in claim 1, wherein a plurality of BERT-adapter models are trained using a migratory learning method, wherein a single BERT-adapter model is obtained by adding an adaptation layer after a forward propagation network layer in each decoder layer of the BERT model, and specifically comprises:
constructing a transfer learning framework, comprising:
the input representation module is used for processing input information, converting the input information into a mode adaptive to the BERT-adapter model module, generating a task sequence and inputting the input information into the BERT-adapter model module according to the task sequence;
the BERT-adapter model module comprises a plurality of decoder layers, wherein each decoder layer comprises an adaptation layer and a normalization layer, wherein only the adaptation layer and the normalization layer are parameter layers which can be adjusted, and the parameters of the rest network layers are loaded by a pre-training model and cannot be modified;
the comparison learning module is used for calculating a loss function;
the adaptation layer memory module is used for loading and storing parameters of the adaptation layer in the BERT-adapter model module;
and the reverse test voting module is used for generating a BERT-adapter model set and evaluating the performance of the BERT-adapter model set.
5. The method for analyzing attribute-level emotion based on contrastive learning and continuous learning of claim 4, wherein the adaptation layer memory module loads the parameters of the adaptation layer in the BERT-adapter model module trained by the previous task in the task sequence as initial parameters when each BERT-adapter model module starts training; and after each BERT-adapter model module is trained, storing parameters of an adaptation layer in the current BERT-adapter model module.
6. The attribute-level emotion analysis method based on contrastive learning and continuous learning as claimed in claim 4, wherein the contrastive learning module is configured to calculate a loss function, and specifically includes:
constructing a deformation for the existing data to obtain a sample with the same type but different coding results;
setting that for a certain sample, only the corresponding deformation is the positive sample in the comparative learning, and other samples are all negative samples, wherein the specific calculation formula is as follows:
wherein τ is temperature coefficient (temperature), h i Output implicit vector for model of sample i, h j Output an implicit vector, h, for the model of the positive sample j k Outputting an implicit vector for a model of a negative sample k, wherein N is the number of samples; 1 k≠j To indicate a function, k is 1 when k is not equal to j, k = j is 0;
obtaining h output by BERT-adapter model module [CLS] And h [label_feature] A vector;
according to h [CLS] And h [label_feature] And vector calculation contrast learning loss functions are used as loss functions trained by the BERT-adapter model module.
7. The method of attribute-level sentiment analysis based on reinforcement learning and persistence learning of claim 6, wherein the reinforcement learning module further passes h [CLS] And h [label_feature] Vector output is subjected to matrix multiplication to obtain a model prediction result, a cross entropy loss function of the model prediction result and a real label is calculated, a joint loss function is further obtained according to the cross entropy loss function and a comparison learning loss function, and the joint loss function is used as a loss function for BerT-adapter model module training:
l＝α 1 l ce +α 2 l cl
wherein alpha is 1 、α 2 As a constant parameter,/ ce For a cross-entropy loss function,/ cl A loss function is learned for comparison.
8. The attribute-level emotion analysis method based on contrastive learning and continuous learning as claimed in claim 4, wherein the backward test voting module correspondingly combines and generates a BERT-adapter model according to the parameters of the adaptation layer stored in the adaptation layer memory module after all training tasks are completed, and a BERT-adapter model set is obtained.
9. An attribute-level emotion analysis system based on comparative learning and continuous learning, comprising:
the text acquisition module is used for acquiring text data of the product comment;
the preprocessing module is used for preprocessing the text data;
the model set training module is used for training a plurality of BERT-adapter models by using a transfer learning method and forming a BERT-adapter model set by the trained plurality of BERT-adapter models;
the emotion analysis module is used for inputting the preprocessed text data into the BERT-adapter model set to obtain a plurality of logic value classification results;
and the result output module is used for obtaining the attribute level emotion analysis result of the text data according to the classification result of the plurality of logic values.
10. A computer-readable storage medium, storing a computer program, wherein the computer program, when executed by a processor, implements the attribute-level emotion analysis method based on contrastive learning and persistent learning according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210748554.4A CN115169449A (en) | 2022-06-29 | 2022-06-29 | Attribute-level emotion analysis method, system and storage medium based on contrast learning and continuous learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210748554.4A CN115169449A (en) | 2022-06-29 | 2022-06-29 | Attribute-level emotion analysis method, system and storage medium based on contrast learning and continuous learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115169449A true CN115169449A (en) | 2022-10-11 |
Family
ID=83488745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210748554.4A Pending CN115169449A (en) | 2022-06-29 | 2022-06-29 | Attribute-level emotion analysis method, system and storage medium based on contrast learning and continuous learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115169449A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028630A (en) * | 2023-03-29 | 2023-04-28 | 华东交通大学 | Implicit chapter relation recognition method and system based on contrast learning and Adapter network |
| CN116204642A (en) * | 2023-03-06 | 2023-06-02 | 上海阅文信息技术有限公司 | Intelligent character implicit attribute recognition analysis method, system and application in digital reading |
| CN116541523A (en) * | 2023-04-28 | 2023-08-04 | 重庆邮电大学 | Legal judgment public opinion classification method based on big data |
-
2022
- 2022-06-29 CN CN202210748554.4A patent/CN115169449A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116204642A (en) * | 2023-03-06 | 2023-06-02 | 上海阅文信息技术有限公司 | Intelligent character implicit attribute recognition analysis method, system and application in digital reading |
| CN116204642B (en) * | 2023-03-06 | 2023-10-27 | 上海阅文信息技术有限公司 | Intelligent character implicit attribute recognition analysis method, system and application in digital reading |
| CN116028630A (en) * | 2023-03-29 | 2023-04-28 | 华东交通大学 | Implicit chapter relation recognition method and system based on contrast learning and Adapter network |
| CN116028630B (en) * | 2023-03-29 | 2023-06-02 | 华东交通大学 | Implicit chapter relation recognition method and system based on contrast learning and Adapter network |
| CN116541523A (en) * | 2023-04-28 | 2023-08-04 | 重庆邮电大学 | Legal judgment public opinion classification method based on big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111783474B (en) | Comment text viewpoint information processing method and device and storage medium | |
| CN111737476B (en) | Text processing method and device, computer readable storage medium and electronic equipment | |
| CN111291183B (en) | Method and device for carrying out classification prediction by using text classification model | |
| CN111444340B (en) | Text classification method, device, equipment and storage medium | |
| CN110222188B (en) | Company notice processing method for multi-task learning and server | |
| Muangkammuen et al. | Automated thai-faq chatbot using rnn-lstm | |
| CN115169449A (en) | Attribute-level emotion analysis method, system and storage medium based on contrast learning and continuous learning | |
| CN111738003B (en) | Named entity recognition model training method, named entity recognition method and medium | |
| CN110795944A (en) | Recommended content processing method and device, and emotion attribute determining method and device | |
| CN112487189B (en) | Implicit discourse text relation classification method for graph-volume network enhancement | |
| CN111625634A (en) | Word slot recognition method and device, computer-readable storage medium and electronic device | |
| CN111259647A (en) | Question and answer text matching method, device, medium and electronic equipment based on artificial intelligence | |
| CN113392209A (en) | Text clustering method based on artificial intelligence, related equipment and storage medium | |
| CN112667782A (en) | Text classification method, device, equipment and storage medium | |
| CN111814489A (en) | Spoken language semantic understanding method and system | |
| CN114372475A (en) | Network public opinion emotion analysis method and system based on RoBERTA model | |
| CN110826327A (en) | Emotion analysis method and device, computer readable medium and electronic equipment | |
| CN110826315B (en) | Method for identifying timeliness of short text by using neural network system | |
| US20180082194A1 (en) | Collective matrix factorization for multi-relational learning | |
| CN112035629B (en) | Method for implementing question-answer model based on symbolized knowledge and neural network | |
| CN113449524B (en) | Named entity identification method, system, equipment and medium | |
| CN115098681A (en) | Open service intention detection method based on supervised contrast learning | |
| CN114595324A (en) | Method, device, terminal and non-transitory storage medium for power grid service data domain division | |
| CN113420561B (en) | Named entity identification method, device, equipment and storage medium | |
| CN111639483B (en) | Evaluation aspect determining method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |