CN113326695A - Emotion polarity analysis method based on transfer learning - Google Patents
Emotion polarity analysis method based on transfer learning Download PDFInfo
- Publication number
- CN113326695A CN113326695A CN202110455888.8A CN202110455888A CN113326695A CN 113326695 A CN113326695 A CN 113326695A CN 202110455888 A CN202110455888 A CN 202110455888A CN 113326695 A CN113326695 A CN 113326695A
- Authority
- CN
- China
- Prior art keywords
- model
- emotion polarity
- sentence
- polarity analysis
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000008451 emotion Effects 0.000 title claims abstract description 66
- 238000004458 analytical method Methods 0.000 title claims abstract description 56
- 238000013526 transfer learning Methods 0.000 title claims abstract description 19
- 238000012163 sequencing technique Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 230000005012 migration Effects 0.000 claims description 4
- 238000013508 migration Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000002457 bidirectional effect Effects 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims description 2
- 230000000717 retained effect Effects 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 230000001617 migratory effect Effects 0.000 claims 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008447 perception Effects 0.000 description 5
- 230000007547 defect Effects 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000288105 Grus Species 0.000 description 1
- 238000002679 ablation Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/237—Lexical tools
- G06F40/247—Thesauruses; Synonyms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses an emotion polarity analysis method based on transfer learning. And then, constructing an emotion polarity analysis model, and transferring the relevant parameters obtained by training the text sequencing model to the corresponding positions in the emotion polarity analysis model. And finally, continuing training the migrated model on the emotion polarity analysis data set. The invention introduces domain prior knowledge into the emotion polarity analysis model by using transfer learning, can improve the domain adaptability of the model, is beneficial to acquiring higher-quality sentence characteristics, and further improves the classification accuracy of the model.
Description
Technical Field
The invention relates to an emotion polarity analysis method based on transfer learning, and belongs to the field of internet and natural language processing.
Background
With the continuous development and progress of mobile internet technology, the general public and the mass of news media are more and more inclined to release the view and attitude of real events in social networks. The emotion polarity analysis is a method for automatically acquiring text emotion tendencies or emotion categories by using a correlation theory of natural language processing, and has great utilization value.
At present, the research on emotional polarity analysis at home and abroad has achieved abundant achievements, and has reference significance for the research work of the invention. The existing emotion polarity analysis method is mainly divided into 3 types based on emotion dictionaries, machine learning and deep learning. The method based on the emotion dictionary introduces expert knowledge into the calculation process of the emotion value, is suitable for scenes lacking large-scale prediction, but has the defects of continuous expansion and poor portability. Compared with the method based on the emotion dictionary, the machine learning method has the advantages of simple modeling and better expansibility and portability. However, the machine learning method requires a high-quality labeled data set, and a certain labeling cost is consumed. The deep learning based method utilizes a neural network model to train a classifier. Compared with the method based on the emotion dictionary and machine learning, the deep learning model has stronger expression capability and obtains better classification indexes. In recent years, with the development of pre-training language models, a deep learning-based mode has achieved better effects. However, considering a specific application scenario, the existing deep learning model still has shortcomings. First, the existing language model lacks a sentence-level pre-training task, so that the logic perception and semantic expression capability of the model still have room for improvement. In addition, for emotion analysis of social comments, the content of the social comments often has the characteristics of being short and random, so that the sentence features are not dense, the noise is high, and the classification robustness of the emotion polarity analysis model is insufficient.
The invention provides an emotion polarity analysis method based on transfer learning, aiming at the problems of insufficient logic perception and semantic expression capability and insufficient classification robustness of a current deep learning model. On one hand, the model uses a transfer learning method, firstly, the logic perception and semantic expression capability of the model is trained in a text sequencing task, and then relevant model parameters are transferred to an emotion polarity analysis model. Through transfer learning, the model can obtain domain prior knowledge, obtain high-quality sentence characteristics and improve the classification accuracy of the model. On the other hand, after the emotion polarity analysis model extracts sentence features, feature noise is further reduced by combining an attention mechanism, and the classification robustness of the model can be improved.
Disclosure of Invention
Aiming at the problems and the defects in the existing emotion polarity analysis technology, the invention provides an emotion polarity analysis method based on transfer learning. Meanwhile, noise interference is reduced for the emotion polarity analysis model based on the attention mechanism, and the classification robustness of the model can be improved.
In order to achieve the purpose, the technical scheme of the invention is as follows: a sentiment polarity analysis method based on transfer learning includes that firstly, positions of characters or words in comment texts are disturbed according to a certain proportion. And then, constructing a text sequencing model, taking the disturbed sentences as input, and taking the sentences with normal word sequence as output to train the model. And then, constructing an emotion polarity analysis model, and transferring the relevant parameters in the text sequencing model to the corresponding positions in the emotion polarity analysis model. And finally, continuing training the emotion polarity analysis model after the migration. The method introduces priori knowledge into the emotion polarity analysis model through transfer learning, can improve the field adaptability of the model, is beneficial to acquiring sentence features with higher quality, and further improves the classification accuracy of the model.
An emotion polarity analysis method based on transfer learning mainly comprises 4 steps, and specifically comprises the following steps:
step 1, a sentence pair data set is constructed. And (3) disturbing the word position of each sentence in the emotion polarity analysis data set according to a set proportion (the proportion is determined according to the comparison experiment result), simultaneously reserving the sentences before disturbance, and forming a piece of training data in the new data set by each group of disturbed sentences and sentences with normal word order.
And 2, training a text sequencing model. Constructing a text sequencing model based on a seq2seq mode, firstly, inputting a disturbed sentence as a model, and extracting sentence characteristics by using an encoder; then decoding word by word, and predicting and outputting words according to the decoding characteristics of the current time step; and finally, comparing the output of the model with the characters at the corresponding positions of the normal word order, and training the parameters of the model based on the cross entropy loss function.
And 3, parameter migration. Firstly, constructing an emotion polarity analysis model, using a coding structure the same as that of the text sequencing model, and then transferring coding parameters and word vector parameters of the text sequencing model into the emotion polarity analysis model.
And 4, training an emotion polarity analysis model. Analyzing a data set based on emotion polarity, firstly inputting a comment text, extracting sentence features by using an encoder, and then further extracting local features, overall features and final features after noise reduction of the sentence by using a convolutional neural network, a cyclic neural network and an attention mechanism; and finally classifying the features.
Compared with the prior art, the invention has the following technical effects:
1. the method is based on transfer learning, firstly a text sequencing model is constructed, and then parameters learned by the text sequencing model are transferred to an emotion polarity analysis model. The method effectively makes up the defect that the current language model lacks of sentence-level pre-training tasks, can improve the logic perception and semantic expression capability of the model, and obtains higher-quality sentence characteristic representation. In the embodiment, the classification accuracy index of the model after the transfer learning is improved by 3.7%, and the effectiveness of the scheme is verified.
2. The emotion polarity analysis model firstly uses CNN (convolutional neural network) to extract partial characteristics of a sentence, then uses BiGRU (bidirectional gated cyclic unit) to extract overall characteristics of the sentence, and finally uses an attention mechanism to reduce characteristic noise, so that the interference of social text randomness on model classification can be reduced. Ablation experiments on the examples show that: compared with the direct classification of the overall features, the increased attention mechanism achieves higher classification robustness.
Drawings
Fig. 1 is an overall frame diagram of an embodiment of the present invention.
Fig. 2 is a frame diagram of a text ordering model according to an embodiment of the present invention.
FIG. 3 is a framework diagram of an emotion polarity analysis model according to an embodiment of the present invention.
Detailed Description
The invention will be further illustrated with reference to specific examples in order to provide a better understanding and appreciation of the invention.
Example 1: an emotion polarity analysis method based on transfer learning is specifically implemented, and the overall working flow of the emotion polarity analysis method is shown in fig. 1. According to the method, a text sequencing model is firstly constructed as shown in FIG. 2, the model makes up for the defect that the current language model lacks a sentence-level pre-training task, and the logic perception and semantic expression capability of the model can be improved. Then, constructing an emotion polarity analysis model as shown in fig. 3, and migrating relevant parameters obtained by training the text ordering model to corresponding positions in the emotion polarity analysis model. And finally, continuing training the migrated model on the emotion polarity analysis data set. The method comprises the following specific implementation steps:
step 1, a sentence pair data set is constructed. Sentences in the emotion polarity analysis data set are scrambled according to a set proportion (set to 25% in this embodiment after a comparison test), sentences before scrambling are retained, and a sentence pair consisting of the scrambled sentences and sentences in a normal word order is used as a piece of training data in the new data set. For example, sentences in normal word order are: all medical personnel are hard. The scrambled sentences are: the medical staff is bitter. Where the context of both "pungent" and "bitter" changes.
And 2, training a text sequencing model. Constructing a text sequencing model in a seq2seq mode, firstly inputting a disturbed sentence as a model, and extracting sentence characteristics by using an encoder; then, decoding word by word, and predicting characters to be output according to the decoding characteristics of the current time step; finally, comparing the output of the model with the normal language sequence, and training the parameters of the model by using a logarithmic loss function. This step can be divided into 3 sub-steps, and the specific implementation is as follows.
Substep 2-1, sentence coding. In the embodiment, text coding features are extracted by Bert, and for convenience of description, meanings of model-related symbols are summarized in table 1. Firstly, inputting a text sequence X (token) with the length of m1,token2,…,tokenm) Index id in vocab according to tokeniExtracting token's coded EmbiAnd calculate eiA text matrix consisting of the entire sentence denoted by s, eiThe specific calculation process of s is shown in formulas (1) and (2), wherein position represents position coding, and segment represents segment coding.
TABLE 1 model-related symbolic meanings
ei＝Bert(Embi+segmenti+postioni) (1)
s＝(e1,e2,…,em) (2)
And then feeding S into a coding model containing 12 layers of transform sequences to extract a final coding output S. In equation (3), LN is the slice normalization operation, and MSA is the multi-headed self-attention operation. Taking the z-th layer as an example, the coded output s of the previous layer is first processed by MSAz-1Then obtained through residual error and LN operation
Finally, FFN pairs are used as shown in formula (3)
Processing and combining residual and LN acquisition layer coded output szAs shown in equation (4). The calculation of FFN is shown in equation (5), where W1、b1、W2、b2Are all model learnable parameters.
FFN(x)＝max(0,xW1+b1)W2+b2 (5)
Substep 2-2, sentence decoding. GRUs are used as the basic unit of the decoding network. The decoding process is shown in equations (6) and (7). In the formula (6), dt-1Represents input, ht-1And representing the hidden layer output of the previous step. d0Corresponding to CLS (a special character in vocab, indicating the beginning of a sentence) as input. Formula (7) represents the implicit layer initialization of the decoding process, i.e. the coded output S is averaged and pooled first, and then the linear layer is used for processing to represent the initial implicit layer input h0Wherein W iss、bsAre all model learnable parameters.
ht＝GRU(dt-1,ht-1) (6)
h0＝Wsavg(S)+bs (7)
Substep 2-3, outputting the prediction, outputting h with the decoding hidden layer of each steptAs query, using the coded output S as key and value, calculating context vector context by dot-product entry as shown in formula (8), and then splicing context and htAnd (3) obtaining the predicted probability distribution p of the model after the final characteristic of the current decoding step and the splicing characteristic are processed by linear transformation and a softmax function, wherein the process is shown as formula (9), wherein W isp、bpAre all model learnable parameters. Finally, the model log loss is calculated based on the predicted value p and the actual value y as shown in equation (10), where m represents the dictionary vocab size.
context＝Attention(ht,S,S) (8)
p＝softmax(Wp[context,ht]+bp) (9)
And 3, parameter migration. And after the text sequencing model is trained, constructing an emotion polarity analysis model by using the same coding structure as the text sequencing model. And then, transferring the coding model parameters and the word vector parameters of the text sequencing model into an emotion polarity analysis model.
And 4, training an emotion polarity analysis model. Based on the emotion polarity analysis data set, firstly, inputting a text into a model, extracting sentence features by using an encoder, and then extracting local features, overall features and final features after noise reduction of the sentence by using a convolutional neural network, a cyclic neural network and an attention mechanism; and finally classifying the features. This step can be divided into 5 sub-steps, and the specific implementation is as follows.
Substep 4-1, sentence coding. Since the coding model is completely consistent with the text ordering model, the coding process of the emotion polarity model is the same as the substep 2-1. The coded output is denoted by S.
And a substep 4-2 of extracting local features. Extracting a local feature representation T of the coded output S using a one-dimensional convolutional network, as shown in equation (11); specific calculation result T of each stepiAs shown in equation (12), where W and b are model learnable parameters; x represents the convolution calculation; i represents the ith step; k denotes the convolution kernel width (this embodiment uses a convolution kernel of width 2).
T＝Conv(S) (11)
Ti＝tanh(W×Si:i+k-1+b) (12)
And a substep 4-3 of extracting the integral features. The extraction of sentence integral characteristics C by Bi-GRU is shown in the formulas (13) and (14). The bidirectional GRU includes a forward GRU with read T generation from left to right
One reverse GRU generation
nt represents the length of the convolution output of the previous step.
And
respectively representing hidden layer output in two directions when the j step of the GRU model is carried out. Will be provided with
And
obtaining the characteristic h of each step after splicingjAs shown in equation (15).
Substep 4-4, attention noise reduction. The actual comment text may be more noisy and it is generally undesirable to make a final prediction of all the parts of the sentence that are input without difference. In combination with the attention mechanism, further optimization of the text feature C can be made. H is to bejObtaining v by perceptron (MLP)jAs in equation (16), where WaAnd baParameters may be learned for the model. By vjThe importance of the similarity measure to context C is shown in equation (17). Calculating final semantic feature C of sentence by means of weighted summationaAs shown in equation (18).
vj＝tanh(Wahj+ba) (16)
Substep 4-5, emotion polarity classification. First, sentence characteristic CaAfter linear change and softmax function processing, the classification probability distribution p of the model is obtained, and the process is shown as formula (19), wherein WpAnd bpLearning parameters for the model; the model log-loss is then calculated based on p and the actual label y as shown in equation (20), where n represents the number of classes.
p＝softmax(WpCa+bp) (19)
It should be understood that these examples are given solely for the purpose of illustration and are not to be construed as limitations on the scope of the invention, as various equivalent modifications of the invention will become apparent to those skilled in the art upon reading the present disclosure and may be made within the scope of the appended claims.
Claims (7)
1. An emotion polarity analysis method based on transfer learning is characterized by comprising the following steps:
step 1, constructing a sentence pair data set,
step 2, training a text sequencing model,
and step 3, transferring the parameters,
and 4, training an emotion polarity analysis model.
2. The emotion polarity analysis method based on migratory learning of claim 1,
step 1, a sentence pair data set is constructed, specifically, the word position of each sentence in the emotion polarity analysis data set is disturbed according to a set proportion, the sentences before disturbance are retained, and each group of disturbed sentences and sentences with normal language sequence form a piece of training data in a new data set.
3. The emotion polarity analysis method based on migratory learning of claim 1,
step 2, training a text sequencing model, constructing the text sequencing model based on a seq2seq mode, firstly, inputting a disturbed sentence as a model, and extracting sentence characteristics by using an encoder; then decoding word by word, and predicting and outputting words according to the decoding characteristics of the current time step; and finally, comparing the output of the model with the characters at the corresponding positions of the normal word order, and training the parameters of the model based on the cross entropy loss function.
4. The emotion polarity analysis method based on migratory learning of claim 1,
and 3, parameter migration, specifically, firstly constructing an emotion polarity analysis model, using a coding structure the same as that of the text sequencing model, and then migrating the coding parameters and word vector parameters of the text sequencing model into the emotion polarity analysis model.
5. The emotion polarity analysis method based on transfer learning of claim 1, wherein, in step 4, an emotion polarity analysis model is trained, and based on the emotion polarity analysis data set, a comment text is firstly input, a sentence feature is extracted by using an encoder, and then a local feature, an overall feature and a final feature after noise reduction of the sentence are further extracted by using a convolutional neural network, a cyclic neural network and an attention mechanism; and finally classifying the features.
6. The emotion polarity analysis method based on transfer learning of claim 1, wherein in step 2, the text ranking model is trained as follows:
substep 2-1, sentence coding, extracting text coding features by Bert, summarizing the meanings of model related symbols into table 1 for description, and inputtingText sequence X (token) of length m1,token2,…,tokenm) Index id in vocab according to tokeniExtracting token's coded EmbiAnd calculate eiA text matrix consisting of the entire sentence denoted by s, eiThe concrete calculation process of s is shown in the formulas (1) and (2), wherein position represents position coding, segment represents segment coding,
TABLE 1 model-related symbolic meanings
ei＝Bert(Embi+segmenti+postioni) (1)
s＝(e1,e2,…,em) (2)
Then, S is sent to a coding model containing 12 layers of transform sequences to extract a final coded output S, in formula (3), LN is a layer normalization operation, MSA is a multi-head self-attention operation, and taking the z-th layer as an example, the coded output S of the previous layer is processed by MSA firstz-1Then obtained through residual error and LN operation
Finally, FFN pairs are used as shown in formula (3)
Processing and combining residual and LN acquisition layer coded output szAs shown in equation (4), the calculation of FFN is shown in equation (5), where W1、b1、W2、b2Are all parameters that can be learned by the model,
FFN(x)＝max(0,xW1+b1)W2+b2 (5)
substep 2-2, sentence decoding, using GRU as basic unit of decoding network, the decoding process is shown in formulas (6) and (7), in formula (6), dt-1Represents input, ht-1Representing hidden layer output of the previous step, d0Corresponding to CLS (a special character in vocab, which represents the beginning of a sentence) as input, formula (7) represents the hidden layer initialization of the decoding process, i.e. the coded output S is averaged and pooled first, and then linear layer processing is used to represent the initial hidden layer input h0Wherein W iss、bsAre all parameters that can be learned by the model,
ht＝GRU(dt-1,ht-1) (6)
h0＝Wsavg(S)+bs (7)
substep 2-3, outputting the prediction, outputting h with the decoding hidden layer of each steptAs query, using the coded output S as key and value, calculating context vector context by dot-product entry as shown in formula (8), and then splicing context and htAnd (3) obtaining the predicted probability distribution p of the model after the final characteristic of the current decoding step and the splicing characteristic are processed by linear transformation and a softmax function, wherein the process is shown as formula (9), wherein W isp、bpAre all model learnable parameters, and finally, the model log loss is calculated based on the predicted value p and the actual value y as shown in equation (10), where m represents the dictionary vocab size,
context＝Attention(ht,S,S) (8)
p＝softmax(Wp[context,ht]+bp) (9)
7. the emotion polarity analysis method based on transfer learning of claim 1, wherein step 4, the emotion polarity analysis model is trained, specifically as follows,
substep 4-1, sentence coding, because the coding model is completely consistent with the text sequencing model, the coding process of the emotion polarity model is the same as substep 2-1, and S is used for representing coding output;
substep 4-2, extracting local features, and extracting a local feature representation T of the coded output S by using a one-dimensional convolution network, as shown in formula (11); specific calculation result T of each stepiAs shown in equation (12), where W and b are model learnable parameters; x represents the convolution calculation; i represents the ith step; k represents the convolution kernel width (this embodiment uses a convolution kernel of width 2),
T＝Conv(S) (11)
Ti＝tanh(W×Si:i+k-1+b) (12)
substep 4-3, extracting integral characteristics C of sentences by using Bi-GRU as shown in formulas (13) and (14), bidirectional GRU comprises forward GRU, reading T generation from left to right
One reverse GRU generation
nt represents the length of the convolution output of the previous step,
and
respectively representing hidden layer outputs in two directions at the j step of the GRU model
And
obtaining the characteristic h of each step after splicingjAs shown in the formula (15),
substep 4-4, reducing the attention, combining the attention mechanism, further optimizing the text characteristic C, and converting h into hjObtaining v by perceptron (MLP)jAs in equation (16), where WaAnd baFor model learnable parameters, by vjThe similarity with the context C is measured, the importance of the similarity is measured, and the final semantic feature C of the sentence is calculated in a weighted summation mode as shown in formula (17)aAs shown in the formula (18),
vj＝tanh(Wahj+ba) (16)
substep 4-5, emotion polarity classification, first, the sentence characteristic CaAfter linear change and softmax function processing, the classification probability distribution p of the model is obtained, and the process is shown as formula (19), wherein WpAnd bpIs a modelParameters can be learned; the model log-loss is then calculated based on p and the actual label y as shown in equation (20), where n represents the number of classes
p＝softmax(WpCa+bp) (19)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110455888.8A CN113326695B (en) | 2021-04-26 | 2021-04-26 | Emotion polarity analysis method based on transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110455888.8A CN113326695B (en) | 2021-04-26 | 2021-04-26 | Emotion polarity analysis method based on transfer learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113326695A true CN113326695A (en) | 2021-08-31 |
| CN113326695B CN113326695B (en) | 2024-04-26 |
Family
ID=77413792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110455888.8A Active CN113326695B (en) | 2021-04-26 | 2021-04-26 | Emotion polarity analysis method based on transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113326695B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113836919A (en) * | 2021-09-30 | 2021-12-24 | 中国建筑第七工程局有限公司 | Building industry text error correction method based on transfer learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107679580A (en) * | 2017-10-21 | 2018-02-09 | 桂林电子科技大学 | A kind of isomery shift image feeling polarities analysis method based on the potential association of multi-modal depth |
| CN110334187A (en) * | 2019-07-09 | 2019-10-15 | 昆明理工大学 | Burmese sentiment analysis method and device based on transfer learning |
| WO2021051598A1 (en) * | 2019-09-19 | 2021-03-25 | 平安科技（深圳）有限公司 | Text sentiment analysis model training method, apparatus and device, and readable storage medium |
-
2021
- 2021-04-26 CN CN202110455888.8A patent/CN113326695B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107679580A (en) * | 2017-10-21 | 2018-02-09 | 桂林电子科技大学 | A kind of isomery shift image feeling polarities analysis method based on the potential association of multi-modal depth |
| CN110334187A (en) * | 2019-07-09 | 2019-10-15 | 昆明理工大学 | Burmese sentiment analysis method and device based on transfer learning |
| WO2021051598A1 (en) * | 2019-09-19 | 2021-03-25 | 平安科技（深圳）有限公司 | Text sentiment analysis model training method, apparatus and device, and readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 方英兰 等: "基于BERT的文本情感分析方法的研究", 信息技术与信息化, no. 02, 28 February 2020 (2020-02-28) * |
| 曲昭伟 等: "基于迁移学习的分层注意力网络情感分析算法", 计算机应用, no. 11, 19 July 2018 (2018-07-19) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113836919A (en) * | 2021-09-30 | 2021-12-24 | 中国建筑第七工程局有限公司 | Building industry text error correction method based on transfer learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113326695B (en) | 2024-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111639175B (en) | Self-supervision dialogue text abstract method and system | |
| CN110059188B (en) | Chinese emotion analysis method based on bidirectional time convolution network | |
| CN109325112B (en) | A kind of across language sentiment analysis method and apparatus based on emoji | |
| CN111382565B (en) | Emotion-reason pair extraction method and system based on multiple labels | |
| CN111274398B (en) | Method and system for analyzing comment emotion of aspect-level user product | |
| WO2018218705A1 (en) | Method for recognizing network text named entity based on neural network probability disambiguation | |
| CN110929030A (en) | Text abstract and emotion classification combined training method | |
| CN111858932A (en) | Multiple-feature Chinese and English emotion classification method and system based on Transformer | |
| CN110287323B (en) | Target-oriented emotion classification method | |
| CN110427616B (en) | Text emotion analysis method based on deep learning | |
| CN111597327A (en) | Public opinion analysis-oriented unsupervised multi-document abstract generation method | |
| CN110188200A (en) | A kind of depth microblog emotional analysis method using social context feature | |
| CN115759119A (en) | Financial text emotion analysis method, system, medium and equipment | |
| CN116661805A (en) | Code representation generation method and device, storage medium and electronic equipment | |
| CN113326695A (en) | Emotion polarity analysis method based on transfer learning | |
| Yu et al. | Multimodal fusion method with spatiotemporal sequences and relationship learning for valence-arousal estimation | |
| CN113159831A (en) | Comment text sentiment analysis method based on improved capsule network | |
| Chowanda et al. | Generative Indonesian conversation model using recurrent neural network with attention mechanism | |
| CN116522165A (en) | Public opinion text matching system and method based on twin structure | |
| CN115840815A (en) | Automatic abstract generation method based on pointer key information | |
| CN113505207B (en) | Machine reading understanding method and system for financial public opinion research report | |
| CN114896969A (en) | Method for extracting aspect words based on deep learning | |
| CN114972907A (en) | Image semantic understanding and text generation based on reinforcement learning and contrast learning | |
| CN110334189B (en) | Microblog topic label determination method based on long-time and short-time and self-attention neural network | |
| CN114936283B (en) | Network public opinion analysis method based on Bert |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |