CN110472245A - A kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks - Google Patents
A kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks Download PDFInfo
- Publication number
- CN110472245A CN110472245A CN201910751989.2A CN201910751989A CN110472245A CN 110472245 A CN110472245 A CN 110472245A CN 201910751989 A CN201910751989 A CN 201910751989A CN 110472245 A CN110472245 A CN 110472245A
- Authority
- CN
- China
- Prior art keywords
- social media
- vector
- data
- mood
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
The present invention proposes a kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks, comprising: original multiple labeling Social Media short essay is divided into training set and test set；One section in training set original multiple labeling Social Media short text data is pre-processed, training set basic emotion list flag data is obtained；Single label mood disaggregated model of the building based on level convolutional neural networks；Emotional intensity value model is constructed based on attention convolutional neural networks；For multiple labeling Social Media short text test data, predicted using single label mood disaggregated model of level convolutional neural networks, the multiple labeling emotional intensity vector after being optimized.Using the multiple labeling emotional intensity prediction technique of the invention based on stratification convolutional neural networks, the accuracy rate of Social Media text emotional intensity prediction can be further improved, be particularly suitable for existing simultaneously the scene of a variety of basic emotions in text.
Description
Technical field
The invention belongs to text minings and the analysis of public opinion field, and in particular to a kind of based on stratification convolutional neural networks
Multiple labeling emotional intensity prediction technique；
Background technique
With the development of development of Mobile Internet technology in recent years, people share oneself using Social Media with can be convenient
Viewpoint and view, Social Media become an important approach of many human hair table oneself viewpoints and view.It is huge simultaneously
User volume causes to generate a large amount of short text data on Social Media daily, these data become Internet public opinion analysis system
An important data source.Mood analyzes a component part as the analysis of public opinion system, and research has important meaning
Justice.Short text data occupies very high ratio in Social Media simultaneously, so the mood analysis for short text is one
Research direction with practical application value.
Current text mood analysis and research mainly concentrate in text classification problem, i.e., according to " happiness ", " anger ",
Text is divided into classification appropriate by basic emotions such as " detest ".But text can not only express the mood that author is transmitted
Classification, the expressed emotional intensity come out of identical mood also has very big difference in the text.Furthermore the expression of human emotion is
Sufficiently complex, a variety of emotional intensities can be often given expression in a Social Media short text.Example below such as:
" for onlooker unexpectedly for the behavior of young man without being censured, the behavior of this dignity ignored the human rights, ignore people is
It is that one ' uncivil ' can gently cross band！"
The short text is to contain " surprised " mood that do not censured to young behavior for onlooker, also includes
" anger " mood of author to this phenomenon, at the same also contain author for ignore the human rights and ignore people dignity behavior
" detest " mood.If indicating the intensity of mood with 0 to 1 numerical value, the intensity of " surprised ", " anger " and " detest " three
Value can be noted as 0.5,0.8 and 0.7.If using the mood analysis method singly marked, it is likely that text can only be analyzed
In strongest " anger " mood of emotional intensity.And presently, there are some multiple labeling mood sorting algorithms can correctly provide this
Text includes " surprised ", " anger " and " disagreeable " these three moods, but the emotional intensity value of unpredictable various moods,
Can not know which mood is prevailing in sentence by algorithm.Practical multiple labeling emotional intensity prediction algorithm,
Meet the rule that people express mood complexity, while in necks such as the early warning of Social Media network public-opinion, the tracking of emergency event public sentiment
Domain will have huge application value.
Summary of the invention
In response to this problem, the present invention proposes a kind of multiple labeling emotional intensity prediction side based on stratification convolutional neural networks
Method (HCNN).Main target is, from training data focusing study mapping function, it is short to give Social Media by deep learning method
Text predicts the corresponding mood label of the Social Media short essay, wherein the mood is labeled as the vector of n real number composition, each
Real number value [0,1] indicates the intensity of corresponding basic emotion.
A kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks, detailed process are as follows:
Step 1: original multiple labeling Social Media short essay is divided into training set and test set；
Step 2: one section in training set original multiple labeling Social Media short text data being pre-processed, is pre-processed
Single flag data afterwards, wherein original multiple labeling Social Media short text training intensive data is n and represents basic emotion intensity
It is worth real vector [e1, e2…ei, en]；
Step 2.1: to Social Media short text data, removing the punctuate unrelated with sentiment analysis, retain question mark and exclamation
Number, other punctuation marks are removed, obtain removing the Social Media short text data after punctuate；
Step 2.2, if remove punctuate after Social Media short text data in have numerical value, with defined a certain numerical value generation
For the numerical value having had in the Social Media short text data removed after punctuate, obtain replacing the Social Media short text after numerical value
Data；
Step 2.3: it sets largest mood classification and possesses num texts, Social Media short essay after replacing numerical value
When notebook data mood category distribution imbalance, i.e., when other mood classifications possess amount of text less than (3 × num)/4, to this
Classification Social Media short text data carries out resampling, finally obtains the close short text data of each mood classification scale, institute
It states close to short text data is defined as: the other data volume of scale infima species will not be less than 0.75 times largest categorical data amount；
If, without resampling, going to step 2.4 when other mood classifications possess amount of text more than or equal to (3 × num)/4；
Step 2.4: for the Social Media short text data after resampling, for including basic emotion eiText, will
It is put into corresponding basic emotion list flag data DiIn, then the Social Media short text data after a resampling, includes n
Basic emotion intensity value will generate n basic emotion list flag data, if wherein every ei> 0, then it is assumed that there are the mood, ei
=0, then it is assumed that the mood is not present；
Step 3: obtaining the original multiple labeling Social Media short text data of multistage and form training set, using step 2 method pair
Every section of original multiple labeling Social Media short text data is handled, and training set basic emotion list flag data D is obtainedi；
Step 4: single label mood disaggregated model of the building based on level convolutional neural networks (HCNN)；
Step 4.1: by training set basic emotion list flag data DiIt is converted into term vector matrix X, and is used to initialize mind
Embeding layer through network model；
Step 4.2: being directed to term vector matrix X, be used in the convolution window and maximum Chi Huacao of convolutional neural networks (CNN)
Make to extract local feature vw:
vw=CNN (X)
Step 4.3: being encoded for term vector matrix X, using two-way length, memory network (BiLSTM) is encoded in short-term
It is indicated to the enhancing vector for each vocabulary one consideration contextual information, indicates sentence S using the vector of this enhancing
Obtain matrix Xc, in matrix XcOn the basis of carry out the feature of extraction logic layer using convolutional neural networks, obtain vector vc:
Xc=BiLSTM (X)
vc=CNN (Xc)
Step 4.4: fusion local feature and logical layer feature form the new vector v of textf。
Wherein, symbolIndicate vector concatenation or vectorial addition operation.
Step 4.5: by the new vector v of textfIt is input to full articulamentum, obtains single label feelings of level convolutional neural networks
Thread disaggregated model；
Step 4.6: by the new vector v of textfIt is input to full articulamentum, using softmax function, obtains level convolution mind
The output of single label mood disaggregated model through network:
Single label mood disaggregated model of the level convolutional neural networks uses cross entropy loss function, as follows:
In above formula, N indicates the number of training examples, yiIt is a binary variable for indicating whether i-th of sample belongs to
In some classification,Indicate that i-th of sample of model prediction belongs to the probability of specified classification.
It is optimized, is used using single label mood disaggregated model of the cross entropy loss function to level convolutional neural networks
Gradient descent algorithm is iterated optimization, when training dataset Global Iterative Schemes recycle L times, terminates optimization process, is lost
Single label mood disaggregated model of level convolutional neural networks after function optimization, as final level convolutional neural networks
Single label mood disaggregated model；
Step 5: emotional intensity value model is constructed based on attention convolutional neural networks (ACNN)；
Step 5.1: for single flag data collection Di, filter out ei=0 text, next for ei> 0 text utilizes
Following steps train emotional intensity value prediction model；
Step 5.2: by single flag data collection DiIn each text S be converted into term vector matrix X ', for initializing nerve
The embeding layer of network model；
Step 5.3: using long memory models in short-term to term vector matrix XPIt is encoded, obtains a task of text S
Correlation indicates vector vs, in which:
vs=LSTM (XP)
Step 5.4: utilizing sentence expression vector vsWith prime word vector matrix XP, term vector is calculated by attention mechanism
Associated weight, wherein attention vector calculation are as follows:
va=XPWvs
Wherein, vaTo pay attention to force vector, W is weight；
By paying attention to force vector, term vector is weighted, i.e., the term vector in subsequent window is zoomed in and out, formula
Are as follows:
αi=l*softmax (va[i:i+l]), i ∈ { 0,1 ..., n-l }
Wherein l represents the size of window at this time,Represent the corresponding term vector of the i-th+0 vocabulary in sentence.Cause
This, the similarity scores in current window are converted into a probability distribution using softmax function, and do multiplication with l and contracted
Put the weight of term vector, then with original term vector XPIt is multiplied, it is zoomed in and out.For each window, it is new to generate text
Weighting characterization Z.
Step 5.5: the feature of weighting characterization Z is extracted using convolutional neural networks.It is the weighting that l is generated for window size
Characterize vector Zl, most significant feature is extracted using CNN network and maximum pond method:
vl=CNN (Zl)
By the v of different windows sizelFeature is spliced, and the final characterization vector v of input text is formedg。
Step 5.6: by vgIt inputs in full articulamentum, obtains the final output of model using softmax function, is i.e. text
Emotional intensity value.
Step 5.7: model being optimized using training data and loss function, the feelings after obtaining optimized parameter and optimization
Thread strength model:
Use the mean square error of the emotional intensity value of practical emotional intensity value and model prediction as emotional intensity model
Loss function:
Wherein, the number of the specific single label emotional training data set sample of N ' expression, piIndicate what i-th of sample was marked
Emotional intensity value,Indicate the emotional intensity value that i-th of sample is obtained by model h ' prediction.
Optimization is iterated using stochastic gradient descent algorithm, when training dataset Global Iterative Schemes circulation L ' is secondary, is terminated
Optimization process obtains single label emotional intensity prediction model.
Step 6: being directed to multiple labeling Social Media short text test data, use single label feelings of level convolutional neural networks
Thread disaggregated model predicted, the multiple labeling emotional intensity vector after being optimized；
Step 6.1: one section in test set original multiple labeling Social Media short text test intensive data is pre-processed,
Obtain pretreated single flag data；Wherein, multiple labeling Social Media short text test intensive data is that a representative of n ' is basic
Emotional intensity value real vector [e '1, e '2…e’i, e 'n’]；
Step 6.1.1: to Social Media short text data, the punctuate unrelated with sentiment analysis is removed, retains question mark and sense
Exclamation removes other punctuation marks, obtains removing the Social Media short text data after punctuate；
Step 6.1.2, if removing in the Social Media short text data after punctuate has numerical value, with defined a certain numerical value
Instead of the numerical value having had in the Social Media short text data after removal punctuate, obtain replacing the Social Media short essay after numerical value
Notebook data；
Step 6.1.3, if removing in the Social Media short text data after punctuate has numerical value, with defined a certain numerical value
Instead of the numerical value having had in the Social Media short text data after removal punctuate, obtain replacing the Social Media short essay after numerical value
Notebook data；
Step 6.1.3: for the Social Media short text data after replacement numerical value, for including basic emotion e 'iText
This, puts it into corresponding basic emotion list flag data D 'iIn, then one replace numerical value after Social Media short text number
According to, include a basic emotion intensity value of n ', n ' basic emotion list flag data will be generated, if wherein every e 'i> 0, then it is assumed that
There are the mood, e 'i=0, then it is assumed that the mood is not present；
Step 6.2: by pretreated single flag data test data D 'iIt is converted into term vector matrix X ', and is used to just
The embeding layer of beginningization neural network model；
Step 6.3: being directed to term vector matrix X ', the convolution window for being used in convolutional neural networks (CNN) extracts part spy
Levy v 'w；
Step 6.4: being encoded for term vector matrix X ', using two-way length, memory network encodes to obtain for every in short-term
The enhancing vector of a vocabulary one consideration contextual information indicates, indicates that sentence S obtains matrix using the vector of this enhancing
X’c, in matrix X 'cOn the basis of the feature for carrying out extraction logic layer is operated using convolution window and maximum pondization, obtain vector v 'c；
Step 6.5: fusion local feature and logical layer feature, formed the new vector v of text 'f, obtain network convolution and
The output vector of pond layer；
Step 6.6: by the new vector v of text 'fIt is input to full articulamentum, using softmax function, obtains level convolution
The output of single label mood disaggregated model of neural network:
Step 6.7: using emotional intensity model ACNN, calculate the emotional intensity value of single label mood disaggregated model output；
Step 6.8: being exported based on ACNN emotional intensity model based on every kind of moodIt combines, after obtaining optimization
Multiple labeling emotional intensity value vector
Advantageous effects:
It, can be further using the multiple labeling emotional intensity prediction technique of the invention based on stratification convolutional neural networks
The accuracy rate of Social Media text emotional intensity prediction is improved, is particularly suitable for existing simultaneously the field of a variety of basic emotions in text
Scape.
Detailed description of the invention
The multiple labeling emotional intensity prediction technique entirety frame based on stratification convolutional neural networks of Fig. 1 embodiment of the present invention
Frame；
The HCNN model structure of Fig. 2 embodiment of the present invention；
Fig. 3 and tradition CNN model experiment comparing result 1
Fig. 4 and tradition CNN model experiment comparing result 2
Specific embodiment
Be described further with reference to the accompanying drawing with specific implementation example to invention: one kind is based on stratification convolutional Neural net
The multiple labeling emotional intensity prediction technique of network, detailed process are as follows:
Step 1: original multiple labeling Social Media short essay is divided into training set and test set；
Step 2: one section in training set original multiple labeling Social Media short text data being pre-processed, is pre-processed
Single flag data afterwards, wherein original multiple labeling Social Media short text training intensive data is n and represents basic emotion intensity
It is worth real vector [e1, e2…ei, en]；
Step 2.1: to Social Media short text data, removing the punctuate unrelated with sentiment analysis, retain question mark and exclamation
Number, other punctuation marks are removed, obtain removing the Social Media short text data after punctuate；
Step 2.2, if remove punctuate after Social Media short text data in have numerical value, with defined a certain numerical value generation
For the numerical value having had in the Social Media short text data removed after punctuate, obtain replacing the Social Media short text after numerical value
Data；
Step 2.3: it sets largest mood classification and possesses num texts, Social Media short essay after replacing numerical value
When notebook data mood category distribution imbalance, i.e., when other mood classifications possess amount of text less than (3 × num)/4, to this
Classification Social Media short text data carries out resampling, finally obtains the close short text data of each mood classification scale, institute
It states close to short text data is defined as: the other data volume of scale infima species will not be less than 0.75 times largest categorical data amount；
If, without resampling, going to step 2.4 when other mood classifications possess amount of text more than or equal to (3 × num)/4；
Step 2.4: for the Social Media short text data after resampling, for including basic emotion eiText, will
It is put into corresponding basic emotion list flag data DiIn, then the Social Media short text data after a resampling, includes n
Basic emotion intensity value will generate n basic emotion list flag data, if wherein every ei> 0, then it is assumed that there are the mood, ei
=0, then it is assumed that the mood is not present；
Step 3: obtaining the original multiple labeling Social Media short text data of multistage and form training set, using step 2 method pair
Every section of original multiple labeling Social Media short text data is handled, and training set basic emotion list flag data D is obtainedi；
The general frame of inventive algorithm is as shown in Fig. 1, mainly includes two parts of model training and prediction, main to calculate
Method 1 and 2 is described as follows:
Step 4: single label mood disaggregated model of the building based on level convolutional neural networks (HCNN), as shown in Figure 2；
Step 4.1: by training set basic emotion list flag data DiIt is converted into term vector matrix X, and is used to initialize mind
Embeding layer through network model；The present invention trains Chinese term vector using Chinese wikipedia, and is used to initialize nerve net
The embeding layer of network model.Can by the Skip-gram model of word2vec tool, when contextual window is set as 5,
It is trained using the optimization method of negative sampling.
Step 4.2: being directed to term vector matrix X, be used in the convolution window and maximum Chi Huacao of convolutional neural networks (CNN)
Make to extract local feature vw:
vw=CNN (X)
Step 4.3: being encoded for term vector matrix X, using two-way length, memory network (BiLSTM) is encoded in short-term
It is indicated to the enhancing vector for each vocabulary one consideration contextual information, indicates sentence S using the vector of this enhancing
Obtain matrix Xc, in matrix XcOn the basis of carry out the feature of extraction logic layer using convolutional neural networks, obtain vector vc:
Xc=BiLSTM (X)
vc=CNN (Xc)
Step 4.4: fusion local feature and logical layer feature form the new vector v of textf。
Wherein, symbolIndicate vector concatenation or vectorial addition operation.
Step 4.5: by the new vector v of textfIt is input to full articulamentum, obtains single label feelings of level convolutional neural networks
Thread disaggregated model；
Step 4.6: by the new vector v of textfIt is input to full articulamentum, using softmax function, obtains level convolution mind
The output of single label mood disaggregated model through network:
Single label mood disaggregated model of the level convolutional neural networks uses cross entropy loss function, as follows:
In above formula, N indicates the number of training examples, yiIt is a binary variable for indicating whether i-th of sample belongs to
In some classification,Indicate that i-th of sample of model prediction belongs to the probability of specified classification.
It is optimized, is used using single label mood disaggregated model of the cross entropy loss function to level convolutional neural networks
Gradient descent algorithm is iterated optimization, when training dataset Global Iterative Schemes recycle L times, terminates optimization process, is lost
Single label mood disaggregated model of level convolutional neural networks after function optimization, as final level convolutional neural networks
Single label mood disaggregated model；
Binary classifier model { the C of every kind of basic emotion can be trained by algorithm 1iAnd emotional intensity prediction model
{Hi, on this basis, the present invention can predict the intensity value of every kind of basic emotion in given short text, more when expressing in text
Kind basic emotion, can be completed the prediction of multiple labeling emotional intensity, is specifically shown in algorithm 2.
Using trained classification and prediction model, can effectively be predicted by algorithm 2 simultaneous more in text
The intensity value of kind mood, the experimental results showed that, method proposed by the present invention can be further improved text emotional intensity prediction effect
Fruit sees attached drawing 3 and attached drawing 4.
Step 5: emotional intensity value model is constructed based on attention convolutional neural networks (ACNN)；
Step 5.1: for single flag data collection Di, filter out ei=0 text, next for ei> 0 text utilizes
Following steps train emotional intensity value prediction model；
Step 5.2: by single flag data collection DiIn each text S be converted into term vector matrix X ', for initializing nerve
The embeding layer of network model；
Step 5.3: using long memory models in short-term to term vector matrix XPIt is encoded, obtains a task of text S
Correlation indicates vector vs, in which:
vs=LSTM (XP)
Step 5.4: utilizing sentence expression vector vsWith prime word vector matrix XP, term vector is calculated by attention mechanism
Associated weight, wherein attention vector calculation are as follows:
va=XPWvs
By paying attention to force vector, term vector is weighted, i.e., the term vector in subsequent window is zoomed in and out, formula
Are as follows:
αi=l*softmax (va[i:i+l]), i ∈ { 0,1 ..., n-l }
Wherein l represents the size of window at this time,Represent the corresponding term vector of the i-th+0 vocabulary in sentence.Cause
This, the similarity scores in current window are converted into a probability distribution using softmax function, and do multiplication with l and contracted
Put the weight of term vector, then with original term vector XPIt is multiplied, it is zoomed in and out.For each window, it is new to generate text
Weighting characterization Z.
Step 5.5: the feature of weighting characterization Z is extracted using convolutional neural networks.It is the weighting that l is generated for window size
Characterize vector Zl, most significant feature is extracted using CNN network and maximum pond method:
vl=CNN (Zl)
By the v of different windows sizelFeature is spliced, and the final characterization vector v of input text is formedg。
Step 5.6: by vgIt inputs in full articulamentum, obtains the final output of model using softmax function, is i.e. text
Emotional intensity value.
Step 5.7: model being optimized using training data and loss function, the feelings after obtaining optimized parameter and optimization
Thread strength model:
Use the mean square error of the emotional intensity value of practical emotional intensity value and model prediction as emotional intensity model
Loss function:
Wherein, the number of the specific single label emotional training data set sample of N ' expression, piIndicate what i-th of sample was marked
Emotional intensity value,Indicate the emotional intensity value that i-th of sample is obtained by model h ' prediction.
Optimization is iterated using stochastic gradient descent algorithm, when training dataset Global Iterative Schemes circulation L ' is secondary, is terminated
Optimization process obtains single label emotional intensity prediction model.
Step 6: being directed to multiple labeling Social Media short text test data, use single label feelings of level convolutional neural networks
Thread disaggregated model predicted, the multiple labeling emotional intensity vector after being optimized；
Step 6.1: one section in test set original multiple labeling Social Media short text test intensive data is pre-processed,
Obtain pretreated single flag data；Wherein, multiple labeling Social Media short text test intensive data is that a representative of n ' is basic
Emotional intensity value real vector [e '1, e '2…e’i, e 'n’]；
Step 6.1.1: to Social Media short text data, the punctuate unrelated with sentiment analysis is removed, retains question mark and sense
Exclamation removes other punctuation marks, obtains removing the Social Media short text data after punctuate；
Step 6.1.2, if removing in the Social Media short text data after punctuate has numerical value, with defined a certain numerical value
Instead of the numerical value having had in the Social Media short text data after removal punctuate, obtain replacing the Social Media short essay after numerical value
Notebook data；
Step 6.1.3, if removing in the Social Media short text data after punctuate has numerical value, with defined a certain numerical value
Instead of the numerical value having had in the Social Media short text data after removal punctuate, obtain replacing the Social Media short essay after numerical value
Notebook data；
Step 6.1.3: for the Social Media short text data after replacement numerical value, for including basic emotion e 'iText
This, puts it into corresponding basic emotion list flag data D 'iIn, then one replace numerical value after Social Media short text number
According to, include a basic emotion intensity value of n ', n ' basic emotion list flag data will be generated, if wherein every e 'i> 0, then it is assumed that
There are the mood, e 'i=0, then it is assumed that the mood is not present；
Step 6.2: by pretreated single flag data test data D 'iIt is converted into term vector matrix X ', and is used to just
The embeding layer of beginningization neural network model；
Step 6.3: being directed to term vector matrix X ', the convolution window for being used in convolutional neural networks (CNN) extracts part spy
Levy v 'w；
Step 6.4: being encoded for term vector matrix X ', using two-way length, memory network encodes to obtain for every in short-term
The enhancing vector of a vocabulary one consideration contextual information indicates, indicates that sentence S obtains matrix using the vector of this enhancing
X’c, in matrix X 'cOn the basis of the feature for carrying out extraction logic layer is operated using convolution window and maximum pondization, obtain vector v 'c；
Step 6.5: fusion local feature and logical layer feature, formed the new vector v of text 'f, obtain network convolution and
The output vector of pond layer；
Step 6.6: by the new vector v of text 'fIt is input to full articulamentum, using softmax function, obtains level convolution
The output of single label mood disaggregated model of neural network:
Step 6.7: using emotional intensity model ACNN, calculate the emotional intensity value of single label mood disaggregated model output；
Step 6.8: being exported based on ACNN emotional intensity model based on every kind of moodIt combines, after obtaining optimization
Multiple labeling emotional intensity value vector
Core of the invention innovation is to propose a kind of level convolutional neural networks model (HCNN), can be used for Social Media
Mood classification and the emotional intensity of text are predicted.The present invention provides a kind of specific embodiment of HCNN model.
(1) trained and test data.Using Chinese blog data collection, totally 19751 sentences, basic emotion are angry, burnt
Consider, expect, detesting, is happy, liking, is grieved, is this eight kinds surprised.Each sentence is carried out according to the mood of expression in data set
Mark, the intensity value range of every kind of mood is in [0,1], and wherein intensity 0 indicates that the sentence does not express this kind of basic emotion.
(2) term vector pre-training.Chinese term vector is trained using Chinese wikipedia, original language material is directly from Wiki
Dump downloading.Term vector training tool selects word2vec, the Skip-gram model that specific choice vector dimension is 200, upper
It when lower text window is set as 5, is trained using the optimization method of Negative Sampling, sampled value size is 1e-
4, the number of iterations of model is 15.
HCNN network training method.HCNN training uses Adam optimization method.The number of convolution kernel is arranged in network
200.The last full articulamentum of model includes two hidden layers, the number of hidden unit is 200 and 100 respectively, this is two layers
Dropout rate value is respectively set to 0.2 and 0.1.For different basic emotions, can be set different convolution window sizes with
And the hidden unit number of two-way length memory network in short-term, the work adjust ginseng to complete by carrying out on verifying collection.
Fig. 3 be and traditional CNN model experiment comparing result 1, wherein DCNN and ACNN be respectively adopted vector splicing and to
The mode of amount addition merges the HCNN model of multilayer feature；RL: sequence loss；HL: Hamming loss；MSE: Averaged Square Error of Multivariate；
SA: subset accuracy rate；Arrow direction indicates that index is the bigger the better upwards, indicates that index is the smaller the better downwards；Fig. 4 be and tradition
CNN model experiment comparing result 2, DCNN and ACNN are vector splicing to be respectively adopted and the mode of vectorial addition merges multilayer feature
HCNN model；OE: single mistake；MaF: macro average F value；MiF: micro- average F value；AP: average accuracy；Arrow direction is upward
It indicates that index is the bigger the better, indicates that index is the smaller the better downwards.
Claims (3)
1. a kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks, which is characterized in that one kind is based on
The multiple labeling emotional intensity prediction technique of stratification convolutional neural networks, detailed process are as follows:
Step 1: original multiple labeling Social Media short essay is divided into training set and test set；
Step 2: one section in training set original multiple labeling Social Media short text data being pre-processed, is obtained pretreated
Single flag data, wherein original multiple labeling Social Media short text training intensive data is n and represents basic emotion intensity value reality
Number vector [e1, e2…ei, en]；
Step 3: obtaining the original multiple labeling Social Media short text data of multistage and form training set, using step 2 method to every section
Original multiple labeling Social Media short text data is handled, and training set basic emotion list flag data D is obtainedi；
Step 4: single label mood disaggregated model of the building based on level convolutional neural networks；
Step 4.1: by training set basic emotion list flag data DiIt is converted into term vector matrix X, and is used to initialize neural network
The embeding layer of model；
Step 4.2: being directed to term vector matrix X, the convolution window and maximum pondization for being used in convolutional neural networks operate extraction office
Portion feature vw:
vw=CNN (X)
Step 4.3: being encoded for term vector matrix X, using two-way length, memory network encodes to obtain for each word in short-term
The enhancing vector of a consideration contextual information of converging indicates, indicates that sentence S obtains matrix X using the vector of this enhancingc, In
Matrix XcOn the basis of carry out the feature of extraction logic layer using convolutional neural networks, obtain vector vc:
Xc=BiLSTM (X)
vc=CNN (Xc)
Step 4.4: fusion local feature and logical layer feature form the new vector v of textf；
Wherein, symbolIndicate vector concatenation or vectorial addition operation；
Step 4.5: by the new vector v of textfIt is input to full articulamentum, obtains single label mood point of level convolutional neural networks
Class model；
Step 4.6: by the new vector v of textfIt is input to full articulamentum, using softmax function, obtains level convolutional Neural net
The output of single label mood disaggregated model of network:
Single label mood disaggregated model of the level convolutional neural networks uses cross entropy loss function, as follows:
In above formula, N indicates the number of training examples, yiIt is a binary variable for indicating whether i-th of sample belongs to some
Classification,Indicate that i-th of sample of model prediction belongs to the probability of specified classification；
It is optimized using single label mood disaggregated model of the cross entropy loss function to level convolutional neural networks, using gradient
Descent algorithm is iterated optimization, when training dataset Global Iterative Schemes recycle L times, terminates optimization process, obtains loss function
Single label mood disaggregated model of level convolutional neural networks after optimization, single mark of as final level convolutional neural networks
Remember mood disaggregated model；
Step 5: constructing emotional intensity value model based on attention convolutional neural networks；
Step 5.1: for single flag data collection Di, filter out ei=0 text, next for ei> 0 text, utilization are following
Step trains emotional intensity value prediction model；
Step 5.2: by single flag data collection DiIn each text S be converted into term vector matrix X ', for initializing neural network mould
The embeding layer of type；
Step 5.3: using long memory models in short-term to term vector matrix XPIt is encoded, obtains a task correlation table of text S
Show vector vs, in which:
vs=LSTM (XP)
Step 5.4: utilizing sentence expression vector vsWith prime word vector matrix XP, it is related that term vector is calculated by attention mechanism
Weight, wherein attention vector calculation are as follows:
va=XPWvs
Wherein, vaTo pay attention to force vector, W is weight；
By paying attention to force vector, term vector is weighted, i.e., the term vector in subsequent window is zoomed in and out, formula are as follows:
αi=l*softmax (va[i:i+l]), i ∈ { 0,1 ..., n-l }
Wherein l represents the size of window at this time,Represent the corresponding term vector of the i-th+0 vocabulary in sentence；Therefore, when
Similarity scores in front window are converted into a probability distribution using softmax function, and with l do multiplication obtain scaling word to
The weight of amount, then with original term vector XPIt is multiplied, it is zoomed in and out；For each window, the new weighting table of text is generated
Levy Z；
Step 5.5: extracting the feature of weighting characterization Z using convolutional neural networks, be that the weighting that l is generated characterizes for window size
Vector Zl, most significant feature is extracted using CNN network and maximum pond method:
vl=CNN (Zl)
By the v of different windows sizelFeature is spliced, and the final characterization vector v of input text is formedg；
Step 5.6: by vgIt inputs in full articulamentum, obtains the final output of model, the i.e. mood of text using softmax function
Intensity value；
Step 5.7: model being optimized using training data and loss function, the mood after obtaining optimized parameter and optimization is strong
Spend model:
Use the mean square error of the emotional intensity value of practical emotional intensity value and model prediction as the loss of emotional intensity model
Function:
Wherein, the number of the specific single label emotional training data set sample of N ' expression, piIndicate the mood that i-th of sample is marked
Intensity value,Indicate the emotional intensity value that i-th of sample is obtained by model h ' prediction；
Optimization is iterated using stochastic gradient descent algorithm, when training dataset Global Iterative Schemes circulation L ' is secondary, terminates optimization
Process obtains single label emotional intensity prediction model；
Step 6: being directed to multiple labeling Social Media short text test data, use single label mood point of level convolutional neural networks
Class model predicted, the multiple labeling emotional intensity vector after being optimized.
2. the multiple labeling emotional intensity prediction technique according to claim 1 based on stratification convolutional neural networks, special
Sign is that the step 2 specifically includes:
Step 2.1: to Social Media short text data, removing the punctuate unrelated with sentiment analysis, retain question mark and exclamation mark, move
Except other punctuation marks, obtain removing the Social Media short text data after punctuate；
Step 2.2, if remove punctuate after Social Media short text data in have numerical value, with defined a certain numerical value replace shifting
Except the numerical value having had in the Social Media short text data after punctuate, obtain replacing the Social Media short text number after numerical value
According to；
Step 2.3: it sets largest mood classification and possesses num texts, Social Media short text number after replacing numerical value
When according to mood category distribution imbalance, i.e., when other mood classifications possess amount of text less than (3 × num)/4, to the category
Social Media short text data carries out resampling, finally obtains the close short text data of each mood classification scale, described to connect
Nearly short text data is defined as: the other data volume of scale infima species will not be less than 0.75 times largest categorical data amount；If working as
When other mood classifications possess amount of text more than or equal to (3 × num)/4, without resampling, step 2.4 is gone to；
Step 2.4: for the Social Media short text data after resampling, for including basic emotion eiText, put it into
To corresponding basic emotion list flag data DiIn, then the Social Media short text data after a resampling, includes n basic feelings
Thread intensity value will generate n basic emotion list flag data, if wherein every ei> 0, then it is assumed that there are the mood, ei=0, then
Think that there is no the moods.
3. the multiple labeling emotional intensity prediction technique according to claim 1 based on stratification convolutional neural networks, special
Sign is that the step 6 specifically includes:
Step 6.1: one section in test set original multiple labeling Social Media short text test intensive data being pre-processed, is obtained
Pretreated list flag data；Wherein, multiple labeling Social Media short text test intensive data represents basic emotion for n ' is a
Intensity value real vector [e '1, e '2…e’i, e 'n’]；
Step 6.1.1: to Social Media short text data, removing the punctuate unrelated with sentiment analysis, retain question mark and exclamation mark,
Other punctuation marks are removed, obtain removing the Social Media short text data after punctuate；
Step 6.1.2 is replaced if removing in the Social Media short text data after punctuate has numerical value with defined a certain numerical value
The numerical value having had in Social Media short text data after removing punctuate obtains replacing the Social Media short text number after numerical value
According to；
Step 6.1.3 is replaced if removing in the Social Media short text data after punctuate has numerical value with defined a certain numerical value
The numerical value having had in Social Media short text data after removing punctuate obtains replacing the Social Media short text number after numerical value
According to；
Step 6.1.3: for the Social Media short text data after replacement numerical value, for including basic emotion e 'iText, will
It is put into corresponding basic emotion list flag data D 'iIn, then one replace numerical value after Social Media short text data, include
A basic emotion intensity value of n ' will generate n ' basic emotion list flag data, if wherein every e 'i> 0, then it is assumed that there is this
Mood, e 'i=0, then it is assumed that the mood is not present；
Step 6.2: by pretreated single flag data test data D 'iIt is converted into term vector matrix X ', and is used to initialize mind
Embeding layer through network model；
Step 6.3: being directed to term vector matrix X ', the convolution window for being used in convolutional neural networks extracts local feature v 'w；
Step 6.4: being encoded for term vector matrix X ', using two-way length, memory network encodes to obtain for each word in short-term
The enhancing vector of a consideration contextual information of converging indicates, indicates that sentence S obtains matrix X ' using the vector of this enhancingc,
In matrix X 'cOn the basis of the feature for carrying out extraction logic layer is operated using convolution window and maximum pondization, obtain vector v 'c；
Step 6.5: fusion local feature and logical layer feature, formed the new vector v of text 'f, obtain convolution and the pond of network
The output vector of layer；
Step 6.6: by the new vector v of text 'fIt is input to full articulamentum, using softmax function, obtains level convolutional Neural net
The output of single label mood disaggregated model of network:
Step 6.7: using emotional intensity model ACNN, calculate the emotional intensity value of single label mood disaggregated model output；
Step 6.8: being exported based on ACNN emotional intensity model based on every kind of moodIt combines, it is more after being optimized
Mark emotional intensity value vector
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910751989.2A CN110472245B (en) | 2019-08-15 | 2019-08-15 | Multi-label emotion intensity prediction method based on hierarchical convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910751989.2A CN110472245B (en) | 2019-08-15 | 2019-08-15 | Multi-label emotion intensity prediction method based on hierarchical convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110472245A true CN110472245A (en) | 2019-11-19 |
| CN110472245B CN110472245B (en) | 2022-11-29 |
Family
ID=68511433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910751989.2A Active CN110472245B (en) | 2019-08-15 | 2019-08-15 | Multi-label emotion intensity prediction method based on hierarchical convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110472245B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110957039A (en) * | 2019-12-16 | 2020-04-03 | 河南科技学院 | Campus psychological coaching method and device based on deep learning |
| CN111862068A (en) * | 2020-07-28 | 2020-10-30 | 福州大学 | Three-model comprehensive decision emotion prediction method fusing data missing data and images |
| CN111985532A (en) * | 2020-07-10 | 2020-11-24 | 西安理工大学 | Scene-level context-aware emotion recognition deep network method |
| CN116306686A (en) * | 2023-05-22 | 2023-06-23 | 中国科学技术大学 | Method for generating multi-emotion-guided co-emotion dialogue |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170308790A1 (en) * | 2016-04-21 | 2017-10-26 | International Business Machines Corporation | Text classification by ranking with convolutional neural networks |
| CN109299253A (en) * | 2018-09-03 | 2019-02-01 | 华南理工大学 | A kind of social text Emotion identification model construction method of Chinese based on depth integration neural network |
| CN110097894A (en) * | 2019-05-21 | 2019-08-06 | 焦点科技股份有限公司 | A kind of method and system of speech emotion recognition end to end |
-
2019
- 2019-08-15 CN CN201910751989.2A patent/CN110472245B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170308790A1 (en) * | 2016-04-21 | 2017-10-26 | International Business Machines Corporation | Text classification by ranking with convolutional neural networks |
| CN109299253A (en) * | 2018-09-03 | 2019-02-01 | 华南理工大学 | A kind of social text Emotion identification model construction method of Chinese based on depth integration neural network |
| CN110097894A (en) * | 2019-05-21 | 2019-08-06 | 焦点科技股份有限公司 | A kind of method and system of speech emotion recognition end to end |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110957039A (en) * | 2019-12-16 | 2020-04-03 | 河南科技学院 | Campus psychological coaching method and device based on deep learning |
| CN111985532A (en) * | 2020-07-10 | 2020-11-24 | 西安理工大学 | Scene-level context-aware emotion recognition deep network method |
| CN111985532B (en) * | 2020-07-10 | 2021-11-09 | 西安理工大学 | Scene-level context-aware emotion recognition deep network method |
| CN111862068A (en) * | 2020-07-28 | 2020-10-30 | 福州大学 | Three-model comprehensive decision emotion prediction method fusing data missing data and images |
| CN111862068B (en) * | 2020-07-28 | 2022-09-13 | 福州大学 | Three-model comprehensive decision emotion prediction method fusing data missing data and images |
| CN116306686A (en) * | 2023-05-22 | 2023-06-23 | 中国科学技术大学 | Method for generating multi-emotion-guided co-emotion dialogue |
| CN116306686B (en) * | 2023-05-22 | 2023-08-29 | 中国科学技术大学 | Method for generating multi-emotion-guided co-emotion dialogue |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110472245B (en) | 2022-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110287320B (en) | Deep learning multi-classification emotion analysis model combining attention mechanism | |
| CN110245229B (en) | Deep learning theme emotion classification method based on data enhancement | |
| CN108009148B (en) | Text emotion classification representation method based on deep learning | |
| Alwehaibi et al. | Comparison of pre-trained word vectors for arabic text classification using deep learning approach | |
| CN109697232A (en) | A kind of Chinese text sentiment analysis method based on deep learning | |
| CN109558487A (en) | Document Classification Method based on the more attention networks of hierarchy | |
| CN110472245A (en) | A kind of multiple labeling emotional intensity prediction technique based on stratification convolutional neural networks | |
| CN113254599A (en) | Multi-label microblog text classification method based on semi-supervised learning | |
| CN109885670A (en) | A kind of interaction attention coding sentiment analysis method towards topic text | |
| CN109977416A (en) | A kind of multi-level natural language anti-spam text method and system | |
| CN107590134A (en) | Text sentiment classification method, storage medium and computer | |
| CN107729309A (en) | A kind of method and device of the Chinese semantic analysis based on deep learning | |
| CN111027595B (en) | Double-stage semantic word vector generation method | |
| CN110096587B (en) | Attention mechanism-based LSTM-CNN word embedded fine-grained emotion classification model | |
| CN107688576B (en) | Construction and tendency classification method of CNN-SVM model | |
| Shanmugavadivel et al. | An analysis of machine learning models for sentiment analysis of Tamil code-mixed data | |
| CN110781290A (en) | Extraction method of structured text abstract of long chapter | |
| Mo et al. | Large language model (llm) ai text generation detection based on transformer deep learning algorithm | |
| CN111814450B (en) | Aspect-level emotion analysis method based on residual attention | |
| CN112287106A (en) | Online comment emotion classification method based on dual-channel hybrid neural network | |
| CN113094502A (en) | Multi-granularity takeaway user comment sentiment analysis method | |
| CN116522945A (en) | Model and method for identifying named entities in food safety field | |
| CN116737922A (en) | Tourist online comment fine granularity emotion analysis method and system | |
| CN115422939A (en) | Fine-grained commodity named entity identification method based on big data | |
| Huang | A CNN model for SMS spam detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |