CN113157919A - Sentence text aspect level emotion classification method and system - Google Patents

Sentence text aspect level emotion classification method and system Download PDF

Info

Publication number
CN113157919A
CN113157919A CN202110372212.2A CN202110372212A CN113157919A CN 113157919 A CN113157919 A CN 113157919A CN 202110372212 A CN202110372212 A CN 202110372212A CN 113157919 A CN113157919 A CN 113157919A
Authority
CN
China
Prior art keywords
representation
context
word
final
sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110372212.2A
Other languages
Chinese (zh)
Other versions
CN113157919B (en
Inventor
鲁燃
李筱雯
刘培玉
朱振方
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Normal University
Original Assignee
Shandong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Normal University filed Critical Shandong Normal University
Priority to CN202110372212.2A priority Critical patent/CN113157919B/en
Publication of CN113157919A publication Critical patent/CN113157919A/en
Application granted granted Critical
Publication of CN113157919B publication Critical patent/CN113157919B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G06F16/355—Class or cluster creation or modification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Databases & Information Systems (AREA)
  • Machine Translation (AREA)

Abstract

The invention provides a sentence text aspect level emotion classification method and system, belonging to the technical field of text emotion classification, and comprising the following steps: performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism; and according to the structural aspect representation and the structural context representation, the syntactic dependency information of the dependency relationship tree is utilized, the information of the average pooling aggregation aspect vector is combined, the final embedding is extracted, the probability distribution of different emotion polarities is calculated by combining a back propagation algorithm, and the final emotion polarity of the sentence is predicted. The invention solves the problem of long-distance word dependence among a plurality of words, and considers the context dependence relationship; the sentences are coded into a multi-dimensional matrix by using a structured self-attention mechanism, each vector can be regarded as a context related to the aspect words, the context representation of the aspect is generated, and the relation between a plurality of semantic segments and the aspect words is revealed.

Description

Sentence text aspect level emotion classification method and system
Technical Field
The invention relates to the technical field of text emotion classification, in particular to a deep learning sentence text aspect emotion classification method and system based on an attention network.
Background
Aspect-level sentiment classification is a popular task for confirming sentiment polarity, and aims to identify the sentiment polarity of a given aspect word in a sentence. For text comments of a certain event, judging the emotion polarity of a text sentence mainly comprises two aspects, namely positive and negative. The noun phrases appearing in the input sentence are used as the aspect words for confirming the emotion polarity, and since some sentences may have several aspects simultaneously, and the aspects may represent different emotion polarities, it is important to perform emotion classification.
The traditional approach is to construct a feature engineering for the model and select a series of good features. Conventional methods such as emotion dictionaries and machine learning are commonly used. Because the deep learning method has obvious advantages in automatically learning text features, the dependence on manually designed features can be avoided, and the features can be mapped into continuous low-dimensional vectors. Therefore, it is widely applied in aspect-level emotion classification.
Classification models based on neural networks, such as Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), have been widely applied to aspect-level emotion classification. The attention-based RNN model enhances the semantic connections between context vocabulary and facet words and is widely used in recent approaches for searching for potentially related words related to a given facet word. CNN-based attention methods have also been proposed to enhance phrase-level representation and achieve good results. Although attention-based models have performed well in multiple tasks, their limitations remain evident because the attention module may highlight irrelevant words due to syntactical loss, which may lead to misprediction of emotional polarity.
Disclosure of Invention
The invention aims to provide a sentence text aspect level emotion classification method and system for deep learning based on an attention network, which are used for accurately classifying text emotion polarities and solve at least one technical problem existing in the background technology.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the invention provides a sentence text aspect level emotion classification method, which comprises the following steps:
performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
extracting final embedding of the classification task by using syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding, and predicting the final emotion polarity of the statement text.
Preferably, preprocessing operation is performed by utilizing GloVE word embedding, and each word is subjected to serialization representation to obtain word embedding representation of a text;
and extracting the characteristics of the sequence from the front direction and the rear direction by using a bidirectional long-time memory network Bi-LSTM to obtain the context sequence information of the captured sequence.
Preferably, a graph attention neural network based on a dependency relationship tree is constructed, and an extraction model is constructed for the dependency relationship by using syntactic dependency information of the dependency relationship tree;
extracting final embedding of classification tasks by using the constructed extraction model and combining information of the vector in the aspect of average pooling aggregation;
and inputting the extracted final embedded data into a final softmax classifier after passing through a full connection layer, so as to predict the final emotion polarity.
Preferably, aiming at Chinese memory and aspect memory of upper and lower sequence information, semantic fragments related to aspect words are extracted, the aspect memory is converted into structured aspect representation by self-attention operation, and an aspect matrix is obtained;
adding a variety of weighted sum vectors in the penalty term acquisition aspect representation;
and obtaining the relation between the aspect matrixes, constructing a context matrix, transforming the context matrix by utilizing a feedforward network, and combining the context matrix with the context matrix to obtain the final structural context expression.
Preferably, the words in the text sentence represent nodes in a dependency tree, and the syntactic dependency path between the words represents a node edge in the dependency tree, and the nodes of the dependency tree are given by real-valued vectors modeled by Bi-LSTM;
distributing attention to a neighbor node set of a central node, normalizing the attention coefficient, and recalculating the weight coefficient;
capturing the influence of neighbor nodes on a central node in different aspects through a multi-head attention mechanism, and splicing the extracted characteristic representations of the plurality of nodes to serve as final node representations;
and combining the recalculated weight coefficients, and using average substitution splicing to obtain the final embedding.
Preferably, the word embedding preprocessing operation is performed using GloVE word embedding, giving a context sentence S of length n ═ w1,w2,...,wnAn aspect context input sequence comprising an aspect, an aspect a ═ wi,wi+1,...,wi+m-1Contains m words;
each word wiWord-embedded vector mapping to a low dimension
Figure BDA0003009718050000037
In dwIs the dimension of the word vector and,
Figure BDA0003009718050000038
is an embedded matrix of pre-trained GloVE, where | V | represents the size of the vocabulary.
Preferably, the hidden state of the forward LSTM output at time t is the hidden state of the forward LSTM output by extracting the characteristics of the sequence from the front direction and the back direction by using the Bi-LSTM network
Figure BDA0003009718050000031
Hidden states of the inverted LSTM output are
Figure BDA0003009718050000032
The hidden state of the Bi-LSTM output is
Figure BDA0003009718050000033
wherein ,
Figure BDA0003009718050000034
Figure BDA0003009718050000035
Figure BDA0003009718050000036
sequence h is divided into context memory McAnd aspect memory Ma；McContaining representations of all context words, MaIncluding all facet word representations.
In a second aspect, the present invention provides a sentence text aspect level emotion classification system, including:
a sequence representation module; the system comprises a word processor, a word processing module and a word processing module, wherein the word processor is used for carrying out serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
the extraction module is used for extracting final embedding by utilizing syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and the prediction module is used for calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding and predicting the final emotion polarity of the statement text.
In a third aspect, the invention provides a non-transitory computer readable storage medium comprising instructions for performing the sentence text aspect level emotion classification method described above.
In a fourth aspect, the invention provides an electronic device comprising a non-transitory computer readable storage medium as described above; and one or more processors capable of executing the instructions of the non-transitory computer-readable storage medium.
The invention has the beneficial effects that: the problem of long-distance word dependence among a plurality of words is solved by using the syntactic dependence structure in the sentence, and the dependence relationship which is neglected in the previous research is solved; a structured self-attention mechanism is designed to encode sentences into a multi-dimensional matrix, wherein each vector can be regarded as a context related to a facet word to generate a context representation of the facet, and the relation of a plurality of semantic segments and the facet word is revealed.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a sentence text aspect level emotion classification method according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of sentence text aspect level emotion classification results according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by way of the drawings are illustrative only and are not to be construed as limiting the invention.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the various embodiments or examples and features of the various embodiments or examples described in this specification can be combined and combined by those skilled in the art without contradiction.
For the purpose of facilitating an understanding of the present invention, the present invention will be further explained by way of specific embodiments with reference to the accompanying drawings, which are not intended to limit the present invention.
It should be understood by those skilled in the art that the drawings are merely schematic representations of embodiments and that the elements shown in the drawings are not necessarily required to practice the invention.
Example 1
The embodiment 1 of the present invention provides a sentence text aspect level emotion classification system, including:
a sequence representation module; the system comprises a word processor, a word processing module and a word processing module, wherein the word processor is used for carrying out serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
the extraction module is used for extracting final embedding by utilizing syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and the prediction module is used for calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding and predicting the final emotion polarity of the statement text.
In this embodiment 1, the system described above is used to implement a sentence text aspect level emotion classification method, which includes:
performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
extracting final embedding of the classification task by using syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding, and predicting the final emotion polarity of the statement text.
Preprocessing by utilizing GloVE word embedding, and performing serialization representation on each word to obtain word embedding representation of a text; and extracting the characteristics of the sequence from the front direction and the rear direction by using a bidirectional long-time memory network Bi-LSTM to obtain the context sequence information of the captured sequence.
Constructing a graph attention neural network based on a dependency relationship tree, and constructing an extraction model for the dependency relationship by using sentence-method dependency information of the dependency relationship tree; extracting final embedding of classification tasks by utilizing the constructed extraction model and combining information of vectors in the aspect of average pooling aggregation; and inputting the extracted final embedded data into a final softmax classifier after passing through a full connection layer, so as to predict the final emotion polarity.
Aiming at Chinese memory and aspect memory of upper and lower sequence information, semantic fragments related to aspect words are extracted, the aspect memory is converted into structured aspect representation by self-attention operation, and a square matrix is obtained; adding a variety of weighted sum vectors in the penalty term acquisition aspect representation; and obtaining the relation between the aspect matrixes, constructing a context matrix, transforming the context matrix by using a feedforward network, and combining the context matrix with the context matrix to obtain the final structural context expression.
Representing words in the text sentence as nodes in a dependency relationship tree, representing syntax dependence paths among the words as node edges in the dependency relationship tree, wherein the nodes of the dependency relationship tree are given by real-value vectors modeled by Bi-LSTM; distributing attention to a neighbor node set of a central node, normalizing the attention coefficient, and recalculating the weight coefficient; capturing the influence of neighbor nodes on a central node in different aspects through a multi-head attention mechanism, and splicing the extracted characteristic representations of the plurality of nodes to serve as final node representations; and combining the recalculated weight coefficients, and using average substitution splicing to obtain the final embedding.
Word embedding preprocessing operation using GloVE word embedding, giving a context sentence S of length n ═ w1,w2,...,wnWhich comprisesContext input sequence of aspect, aspect a ═ { wi,wi+1,...,wi+m-1Contains m words;
each word wiWord-embedded vector mapping to a low dimension
Figure BDA0003009718050000071
In dwIs the dimension of the word vector and,
Figure BDA0003009718050000072
is an embedded matrix of pre-trained GloVE, where | V | represents the size of the vocabulary.
The characteristics of the sequence are extracted from the front direction and the back direction by using the Bi-LSTM network, and the hidden state of the forward LSTM output at the time t is
Figure BDA0003009718050000073
Hidden states of the inverted LSTM output are
Figure BDA0003009718050000074
The hidden state of the Bi-LSTM output is
Figure BDA0003009718050000075
wherein ,
Figure BDA0003009718050000076
Figure BDA0003009718050000081
Figure BDA0003009718050000082
sequence h is divided into context memory McAnd aspect memory Ma；McContaining representations of all context words, MaIncluding all facet word representations.
Example 2
In embodiment 2 of the present invention, a structural self-attention mechanism and a graphical attention network are used to provide a sentence text aspect level emotion prediction method. Firstly, capturing context information between sentences by using a bidirectional long-short term memory network (BI-LSTM) to learn sentence representation; and then, a structured self-attention mechanism is utilized to capture context segments related to the emotion of the aspect words, and embedding is further enhanced through a graph attention network directly acting on a dependency relationship tree, so that syntactic information and word dependency relationships are obtained.
The syntactic dependency structure in the sentence is used for solving the problem of long-distance word dependency among a plurality of words and solving the dependency relationship which is neglected in the past research. A structured self-attention mechanism is designed to encode sentences into a multi-dimensional matrix, wherein each vector can be regarded as context related to aspect words to generate a context representation of the aspect, and the relation of a plurality of semantic segments and the aspect words is disclosed.
As shown in fig. 1, the method for predicting emotion in sentence text aspect level described in this embodiment 2 includes the following steps:
step S1: preprocessing operation is carried out by utilizing GloVE word embedding, and each word is subjected to serialization representation to obtain word embedding representation of a text;
step S2: the Bi-LSTM network can be used for extracting the characteristics of the sequence from the front direction and the back direction, and the context sequence information of the sequence can be well captured;
step S3: generating a structured aspect representation and a structured contextual representation by a structured self-attention mechanism;
step S4: constructing a graph attention neural network based on a dependency relationship tree, and modeling the dependency relationship by using syntactic dependency information of the dependency relationship tree;
step S5: when the final embedding of the classification task is extracted, the information of the vector in the aspect of average pooling aggregation is utilized;
step S6: and inputting the result after pooling into a final softmax classifier after passing through a full connection layer, thereby predicting the final emotion polarity.
In the step S1: given a context sentence of length n, S ═ w1,w2,...,wnThis is a context input sequence containing an aspect, a ═ wi,wi+1,...,wi+m-1Contains m words and the task is to guess the emotional polarity of aspect a in sentence S.
In this embodiment 2, each input word w is first enterediWord-embedded vector mapping to a low dimension
Figure BDA0003009718050000091
In dwIs the dimension of the word vector and,
Figure BDA0003009718050000092
is an embedded matrix of pre-trained GloVe, where | V | is the size of the vocabulary.
In the step S2: Bi-LSTM combines a forward hidden layer with a backward hidden layer to enable systematic and selective use of both the forward and backward information, often used to process context information in natural language processing tasks.
The Bi-LSTM network can extract the characteristics of the sequence from the front direction and the back direction and can well capture the context sequence information of the sequence. The hidden state of the positive LSTM output at time t is
Figure BDA0003009718050000093
Hidden states of the inverted LSTM output are
Figure BDA0003009718050000094
The hidden state of the Bi-LSTM output is
Figure BDA0003009718050000095
wherein ,
Figure BDA0003009718050000096
Figure BDA0003009718050000097
Figure BDA0003009718050000098
sequence h is divided into context memory McAnd aspect memory Ma；McContaining representations of all context words, MaIncluding all facet word representations.
In the step S3: given context memory McAnd aspect memory MaExtracting semantic segments related to the aspect words and memorizing the aspects M by using self-attention operationaConversion to structured aspect representation RaAs follows:
Figure BDA0003009718050000101
wherein ,AaIs a matrix of the weights that is,
Figure BDA0003009718050000102
and
Figure BDA0003009718050000103
are two parameters of the self-care layer.
Figure BDA0003009718050000104
Represents MaThe transposing of (1).
Multiplying the weight matrix with the aspect word to compute a weighted sum to obtain an aspect representation:
Ra＝AaMa；
if the attention mechanism always provides a similar weighted sum, the embedding matrix has redundancy problems. Therefore, in this embodiment 2, a penalty term is needed to encourage diversity of the weighted sum vectors.
Use of a penalty term P in the loss function to encourage conversion at RaThe multiplicity of rows captured.
Figure BDA0003009718050000105
wherein ,
Figure BDA0003009718050000106
is AaI represents a unit matrix, | × | | non-conducting phosphorFRepresents the Frobenius norm of the matrix.
Given aspect matrix RaMemorizing M from contextcFinding semantic segments associated with the aspect.
First, a matrix A is establishedcTo capture the relationships between the aspect matrices. A bilinear attention mechanism is used to capture the relationship between the context memory and the aspect matrix. Secondly, it is used to construct a context matrix
Figure BDA0003009718050000107
Each row in the matrix can be considered as a semantic segment related to an aspect:
Figure BDA0003009718050000108
Figure BDA0003009718050000109
wherein ,WcIs a parameter for the operation of the bilinear attention mechanism,
Figure BDA00030097180500001010
is McThe transposing of (1).
Further generating a transformed representation T using a feed-forward networkc：
Figure BDA00030097180500001011
wherein ,
Figure BDA00030097180500001012
is a learnable parameter of the feed-forward network.
The remaining connections are used to combine the two matrices to obtain the final structured context representation. Layer normalization is used to help prevent gradual extinction and explosion:
Figure BDA0003009718050000111
in the step S4: a dependency tree may be understood as a graph with N nodes, where the nodes represent words in a sentence and the edges represent syntactic dependency paths between words in the graph, and the nodes of the dependency tree are given by real-valued vectors modeled by Bi-LSTM.
The set of input feature vectors for a single graph attention layer is
Figure BDA0003009718050000112
The output node feature vector set is
Figure BDA0003009718050000113
The attention coefficient between the central node and the neighbor nodes is:
Figure BDA0003009718050000114
wherein ,
Figure BDA0003009718050000115
the input representing the ith node is embedded,
Figure BDA0003009718050000116
and the parameter linear transformation matrix is a parameter linear transformation matrix which maps the dimension of the input characteristic vector to the output dimension.
self-attention will assign attention to all nodes in the graph, which will lose structural information. In this embodiment 2, to solve this problem, attention is assigned to the neighbor node set of node i by using masked self-attribute. And, performing softmax normalization on the attention coefficient, recalculating the weight coefficient, wherein the updated coefficient is as follows:
Figure BDA0003009718050000117
and capturing the influence of the neighbor nodes on the central node in different aspects by a multi-head attention mechanism. And splicing the node feature representations respectively extracted by the K heads to obtain a final node representation:
Figure BDA0003009718050000118
wherein, | | represents the splicing operation,
Figure BDA0003009718050000119
denotes the normalized attention coefficient, W, calculated by the kth attention mechanismkIs the weight matrix of the corresponding input linear transformation.
Finally, using average substitution stitching to obtain the final embedding:
Figure BDA0003009718050000121
in the step S5: when extracting the final embeddings of the classification task, the information on the average pooled aggregated aspect vector is used:
Figure BDA0003009718050000122
where f (-) is the average function of the enhancement aspect vector.
In the step S6: the hidden state represents the probability distribution p (a) that is different emotion polarities is output through a fully connected softmax layer.
Figure BDA0003009718050000123
wherein ,WpIs a matrix of weight coefficients, bpIs a bias matrix.
In this embodiment 2, a network model is trained by using a back propagation algorithm, and an objective function loss is defined as:
Figure BDA0003009718050000124
wherein D is a training data set. Lambda [ alpha ]1 and λ2Control L2The variation of the regularization term, θ, represents all parameters. p is a radical ofiIs the i-th element of P, PiIs a penalty term for the ith training.
As shown in fig. 2, in which The comment is about a restaurant, The emotional polarity is judged for two aspects of The sentence "The noodles lower delicious, but The service water terriple", which represents The meaning "noodles are good for eating but have poor service", respectively, positive and negative. Among them, the emotional polarity evaluation for the facet "noddles" noodle is "positive", i.e., positive, and the emotional polarity evaluation for the facet "service is" negative ", i.e., negative.
Example 3
Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium including instructions for executing a sentence text aspect level emotion classification method, the method including:
performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
extracting final embedding of the classification task by using syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding, and predicting the final emotion polarity of the statement text.
Example 4
Embodiment 4 of the present invention provides an electronic device, including a non-transitory computer readable storage medium; and one or more processors capable of executing the instructions of the non-transitory computer-readable storage medium. The non-transitory computer readable storage medium includes instructions for performing a method for contextual emotion classification of sentence text, the method comprising:
performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
extracting final embedding of the classification task by using syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding, and predicting the final emotion polarity of the statement text.
Example 5
An embodiment 5 of the present invention provides an electronic device, where the device includes an instruction for executing a sentence text aspect level emotion classification method, and the method includes:
performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
extracting final embedding of the classification task by using syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding, and predicting the final emotion polarity of the statement text.
In summary, the sentence text aspect level emotion classification method and system provided by the embodiment of the invention utilize a structural self-attention mechanism and a graphical attention network. The model first learns sentence characterization by capturing context information between sentences by using a bidirectional long-short term memory network (BI-LSTM); and then, a structured self-attention mechanism is utilized to capture context segments related to the emotion of the aspect words, and embedding is further enhanced through a graph attention network directly acting on a dependency relationship tree, so that syntactic information and word dependency relationships are obtained. The syntactic dependency structure in the sentence is used for solving the problem of long-distance word dependency among a plurality of words and solving the dependency relationship which is neglected in the previous research. A structured self-attention mechanism is designed to encode sentences into a multi-dimensional matrix, wherein each vector can be regarded as a context related to an aspect word to generate a context representation of the aspect, and the relation of a plurality of semantic segments and the aspect word is disclosed.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Although the present disclosure has been described with reference to the specific embodiments shown in the drawings, it is not intended to limit the scope of the present disclosure, and it should be understood that various modifications and alterations can be made by those skilled in the art without inventive efforts based on the technical solutions disclosed in the present disclosure.

Claims (10)

1. A sentence text aspect level emotion classification method is characterized by comprising the following steps:
performing serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
extracting final embedding of the classification task by using syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding, and predicting the final emotion polarity of the sentence text.
2. The sentence text aspect level emotion classification method of claim 1, wherein:
preprocessing operation is carried out by utilizing GloVE word embedding, and each word is subjected to serialization representation to obtain word embedding representation of a text;
and extracting the characteristics of the sequence from the front direction and the rear direction by using a bidirectional long-time memory network Bi-LSTM to obtain the context sequence information of the captured sequence.
3. The sentence text aspect level emotion classification method of claim 2, wherein:
constructing a graph attention neural network based on a dependency relationship tree, and constructing an extraction model for the dependency relationship by using syntactic dependency information of the dependency relationship tree;
extracting final embedding of classification tasks by utilizing the constructed extraction model and combining information of vectors in the aspect of average pooling aggregation;
and inputting the extracted final embedded data into a final softmax classifier after passing through a full connection layer, so as to predict the final emotion polarity.
4. The sentence text aspect level emotion classification method of claim 3, wherein:
aiming at Chinese memory and aspect memory of upper and lower sequence information, semantic fragments related to aspect words are extracted, the aspect memory is converted into structured aspect representation by self-attention operation, and an aspect matrix is obtained;
adding a variety of weighted sum vectors in the penalty term acquisition aspect representation;
and obtaining the relation between the aspect matrixes, constructing a context matrix, transforming the context matrix by using a feedforward network, and combining the transformed context matrix with the context matrix to obtain the final structural context representation.
5. The sentence text aspect level emotion classification method of claim 4, wherein:
representing words in the text sentence as nodes in a dependency relationship tree, representing syntax dependence paths among the words as node edges in the dependency relationship tree, wherein the nodes of the dependency relationship tree are given by real-valued vectors modeled by Bi-LSTM;
distributing attention to a neighbor node set of a central node, normalizing the attention coefficient, and recalculating the weight coefficient;
capturing the influence of neighbor nodes on a central node in different aspects through a multi-head attention mechanism, and splicing the extracted characteristic representations of the plurality of nodes to serve as final node representations;
and combining the recalculated weight coefficients, and using average substitution splicing to obtain the final embedding.
6. The sentence text aspect level emotion classification method of claim 5, wherein:
word embedding preprocessing operation is performed by using GloVE word embedding, and a context sentence S with the length of n is given as { w ═ w1,w2,...,wnAn aspect context input sequence comprising an aspect, an aspect a ═ wi,wi+1,...,wi+m-1Contains m words;
each word wiWord-embedded vector mapping to a low dimension
Figure FDA0003009718040000021
In dwIs the dimension of the word vector and,
Figure FDA0003009718040000022
is an embedded matrix of pre-trained GloVE, where | V | represents the size of the vocabulary.
7. The sentence text aspect level emotion classification method of claim 6, wherein:
the characteristics of the sequence are extracted from the front direction and the back direction by using the Bi-LSTM network, and the hidden state of the forward LSTM output at the time t is
Figure FDA0003009718040000023
Hidden states of the inverted LSTM output are
Figure FDA0003009718040000024
The hidden state of the Bi-LSTM output is
Figure FDA0003009718040000025
wherein ,
Figure FDA0003009718040000026
Figure FDA0003009718040000027
Figure FDA0003009718040000028
sequence h is divided into context memory McAnd aspect memory Ma；McContaining representations of all context words, MaIncluding all facet word representations.
8. A sentence text aspect level emotion classification system, comprising:
a sequence representation module; the system comprises a word processor, a word processor and a word processor, wherein the word processor is used for carrying out serialization representation on each word, acquiring context sequence information of a sequence, and generating a structured aspect representation and a structured context representation through a structured self-attention mechanism;
the extraction module is used for extracting final embedding by utilizing syntactic dependency information of a dependency relationship tree and combining information of an average pooling aggregation aspect vector according to the structural aspect representation and the structural context representation;
and the prediction module is used for calculating probability distribution of different emotion polarities by combining a back propagation algorithm according to the final embedding and predicting the final emotion polarity of the statement text.
9. A non-transitory computer-readable storage medium comprising instructions for performing the sentence text aspect level emotion classification method of any of claims 1-7.
10. An electronic device, characterized in that: comprising the non-transitory computer-readable storage medium of claim 9; and one or more processors capable of executing the instructions of the non-transitory computer-readable storage medium.
CN202110372212.2A 2021-04-07 2021-04-07 Sentence text aspect-level emotion classification method and sentence text aspect-level emotion classification system Active CN113157919B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110372212.2A CN113157919B (en) 2021-04-07 2021-04-07 Sentence text aspect-level emotion classification method and sentence text aspect-level emotion classification system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110372212.2A CN113157919B (en) 2021-04-07 2021-04-07 Sentence text aspect-level emotion classification method and sentence text aspect-level emotion classification system

Publications (2)

Publication Number Publication Date
CN113157919A true CN113157919A (en) 2021-07-23
CN113157919B CN113157919B (en) 2023-04-25

Family

ID=76888564

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110372212.2A Active CN113157919B (en) 2021-04-07 2021-04-07 Sentence text aspect-level emotion classification method and sentence text aspect-level emotion classification system

Country Status (1)

Country Link
CN (1) CN113157919B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113609867A (en) * 2021-08-23 2021-11-05 南开大学 Method and system for learning context information based on single-layer network structure
CN113869058A (en) * 2021-08-31 2021-12-31 合肥工业大学 LC-GCN (LC-GCN-based) aspect level emotion analysis method and system, storage medium and electronic equipment
CN113869034A (en) * 2021-09-29 2021-12-31 重庆理工大学 Aspect emotion classification method based on reinforced dependency graph
CN114707518A (en) * 2022-06-08 2022-07-05 四川大学 Semantic fragment-oriented target emotion analysis method, device, equipment and medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108829662A (en) * 2018-05-10 2018-11-16 浙江大学 A kind of conversation activity recognition methods and system based on condition random field structuring attention network
CN109543039A (en) * 2018-11-23 2019-03-29 中山大学 A kind of natural language sentiment analysis method based on depth network
CN111078833A (en) * 2019-12-03 2020-04-28 哈尔滨工程大学 Text classification method based on neural network
CN111461004A (en) * 2020-03-31 2020-07-28 北京邮电大学 Event detection method and device based on graph attention neural network and electronic equipment
CN111488734A (en) * 2020-04-14 2020-08-04 西安交通大学 Emotional feature representation learning system and method based on global interaction and syntactic dependency
CN111783474A (en) * 2020-07-16 2020-10-16 厦门市美亚柏科信息股份有限公司 Comment text viewpoint information processing method and device and storage medium
CN112347248A (en) * 2020-10-30 2021-02-09 山东师范大学 Aspect-level text emotion classification method and system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108829662A (en) * 2018-05-10 2018-11-16 浙江大学 A kind of conversation activity recognition methods and system based on condition random field structuring attention network
CN109543039A (en) * 2018-11-23 2019-03-29 中山大学 A kind of natural language sentiment analysis method based on depth network
CN111078833A (en) * 2019-12-03 2020-04-28 哈尔滨工程大学 Text classification method based on neural network
CN111461004A (en) * 2020-03-31 2020-07-28 北京邮电大学 Event detection method and device based on graph attention neural network and electronic equipment
CN111488734A (en) * 2020-04-14 2020-08-04 西安交通大学 Emotional feature representation learning system and method based on global interaction and syntactic dependency
CN111783474A (en) * 2020-07-16 2020-10-16 厦门市美亚柏科信息股份有限公司 Comment text viewpoint information processing method and device and storage medium
CN112347248A (en) * 2020-10-30 2021-02-09 山东师范大学 Aspect-level text emotion classification method and system

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
JIE LIU等: "Graph Convolutional Networks with Bidirectional Attention for Aspect-Based Sentiment Classification", 《APPLIED SCIENCES》 *
QIUYUE ZHANG等: "Interactive Multi-Head Attention Networks for Aspect-Level Sentiment Classification", 《IEEE ACCESS (VOLUME: 7)》 *
宋婷等: "基于方面情感的层次化双注意力网络", 《信息技术与网络安全》 *
张秋悦: "基于注意力网络的文本情感分析在舆情***中的应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
李欢等: "依存关系树在文本多语义约束检测中的运用", 《小型微型计算机***》 *
杨玉亭等: "面向上下文注意力联合学习网络的方面级情感分类模型", 《模式识别与人工智能》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113609867A (en) * 2021-08-23 2021-11-05 南开大学 Method and system for learning context information based on single-layer network structure
CN113609867B (en) * 2021-08-23 2024-02-02 南开大学 Method and system for learning context information based on single-layer network structure
CN113869058A (en) * 2021-08-31 2021-12-31 合肥工业大学 LC-GCN (LC-GCN-based) aspect level emotion analysis method and system, storage medium and electronic equipment
CN113869034A (en) * 2021-09-29 2021-12-31 重庆理工大学 Aspect emotion classification method based on reinforced dependency graph
CN113869034B (en) * 2021-09-29 2022-05-20 重庆理工大学 Aspect emotion classification method based on reinforced dependency graph
CN114707518A (en) * 2022-06-08 2022-07-05 四川大学 Semantic fragment-oriented target emotion analysis method, device, equipment and medium
CN114707518B (en) * 2022-06-08 2022-08-16 四川大学 Semantic fragment-oriented target emotion analysis method, device, equipment and medium

Also Published As

Publication number Publication date
CN113157919B (en) 2023-04-25

Similar Documents

Publication Publication Date Title
CN110083705B (en) Multi-hop attention depth model, method, storage medium and terminal for target emotion classification
CN111368993B (en) Data processing method and related equipment
WO2023024412A1 (en) Visual question answering method and apparatus based on deep learning model, and medium and device
CN113157919B (en) Sentence text aspect-level emotion classification method and sentence text aspect-level emotion classification system
CN110321563B (en) Text emotion analysis method based on hybrid supervision model
CN110502753A (en) A kind of deep learning sentiment analysis model and its analysis method based on semantically enhancement
CN109947931B (en) Method, system, device and medium for automatically abstracting text based on unsupervised learning
CN112163426A (en) Relationship extraction method based on combination of attention mechanism and graph long-time memory neural network
CN111753081A (en) Text classification system and method based on deep SKIP-GRAM network
CN111368086A (en) CNN-BilSTM + attribute model-based sentiment classification method for case-involved news viewpoint sentences
CN111414749B (en) Social text dependency syntactic analysis system based on deep neural network
CN112232087B (en) Specific aspect emotion analysis method of multi-granularity attention model based on Transformer
CN112667818A (en) GCN and multi-granularity attention fused user comment sentiment analysis method and system
CN108536735B (en) Multi-mode vocabulary representation method and system based on multi-channel self-encoder
CN107679225A (en) A kind of reply generation method based on keyword
CN113535953A (en) Meta learning-based few-sample classification method
CN114925205B (en) GCN-GRU text classification method based on contrast learning
CN113255366A (en) Aspect-level text emotion analysis method based on heterogeneous graph neural network
CN114254645A (en) Artificial intelligence auxiliary writing system
Xu et al. BERT gated multi-window attention network for relation extraction
CN113806543B (en) Text classification method of gate control circulation unit based on residual jump connection
CN111259147A (en) Sentence-level emotion prediction method and system based on adaptive attention mechanism
CN113722439A (en) Cross-domain emotion classification method and system based on antagonism type alignment network
CN116757195B (en) Implicit emotion recognition method based on prompt learning
CN115422369B (en) Knowledge graph completion method and device based on improved TextRank

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant