CN113961698A - Intention classification method, system, terminal and medium based on neural network model - Google Patents
Intention classification method, system, terminal and medium based on neural network model Download PDFInfo
- Publication number
- CN113961698A CN113961698A CN202010681562.2A CN202010681562A CN113961698A CN 113961698 A CN113961698 A CN 113961698A CN 202010681562 A CN202010681562 A CN 202010681562A CN 113961698 A CN113961698 A CN 113961698A
- Authority
- CN
- China
- Prior art keywords
- intention classification
- neural network
- finite state
- regular expression
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000003062 neural network model Methods 0.000 title claims abstract description 24
- 230000014509 gene expression Effects 0.000 claims abstract description 48
- 238000013145 classification model Methods 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 16
- 238000012216 screening Methods 0.000 claims abstract description 11
- 239000011159 matrix material Substances 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 238000010187 selection method Methods 0.000 claims 1
- 238000013528 artificial neural network Methods 0.000 abstract description 30
- 230000000306 recurrent effect Effects 0.000 abstract description 9
- 125000004122 cyclic group Chemical group 0.000 abstract description 2
- 238000004422 calculation algorithm Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000013140 knowledge distillation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an intention classification method based on a neural network model, which comprises the following steps: acquiring intention classification task data; writing a regular expression rule and corresponding logic selection operation aiming at the intention classification dataset; converting the regular expression into a finite state automaton; converting the finite state automaton into a three-dimensional tensor; decomposing the three-dimensional tensor by using a tensor decomposition method to obtain parameters of three matrix initialization TFRNN; initializing parameters of a logic screening layer, and forming an intention classification model by the logic screening layer and the TFRNN; and inputting the intention classification task data into an intention classification model for training, and outputting an intention classification result. The method converts the regular expression into a corresponding cyclic neural network form, and initializes the neural network correctly, so that the operation of the neural network in untrained time is equivalent to the direct operation of the regular expression. The recurrent neural network is very strong in interpretability after being trained.
Description
Technical Field
The invention relates to the technical field of software, in particular to an intention classification method, system, terminal and medium based on a neural network model.
Background
The regular expression is a mature rule system and has important application in the field of natural language processing. Due to the characteristics of controllable and interpretable rules and high accuracy, the method is not a choice in some natural language processing applications with high requirements on model accuracy. However, rule-based systems also have certain drawbacks, such as the labor cost required to write a large amount of rules, and the inability to make good use of large amounts of data. In the big data era, the neural network is used as a model based on data learning, so that a large amount of data can be well utilized to learn knowledge in the data, but correspondingly, the neural network also has obvious defects, such as poor interpretability, uncontrollable property, labor cost for marking a large amount of data, and difficulty in achieving high accuracy rate. The existing methods that combine regular expressions with neural networks have some drawbacks, and the mainstream methods usually rely on multitask LEARNING or knowledge distillation, such as "LEARNING FROM fans genetic training street explorer". Although good results can be achieved on downstream tasks, they do not completely merge the advantages of neural networks and regular expressions, rules are just one way to assist training, neural networks are still unexplainable, and the overall system is still uncontrollable. Meanwhile, in some systems, the result of the usage rule still cannot be got rid of after the model training is finished as the input of the model.
Disclosure of Invention
Aiming at the defects in the prior art, the embodiment of the invention provides an intention classification method, system, terminal and medium based on a neural network model, which have cold start capability, can achieve the accuracy rate close to that of a regular expression when no training is needed, and the trained model has strong interpretability.
In a first aspect, an intention classification method based on a neural network model provided in an embodiment of the present invention includes:
acquiring intention classification task data;
writing a regular expression rule and corresponding logic selection operation aiming at the intention classification dataset;
converting the regular expression into a finite state automaton;
converting the finite state automaton into a three-dimensional tensor;
decomposing the three-dimensional tensor by using a tensor decomposition method to obtain parameters of three matrix initialization TFRNN;
initializing parameters of a logic screening layer, and forming an intention classification model by the logic screening layer and the TFRNN;
and inputting the intention classification task data into an intention classification model for training, and outputting an intention classification result.
Further, the three-dimensional tensor contains all information of the regular expression, the first dimension is the size of a word list of words related to the finite state automaton, and the second dimension and the third dimension are the number of states of the finite state automaton.
Further, the logical selection layer includes three logical operations including and operation, or operation, and priority selection.
Further, the method for selecting the priority comprises the following steps: low priority score-low priority result-high priority result.
In a second aspect, an intention classification system based on a neural network model provided by an embodiment of the present invention includes: a data acquisition module, a rule writing module, a first conversion module, a second conversion module, a tensor decomposition module, an intention classification model generation module and a training module,
the data acquisition module is used for acquiring intention classification task data;
the rule writing module is used for writing a regular expression rule and corresponding logic selection operation aiming at the intention classification data set;
the first conversion module is used for converting the regular expression into a finite state automaton;
the second conversion module is used for converting the finite state automaton into a three-dimensional tensor;
the tensor decomposition module is used for decomposing the three-dimensional tensor by using a tensor decomposition method to obtain parameters of three matrix initialization TFRNN;
the intention classification model generation module is used for initializing parameters of the logic screening layer and forming an intention classification model by the logic screening layer and the TFRNN;
the training module is used for inputting the intention classification task data into the intention classification model for training and outputting the intention classification result.
Further, the three-dimensional tensor contains all information of the regular expression, the first dimension is the size of a word list of words related to the finite state automaton, and the second dimension and the third dimension are the number of states of the finite state automaton.
Further, the logical selection layer includes three logical operations including and operation, or operation, and priority selection.
Further, the method for selecting the priority comprises the following steps: low priority score-low priority result-high priority result.
In a third aspect, an intelligent terminal provided in an embodiment of the present invention includes a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are connected to each other, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method described in the foregoing embodiment.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, the computer program including program instructions, which, when executed by a processor, cause the processor to execute the method described in the above embodiment.
The invention has the beneficial effects that:
the intent classification method, system, terminal and medium based on the neural network model provided by the embodiment of the invention convert the regular expression into the corresponding circular neural network form, and initialize the neural network correctly, so that the operation of the neural network is equivalent to the direct operation of the regular expression when the neural network is not trained. The recurrent neural network is very strong in interpretability after being trained.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
FIG. 1 is a flow chart of an intent classification method based on a neural network model according to a first embodiment of the present invention;
FIG. 2 is a block diagram illustrating an intention classification system based on a neural network model according to another embodiment of the present invention;
fig. 3 shows a block diagram of an intelligent terminal according to another embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
As shown in fig. 1, there is shown a flowchart of an intent classification method based on a neural network model according to a first embodiment of the present invention, the method includes the following steps:
and S101, acquiring intention classification task data.
Specifically, the intention classification task is a prediction of an intention category for a given text, and is a text classification task.
And S102, writing a regular expression rule and corresponding logic selection operation aiming at the intention classification data set.
Specifically, there may be several regular expressions for each category of text, and if the text is matched by a certain category of regular expressions, it is considered that the intention category of the text is likely to belong to this category. If the regular expressions of multiple categories are matched with texts, a logic selection mechanism is needed, so that the regular expression rules and the corresponding logic selection operation are written aiming at the intention classification data sets, and the data processing is convenient to perform later.
And S103, converting the regular expression into a finite state automaton.
And S104, converting the finite state automaton into a three-dimensional tensor.
The three-dimensional tensor contains all information of the regular expression, the first dimension is the size of a word list of words related to the finite state automaton, and the second dimension and the third dimension are the number of states of the finite state automaton.
And S105, decomposing the three-dimensional tensor by using a tensor decomposition method to obtain the parameters of the three matrix initialization TFRNN.
And S106, initializing parameters of the logic selection layer, and forming an intention classification model by the logic selection layer and the TFRNN.
Specifically, the regular expressions of each category are converted into their corresponding finite state automata. For example, we have a simple regular expression for the intent "query time of flight", as shown in the second row of table 1, where "$" indicates that any input is accepted and "-" indicates that the preceding character can be repeated any number of times. Converting the regular expression into a finite state automaton in the third row of table 1, wherein s0 of the finite state automaton represents a starting state, s2 represented by two circles represents an ending state, and if the finite state automaton reaches the ending state from the starting state after reading the text, judging that the regular expression is matched.
TABLE 1 conversion of regular expressions into finite state automata
The finite state automaton is then converted to a three-dimensional tensor that contains all the information for the regular expression. The first dimension is the vocabulary size of the words related to the finite state automaton, and the second dimension and the third dimension are the state number of the finite state automaton. This tensor can be viewed as being a stack of transition matrices for each word. Taking the automaton of Table 1 as an example, for the word flight, its transfer matrix T has a size of 3x3, where T [0 ]][2]＝T[0][0]＝T[1][1]＝T[2][2]1, the rest is 0. After conversion to tensor representation, we can run finite state automata using the classical Forward Algorithm (Forward Algorithm) and the Viterbi Algorithm (Viterbi Algorithm) in dynamic programming. Taking the viterbi algorithm as an example, a vector with one dimension as the size of the state number, called a state vector h, can be obtained at each time tt. Wherein the ith element represents what the highest scoring path is from the start state to state i after t words have been accepted, read. The state vector at the later moment t +1 can be calculated by the state vector at the moment t and the state transition matrix corresponding to the t +1 th word to obtain ht+1＝f(ht，Tt+1). This process is very similar to the calculation process of the recurrent neural network (TFRNN), and the state vector can be regarded as a hidden vector (hidden vector) in the recurrent neural network. Through loop calculation, in the state vector of the last moment, the value corresponding to the end state determines whether the regular expression is matched after the whole sentence is read. Specifically, for the forward algorithm, if the final value of the end state is greater than or equal to 1, the regular expressions match, and for the viterbi algorithm, if the value of the end state is equal to 1, the regular expressions match.
Because the three-dimensional tensor may be overlarge, tensor rank decomposition is carried out on the three-dimensional tensor by adopting a tensor decomposition method, the three-dimensional tensor of A multiplied by B multiplied by C is decomposed into three matrixes of size A multiplied by R, B multiplied by R and C multiplied by R, wherein R is the size of the rank, the tensor is approximated by the three matrixes, and cyclic calculation is carried out by the three matrixes obtained by decomposition. By the method, parameters of the intention classification model can be greatly reduced, computing resources consumed by training the intention classification model are reduced, the training speed of the intention classification model is increased, and the word vector is introduced to improve the effect and generalization capability of the model.
And the logic selection layer is used for sorting the operation result of the finite state automaton into a score of each category. The logic selection layer comprises three logic operations including AND operation, OR operation and priority selection. The method for selecting the priority comprises the following steps: low priority score-low priority result-high priority result.
Through steps S103-S106, the regular expression is converted into a corresponding trainable recurrent neural network, and the recurrent neural network is initialized, so that the operation of the neural network in the training process is equivalent to the direct operation of the regular expression, and the accuracy rate close to that of the regular expression can be achieved without any training. And the logic screening layer and the TFRNN form an intention classification model, and the whole model can be trained.
And S107, inputting the intention classification task data into an intention classification model for training, and outputting an intention classification result.
Tests were performed on the SMS text classification dataset using three different settings of zero data, a small amount of data, and all data, and the results of the obtained experimental data are shown in table 2:
table 2 SMS data set experimental results
Through the above experiment, the intention classification model in this embodiment has better effect than the traditional recurrent neural network, and mainly reflects in:
1. better performance than the baseline model under small data training.
2. The full data set is fully trained, still due to the fully trained baseline model.
3. The trained model can be restored into a finite state automaton, and has extremely strong interpretability.
In the intention classification method based on the neural network model provided by the embodiment, the regular expression is converted into the corresponding circular neural network form, and the neural network is initialized correctly, so that the operation of the neural network is equivalent to the direct operation of the regular expression when the neural network is not trained. The recurrent neural network is very strong in interpretability after being trained.
Correspondingly, another embodiment of the invention also provides an intention classification system based on the neural network model. Please refer to fig. 2, which is a schematic structural diagram of an intention classification system based on a neural network model according to another embodiment of the present invention. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
As shown in fig. 2, it is shown that an intention classification system based on a neural network model according to another embodiment of the present invention includes: the system comprises a data acquisition module 201, a rule composition module 202, a first conversion module 203, a second conversion module 204, a tensor decomposition module 205, an intention classification model generation module 206 and a training module 207, wherein the data acquisition module 201 is used for acquiring input intention classification data; the rule writing module 202 is configured to write a regular expression rule and a corresponding logical selection operation for the intention classification dataset; the first conversion module 203 is used for converting the regular expression into a finite state automaton; the second conversion module 204 is configured to convert the finite state automaton into a three-dimensional tensor; the tensor decomposition module 205 is configured to decompose a three-dimensional tensor by using a tensor decomposition method to obtain parameters of three matrix initialization TFRNNs; the three-dimensional tensor contains all information of the regular expression, the first dimension is the size of a word list of words designed by the finite state automaton, and the second dimension and the third dimension are the number of states of the finite state automaton. The intention classification model generation module 206 is configured to initialize parameters of the logical selection layer, and form an intention classification model from the logical selection layer and the TFRNN; the training module 207 is configured to input the input intention classification data into an intention classification model for training, and output an intention classification result. The logic selection layer comprises three logic operations including AND operation, OR operation and priority selection. The method for selecting the priority comprises the following steps: low priority score-low priority result-high priority result.
In the intention classification system based on the neural network model provided by the embodiment, the regular expression is converted into the corresponding circular neural network form, and the neural network is initialized correctly, so that the operation of the neural network is equivalent to the direct operation of the regular expression when the neural network is not trained. The recurrent neural network is very strong in interpretability after being trained.
In another embodiment of the present invention, an intelligent terminal is further provided, as shown in fig. 3, which shows a block diagram of a structure of the intelligent terminal, the terminal includes a processor 301, an input device 302, an output device 303, and a memory 304, the processor 301, the input device 302, the output device 303, and the memory 304 are connected to each other, the memory 304 is used for storing a computer program, the computer program includes program instructions, and the processor 301 is configured to call the program instructions to execute the method described in the above embodiment.
It should be understood that, in the embodiment of the present invention, the Processor 301 may be a Central Processing Unit (CPU), and the Processor may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 302 may include a touch pad, a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.
The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.
In a specific implementation, the processor 301, the input device 302, and the output device 303 described in this embodiment of the present invention may execute the implementation described in the method embodiment provided in this embodiment of the present invention, and may also execute the implementation described in the system embodiment described in this embodiment of the present invention, which is not described herein again.
The invention also provides an embodiment of a computer-readable storage medium, in which a computer program is stored, which computer program comprises program instructions that, when executed by a processor, cause the processor to carry out the method described in the above embodiment.
The computer readable storage medium may be an internal storage unit of the terminal described in the foregoing embodiment, for example, a hard disk or a memory of the terminal. The computer readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the terminal. The computer-readable storage medium is used for storing the computer program and other programs and data required by the terminal. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the terminal and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed terminal and method can be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (10)
1. An intention classification method based on a neural network model is characterized by comprising the following steps:
acquiring intention classification task data;
writing a regular expression rule and corresponding logic selection operation aiming at the intention classification dataset;
converting the regular expression into a finite state automaton;
converting the finite state automaton into a three-dimensional tensor;
decomposing the three-dimensional tensor by using a tensor decomposition method to obtain parameters of three matrix initialization TFRNN;
initializing parameters of a logic screening layer, and forming an intention classification model by the logic screening layer and the TFRNN;
and inputting the intention classification task data into an intention classification model for training, and outputting an intention classification result.
2. The neural network model-based intention classification method of claim 1, wherein the three-dimensional tensor contains all information of a regular expression, a first dimension is a vocabulary size of a word related to the finite state automaton, and a second dimension and a third dimension are state numbers of the finite state automaton.
3. The neural network model-based intent classification method of claim 1, wherein the logical selection layer comprises three logical operations including and operation, or operation and priority selection.
4. The neural network model-based intention classification method according to claim 3, characterized in that the priority selection method is as follows: low priority score-low priority result-high priority result.
5. An intent classification system based on a neural network model, comprising: a data acquisition module, a rule writing module, a first conversion module, a second conversion module, a tensor decomposition module, an intention classification model generation module and a training module,
the data acquisition module is used for acquiring intention classification task data;
the rule writing module is used for writing a regular expression rule and corresponding logic selection operation aiming at the intention classification data set;
the first conversion module is used for converting the regular expression into a finite state automaton;
the second conversion module is used for converting the finite state automaton into a three-dimensional tensor;
the tensor decomposition module is used for decomposing the three-dimensional tensor by using a tensor decomposition method to obtain parameters of three matrix initialization TFRNN;
the intention classification model generation module is used for initializing parameters of the logic screening layer and forming an intention classification model by the logic screening layer and the TFRNN;
the training module is used for inputting the intention classification task data into the intention classification model for training and outputting the intention classification result.
6. The neural network model-based intention classification system of claim 5, characterized in that the three-dimensional tensor contains all information of regular expressions, a first dimension is a vocabulary size of a word involved in the finite state automaton, and a second dimension and a third dimension are state numbers of the finite state automaton.
7. The neural network model-based intent classification system of claim 5, wherein the logical selection layer includes three logical operations including an AND operation, an OR operation, and priority selection.
8. The neural network model-based intention classification system of claim 6, characterized in that the method of priority selection is: low priority score-low priority result-high priority result.
9. An intelligent terminal comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, the memory being adapted to store a computer program, the computer program comprising program instructions, characterized in that the processor is configured to invoke the program instructions to perform the method according to any of claims 1-4.
10. A computer-readable storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010681562.2A CN113961698A (en) | 2020-07-15 | 2020-07-15 | Intention classification method, system, terminal and medium based on neural network model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010681562.2A CN113961698A (en) | 2020-07-15 | 2020-07-15 | Intention classification method, system, terminal and medium based on neural network model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113961698A true CN113961698A (en) | 2022-01-21 |
Family
ID=79459305
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010681562.2A Pending CN113961698A (en) | 2020-07-15 | 2020-07-15 | Intention classification method, system, terminal and medium based on neural network model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113961698A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114860942A (en) * | 2022-07-05 | 2022-08-05 | 北京云迹科技股份有限公司 | Text intention classification method, device, equipment and storage medium |
| CN114997168A (en) * | 2022-05-27 | 2022-09-02 | 国家电网公司华中分部 | Power grid fault handling plan analysis method based on neural regular expression |
| CN117971354A (en) * | 2024-03-29 | 2024-05-03 | 苏州元脑智能科技有限公司 | Heterogeneous acceleration method, device, equipment and storage medium based on end-to-end learning |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102142009A (en) * | 2010-12-09 | 2011-08-03 | 华为技术有限公司 | Method and device for matching regular expressions |
| US20180012139A1 (en) * | 2016-07-06 | 2018-01-11 | Facebook, Inc. | Systems and methods for intent classification of messages in social networking systems |
| KR101938212B1 (en) * | 2018-05-29 | 2019-01-15 | 전남대학교산학협력단 | Subject based document automatic classification system that considers meaning and context |
| CN109284406A (en) * | 2018-09-03 | 2019-01-29 | 四川长虹电器股份有限公司 | Intension recognizing method based on difference Recognition with Recurrent Neural Network |
| US20190043487A1 (en) * | 2017-08-02 | 2019-02-07 | Veritone, Inc. | Methods and systems for optimizing engine selection using machine learning modeling |
| CN109902175A (en) * | 2019-02-20 | 2019-06-18 | 上海方立数码科技有限公司 | A kind of file classification method and categorizing system based on neural network structure model |
| KR20190089100A (en) * | 2017-12-26 | 2019-07-30 | 삼성생명보험주식회사 | Computer program stored on computer-readable medium, method and server for responding to a customer inquiry |
| CN110162610A (en) * | 2019-04-16 | 2019-08-23 | 平安科技（深圳）有限公司 | Intelligent robot answer method, device, computer equipment and storage medium |
| US10528866B1 (en) * | 2015-09-04 | 2020-01-07 | Google Llc | Training a document classification neural network |
| US20200065384A1 (en) * | 2018-08-26 | 2020-02-27 | CloudMinds Technology, Inc. | Method and System for Intent Classification |
| CN110909864A (en) * | 2019-10-22 | 2020-03-24 | 北京大学 | Natural language task processing method and device combining regular expression and neural network |
| CN111221944A (en) * | 2020-01-13 | 2020-06-02 | 平安科技（深圳）有限公司 | Text intention recognition method, device, equipment and storage medium |
-
2020
- 2020-07-15 CN CN202010681562.2A patent/CN113961698A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102142009A (en) * | 2010-12-09 | 2011-08-03 | 华为技术有限公司 | Method and device for matching regular expressions |
| US10528866B1 (en) * | 2015-09-04 | 2020-01-07 | Google Llc | Training a document classification neural network |
| US20180012139A1 (en) * | 2016-07-06 | 2018-01-11 | Facebook, Inc. | Systems and methods for intent classification of messages in social networking systems |
| US20190043487A1 (en) * | 2017-08-02 | 2019-02-07 | Veritone, Inc. | Methods and systems for optimizing engine selection using machine learning modeling |
| KR20190089100A (en) * | 2017-12-26 | 2019-07-30 | 삼성생명보험주식회사 | Computer program stored on computer-readable medium, method and server for responding to a customer inquiry |
| KR101938212B1 (en) * | 2018-05-29 | 2019-01-15 | 전남대학교산학협력단 | Subject based document automatic classification system that considers meaning and context |
| US20200065384A1 (en) * | 2018-08-26 | 2020-02-27 | CloudMinds Technology, Inc. | Method and System for Intent Classification |
| CN109284406A (en) * | 2018-09-03 | 2019-01-29 | 四川长虹电器股份有限公司 | Intension recognizing method based on difference Recognition with Recurrent Neural Network |
| CN109902175A (en) * | 2019-02-20 | 2019-06-18 | 上海方立数码科技有限公司 | A kind of file classification method and categorizing system based on neural network structure model |
| CN110162610A (en) * | 2019-04-16 | 2019-08-23 | 平安科技（深圳）有限公司 | Intelligent robot answer method, device, computer equipment and storage medium |
| CN110909864A (en) * | 2019-10-22 | 2020-03-24 | 北京大学 | Natural language task processing method and device combining regular expression and neural network |
| CN111221944A (en) * | 2020-01-13 | 2020-06-02 | 平安科技（深圳）有限公司 | Text intention recognition method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 孟奎;刘梦赤;胡婕;: "基于字符级循环网络的查询意图识别模型", 计算机工程, no. 03, 15 March 2017 (2017-03-15) * |
| 陈振舜;刘建明;: "基于意图的神经网络对话模型", 桂林电子科技大学学报, no. 05, 25 October 2018 (2018-10-25) * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114997168A (en) * | 2022-05-27 | 2022-09-02 | 国家电网公司华中分部 | Power grid fault handling plan analysis method based on neural regular expression |
| CN114997168B (en) * | 2022-05-27 | 2024-05-28 | 国家电网公司华中分部 | Power grid fault handling plan analysis method based on neuronization regular expression |
| CN114860942A (en) * | 2022-07-05 | 2022-08-05 | 北京云迹科技股份有限公司 | Text intention classification method, device, equipment and storage medium |
| CN114860942B (en) * | 2022-07-05 | 2022-10-04 | 北京云迹科技股份有限公司 | Text intention classification method, device, equipment and storage medium |
| CN117971354A (en) * | 2024-03-29 | 2024-05-03 | 苏州元脑智能科技有限公司 | Heterogeneous acceleration method, device, equipment and storage medium based on end-to-end learning |
| CN117971354B (en) * | 2024-03-29 | 2024-06-14 | 苏州元脑智能科技有限公司 | Heterogeneous acceleration method, device, equipment and storage medium based on end-to-end learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Raschka et al. | Python machine learning: Machine learning and deep learning with Python, scikit-learn, and TensorFlow 2 | |
| CN111753081B (en) | System and method for text classification based on deep SKIP-GRAM network | |
| Chen et al. | Big data deep learning: challenges and perspectives | |
| CN113961698A (en) | Intention classification method, system, terminal and medium based on neural network model | |
| CN111310441A (en) | Text correction method, device, terminal and medium based on BERT (binary offset transcription) voice recognition | |
| Zhang et al. | Deep autoencoding topic model with scalable hybrid Bayesian inference | |
| US20180365594A1 (en) | Systems and methods for generative learning | |
| Rivas | Deep Learning for Beginners: A beginner's guide to getting up and running with deep learning from scratch using Python | |
| KR102250728B1 (en) | Sample processing method and device, related apparatus and storage medium | |
| KR102444457B1 (en) | Method for dialogue summarization with word graphs | |
| CN113239169A (en) | Artificial intelligence-based answer generation method, device, equipment and storage medium | |
| US20190057084A1 (en) | Method and device for identifying information | |
| CN112686049A (en) | Text auditing method, device, equipment and storage medium | |
| CN111339775A (en) | Named entity identification method, device, terminal equipment and storage medium | |
| US20230334075A1 (en) | Search platform for unstructured interaction summaries | |
| CN111985243A (en) | Emotion model training method, emotion analysis device and storage medium | |
| Ahmad | 40 Algorithms Every Programmer Should Know: Hone your problem-solving skills by learning different algorithms and their implementation in Python | |
| CN112132281B (en) | Model training method, device, server and medium based on artificial intelligence | |
| CN113626576A (en) | Method and device for extracting relational characteristics in remote supervision, terminal and storage medium | |
| WO2022174499A1 (en) | Method and apparatus for predicting text prosodic boundaries, computer device, and storage medium | |
| CN112560463A (en) | Text multi-labeling method, device, equipment and storage medium | |
| WO2023116572A1 (en) | Word or sentence generation method and related device | |
| CN113177406B (en) | Text processing method, text processing device, electronic equipment and computer readable medium | |
| CN115906861A (en) | Statement emotion analysis method and device based on interaction aspect information fusion | |
| CN111767710B (en) | Indonesia emotion classification method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |