CN112417859A - Intention recognition method, system, computer device and computer-readable storage medium - Google Patents

Intention recognition method, system, computer device and computer-readable storage medium Download PDF

Info

Publication number
CN112417859A
CN112417859A CN202011328294.2A CN202011328294A CN112417859A CN 112417859 A CN112417859 A CN 112417859A CN 202011328294 A CN202011328294 A CN 202011328294A CN 112417859 A CN112417859 A CN 112417859A
Authority
CN
China
Prior art keywords
text
keyword
character
recognized
keywords
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011328294.2A
Other languages
Chinese (zh)
Inventor
江丹丹
景艳山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Minglue Zhaohui Technology Co Ltd
Original Assignee
Beijing Minglue Zhaohui Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Minglue Zhaohui Technology Co Ltd filed Critical Beijing Minglue Zhaohui Technology Co Ltd
Priority to CN202011328294.2A priority Critical patent/CN112417859A/en
Publication of CN112417859A publication Critical patent/CN112417859A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/12—Use of codes for handling textual entities
    • G06F40/126—Character encoding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Character Discrimination (AREA)

Abstract

The present application relates to an intention recognition method, system, computer device and computer-readable storage medium, wherein the intention recognition method includes: a data acquisition step, which is used for acquiring a text to be identified; a data preprocessing step, namely segmenting the text to be recognized, extracting keywords, and converting segmentation results and the keywords into vector representations to obtain character vectors and keyword vectors of the text to be recognized; a text coding step, which is used for respectively coding the character vector and the keyword vector by utilizing a recurrent neural network to obtain the character code and the keyword code of the text to be recognized; and an intention identification step, namely splicing the character codes and the keyword codes to obtain sentence codes of the text to be identified, classifying the sentence codes through a first full connection layer and a Softmax layer, outputting text classification results, and finishing intention identification. Through the method and the device, the word segmentation effect accuracy is effectively improved, and the problem that the classification effect of simple samples is good and the classification effect of difficult samples is poor in the prior art is solved.

Description

Intention recognition method, system, computer device and computer-readable storage medium
Technical Field
The present application relates to the field of dialog system technology, and in particular, to an intention recognition method, system, computer device, and computer-readable storage medium.
Background
In a dialog system, intention identification is an indispensable task, and for each request sent to the system by a user, the system needs to accurately understand the intention of the user and then give a correct response, so that the satisfaction degree of the user is improved. And the person interacts with the system for multiple times, so that multiple rounds of conversation can be realized. If the intention is recognized incorrectly, the given response also deviates from the user's true needs, which will inevitably reduce the user's interest in continuing to use the dialog system.
The intention identification is a classification problem, and the existing method mainly comprises the following steps:
1) the method for identifying the intention based on the rule template needs to artificially construct the rule template and the category information to classify the intention text of the user. For example, for the aeronautical field, there are many different expressions of "ticket booking":
'I order an air ticket';
②' flight with XX removal does ";
and thirdly, helping me to check an air ticket, and the like.
The intention recognition according to the rule template for the intention "booking ticket" is that the rule formulated for this intention is: three words of 'air ticket booking' appear in the sentence, and the intention of the sentence is 'air ticket booking'. So far, only the first sentence can be recognized as "air ticket booking", and the second sentence and the third sentence cannot be recognized.
Although a certain recognition accuracy can be guaranteed without a large amount of training data, the coverage rate of the template is low, and many intentions of query input cannot be recognized, especially under the condition of serious spoken language; when the text is intended to replace the category, a professional is required to reconstruct the template, and the problem of high cost caused by the reconstruction cannot be solved.
2) The method needs to extract key features of a corpus text, such as keywords, N-Gram and the like, and then train a classifier to perform intention classification. Common methods include naive Bayes, support vector machines, logistic regression, Adaboost, etc. Taking a support vector machine as an example for intention identification, the method comprises the following four steps: text feature extraction, text feature representation, normalization processing and text classification.
When the text characteristics are extracted, the words are firstly segmented and stop words are removed, the characteristics are extracted by a word frequency characteristic selection method, for example, the weight is calculated by using the word frequency and the words are sequenced, and TopK words are selected to represent the semantics of the text. When the word frequency is directly used for comparison, larger deviation is easy to occur, and text classification can be more accurate by normalization. After feature extraction, feature representation and normalization processing, the original text information is abstracted into a vectorized sample set, similarity calculation is carried out on the sample set and the trained template file in the text classification step, and if the sample set does not belong to the category, the sample set and template files of other categories are calculated until the sample set is classified into the corresponding category.
The intention identification method based on the traditional machine learning needs manual extraction of features such as keywords, N-Gram and the like, the cost is high, and the accuracy of the features cannot be guaranteed; the problem of data sparseness exists; meanwhile, the traditional machine learning method is low in classification accuracy and poor in generalization performance.
3) Intention identification method based on neural network
With the continuous development of deep learning, more and more students apply Chinese character vectors, convolutional neural networks and cyclic neural networks to the task of intention recognition. The data sparseness problem can be caused by using the original lexical characteristics, the data sparseness problem can be solved by introducing the Chinese character vectors, and the representation capability and the field expandability of different classification contents are better; the convolutional neural network is adopted to extract text vector representation as query classification features to identify the user intention, so that deeper feature representation can be obtained compared with the traditional machine learning method, but the convolutional neural network can only extract local features and cannot keep semantic consistency; a model based on a recurrent neural network (e.g., LSTM/GRU) provides for user input in the form of sequences, the semantic information of which is learned from context. The final output of the hidden state is used as the intention text representation, so that the intention category result is obtained.
However, the intention recognition method based on deep learning only considers word vectors or word vectors for the Chinese intention recognition model, and only considers the word vectors and cannot accurately express the information of the current context because the words have various combinations and ambiguity; considering only the word vectors may introduce noise, since the word segmentation effect is not completely correct. On the other hand, the cross entropy is basically adopted as a loss function, the effect of each sample on the model is considered equally, the importance of samples difficult to classify is not considered in a key mode, and the problem that the classification effect of simple samples is good and the classification effect of samples difficult to classify is poor is caused.
Disclosure of Invention
The embodiment of the application provides an intention identification method, an intention identification system, computer equipment and a computer readable storage medium, wherein intention identification based on multi-granularity vectors is realized by fusing word vectors and keyword vectors, and the system pays more attention to harder samples by utilizing Focal loss to inhibit the weight of the simple samples, so that the word segmentation effect accuracy is effectively improved, and the problem that the classification effect of the prior art on the simple samples is good and the classification effect of the harder samples is poor is solved.
In a first aspect, an embodiment of the present application provides an intention identification method, including:
a data acquisition step, which is used for acquiring a text to be identified;
a data preprocessing step, namely segmenting the text to be recognized, extracting keywords, and converting segmentation results and the keywords into vector representations to obtain character vectors and keyword vectors of the text to be recognized;
a text coding step, which is used for respectively coding the character vector and the keyword vector by utilizing a recurrent neural network to obtain the character code and the keyword code of the text to be recognized; the character codes are used for representing texts based on word vectors and increasing inter-word information in words, and the keyword codes are used for representing texts based on word vectors and are used for weakening the high ambiguity of characters and accurately expressing the current context information;
and an intention identification step, namely splicing the character codes and the keyword codes to obtain sentence codes of the text to be identified, classifying the sentence codes through a first full connection layer and a Softmax layer, outputting text classification results, and finishing intention identification. Optionally, the sentence codes are obtained by vector addition of the character codes and the keyword codes.
Through the steps, the text is expressed by fusing the character vector and the keyword vector of the text, on one hand, the character vector is used for fully expressing the literal information of the text, on the other hand, the keyword vector is used for reserving the semantic characteristics of the text, and the text information is reserved to the greatest extent.
Considering that in the training process, the loss function generally selects a cross entropy loss function, the cross entropy loss function treats all samples equally, and the effect is poor when the sample classification difficulty is unbalanced or positive and negative samples are unbalanced. In the intention recognition task, on one hand, as training samples are spoken contents of a user, some texts are easy to recognize intentions, while some texts are difficult to recognize intentions, and the difficulty degree of the samples is greatly different; on the other hand, for a vertical domain, the user may only care about some aspects and pay less attention to others, resulting in the class training sample distribution being unbalanced, and in some embodiments, the classification loss function of the Softmax layer is a Focal loss function. Specifically, the Focal distance is a distance for solving the problems of unbalanced classification and difficulty difference in classification, and the model is more concentrated on samples difficult to classify during training by reducing the weight of samples easy to classify. The method solves the problems of large difference and unbalanced samples in the prior art due to the difficulty in identifying the samples by using the Focal length as a loss function of the intention identification.
In some of these embodiments, the data preprocessing step further comprises:
a data extraction step, which is used for segmenting the text to be recognized and extracting keywords of the text to be recognized to obtain characters and keywords of the text; specifically, the word segmentation is obtained by using a word segmentation tool such as Jieba or HanLP (HanLP for short), and the keyword extraction is obtained by TF-IDF (Term Frequency-Inverse Document Frequency, TF-IDF for short), textrank algorithm and LDA (late Dirichlet Allocation, LDA for short);
a data coding step, which is used for carrying out vector coding on the characters and the keywords by utilizing One-Hot coding and converting the characters and the keywords into three-dimensional character vectors and three-dimensional keyword vectors through a second full connection layer; One-Hot coding, also known as One-bit-efficient coding, is a method of using an N-bit status register to encode N states, each state being represented by its own independent register bit and only One of which is active at any time. Specifically, the weight of the second fully-connected layer is initialized randomly and optimized continuously in the training process. Optionally, the data encoding step may also implement vector transformation by using a BERT pre-training model.
In some of these embodiments, the recurrent neural network is one or any combination of LSTM, GRU, Bi-LSTM, and Bi-GRU.
In a second aspect, an embodiment of the present application provides an intention recognition system, including:
the data acquisition module is used for acquiring a text to be identified;
the data preprocessing module is used for segmenting words of the text to be recognized, extracting keywords, converting segmentation results and the keywords into vector representations, and obtaining character vectors and keyword vectors of the text to be recognized;
the text coding module is used for coding the character vector and the keyword vector respectively by utilizing a recurrent neural network to obtain a character code and a keyword code of the text to be recognized; the character codes are used for representing texts based on word vectors and increasing inter-word information in words, and the keyword codes are used for representing texts based on word vectors and are used for weakening the high ambiguity of characters and accurately expressing the current context information;
and the intention identification module is used for splicing the character codes and the keyword codes to obtain sentence codes of the text to be identified, classifying the sentence codes through a first full-connection layer and a Softmax layer, outputting text classification results and finishing intention identification. Optionally, the sentence codes are obtained by vector addition of the character codes and the keyword codes.
Through the module, the text is expressed by fusing the character vector and the keyword vector of the text, on one hand, the character vector is used for fully expressing the literal information of the text, on the other hand, the keyword vector is used for reserving the semantic characteristics of the text, and the text information is reserved to the greatest extent.
In some of these embodiments, the classification loss function of the Softmax layer is a Focal loss function. By using the Focal length as a loss function of the intention identification, the problems of large difference of difficulty in identifying the samples and unbalanced samples are solved.
In some embodiments, the data preprocessing module further comprises:
the data extraction module is used for segmenting the text to be recognized and extracting keywords of the text to be recognized to obtain characters and keywords of the text; specifically, the word segmentation is obtained by using a word segmentation tool such as Jieba or HanLP, and the keyword extraction is obtained by TF-IDF, textrank and LDA extraction methods;
the data coding module is used for carrying out vector coding on the characters and the keywords by utilizing One-Hot coding and converting the characters and the keywords into three-dimensional character vectors and three-dimensional keyword vectors through a second full connection layer; specifically, the weight of the second fully-connected layer is initialized randomly and optimized continuously in the training process. Optionally, the data encoding module may also implement vector transformation through a BERT pre-training model.
In some of these embodiments, the recurrent neural network is one or any combination of LSTM, GRU, Bi-LSTM, and Bi-GRU.
In a third aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor, when executing the computer program, implements the intent recognition method according to the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the intent recognition method as described in the first aspect above.
Compared with the related art, the intention recognition method, the system, the computer equipment and the computer readable storage medium provided by the embodiment of the application realize intention recognition by fusing text character vectors and keyword vectors, solve the problem of semantic loss only considering word vectors or only considering word vectors in the prior art, improve recognition accuracy, do not need manual feature extraction, reduce working cost, and simultaneously use FocalLoss as a loss function, solve the problems that a common cross entropy loss function lacks sample importance weight and samples are unbalanced, weaken the weight of simple samples, and pay more attention to the influence of the samples on the model.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic flow diagram of an intent recognition method according to an embodiment of the present application;
FIG. 2 is a block diagram of an intent recognition system according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an intent recognition method in accordance with a preferred embodiment of the present application;
fig. 4 is a schematic diagram of the encoding principle of step S202 according to the preferred embodiment of the present application.
Description of the drawings:
10. a data acquisition module; 20. a data preprocessing module; 30. a text encoding module;
40. an intent recognition module; 201. a data extraction module; 202. and a data encoding module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.
It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.
The present embodiment provides an intention identifying method. Fig. 1 is a schematic flow chart of an intention identification method according to an embodiment of the present application, and referring to fig. 1, the flow chart includes the following steps:
a data acquisition step S10, configured to acquire a text to be recognized;
a data preprocessing step S20, configured to perform word segmentation on the text to be recognized, extract a keyword, convert a word segmentation result and the keyword into vector representations, and obtain a character vector and a keyword vector of the text to be recognized;
a text encoding step S30, configured to encode the character vector and the keyword vector respectively by using a recurrent neural network, so as to obtain a character code and a keyword code of the text to be recognized; wherein the recurrent neural network is one or any combination of LSTM, GRU, Bi-LSTM and Bi-GRU;
and an intention recognition step S40, which is used for splicing the character codes and the keyword codes to obtain sentence codes of the text to be recognized, classifying the sentence codes through a first full-link layer and a Softmax layer, outputting text classification results, and finishing intention recognition. Optionally, the sentence codes are obtained by vector addition of character codes and keyword codes, and the classification loss function of the Softmax layer is a local loss function.
Through the steps, the text is expressed by fusing the character vector and the keyword vector of the text, on one hand, the character vector is used for fully expressing the literal information of the text, on the other hand, the keyword vector is used for reserving the semantic characteristics of the text, and the text information is reserved to the greatest extent; by using the Focal length as a loss function of the intention identification, the problems of large difference of difficulty in identifying the samples and unbalanced samples are solved.
In some of these embodiments, the data preprocessing step S20 further includes:
a data extraction step S201, configured to perform word segmentation on the text to be recognized, and extract keywords of the text to be recognized, so as to obtain characters and keywords of the text; specifically, the word segmentation is obtained by using a word segmentation tool such as Jieba or HanLP, and the keyword extraction is obtained by one or any of TF-IDF, textrank and LDA extraction methods;
a data encoding step S202, which is used for carrying out vector encoding on the characters and the keywords by using One-Hot encoding and converting the characters and the keywords into three-dimensional character vectors and three-dimensional keyword vectors through a second full connection layer; specifically, the weight of the second fully-connected layer is initialized randomly and optimized continuously in the training process. Optionally, the data encoding step may also implement vector transformation by using a BERT pre-training model.
The embodiments of the present application are described and illustrated below by means of preferred embodiments. Fig. 3-4 are schematic diagrams illustrating the intent recognition method according to the preferred embodiment of the present application, and the preferred embodiment of the present application will be described and illustrated with reference to fig. 1 and 3-4.
First, a text to be recognized is acquired via step S10, and the process advances to step S20.
In step S20, considering that the chinese vocabulary is somewhat more specific than the simple words and contains richer semantic information, for example, we can see that "air ticket" can clearly know what it is, and there are many possible meanings for "air ticket" and "ticket" when they exist alone can not accurately represent the current context information, so that word segmentation is required for the text. The text contains some stop words such as 'and' without positive influence on the semantics of the text, but with interference, the training and prediction speed of the model is reduced. The keywords in the text often represent the main information of the text most, and the text content is embodied more accurately, so the text keywords need to be extracted in step S201 first. Optionally, word segmentation is performed on the training text by using a word segmentation tool such as Jieba or HanLP, and then keywords of the text are extracted by using a TF-IDF method.
Referring to fig. 3, for the input text "i go on business tomorrow, help me see the ticket. "the jieba word segmentation result is the following characters: ' I ', ' tomorrow ', ' to ', ' go, ', ' help ', ' I ', ' see ', ' down ', ' ticket ', ' etc. ' ], keywords are calculated as [ ' business trip ', ' ticket ' ] by using a TF-IDF algorithm.
The characters and keywords are then data-encoded via step S202. Specifically, for each character, assuming that the dimension d after encoding is 3 (d is 128 in the experiment), as shown in fig. 4, the character is encoded into [0, 0, 1, 0], and then a three-dimensional encoding vector of the character is obtained through the second full-connected layer. And the weight of the second full-connection layer is initialized randomly, and is optimized continuously in the training process to obtain the final weight. And similarly, coding each keyword to obtain a three-dimensional vector. Thus, character vectors and keyword vectors of the text are obtained.
Then, in step S30, a sentence coding layer based on characters and a sentence coding layer based on keywords are performed, that is, the character vectors and the keyword vectors of the text are coded to obtain the character codes and the keyword codes of the text. As shown in FIG. 3, there is a two-part sequence model LSTM, LSTM1 uses the character vector in the text as input, and the obtained text is based onRepresentation of character vectors, LSTM2In the method, keywords in the text are used as input to obtain the representation of the text based on the keywords. The information among words in the words is increased based on the expression of the word vector, and the face information of the sentence can be retained to the greatest extent; the word vector-based representation can weaken the high ambiguity of characters and more accurately express the current context information. Through step S30, the character code e of the text can be obtainedsentenceAnd a keyword code ekeywords。
Finally, the characters of the text are coded esentenceAnd a keyword code ekeywordsSplicing and fusing to obtain a whole sentence code e'sentenceThe whole sentence is encoded e 'via a first full concatenation layer and a Softmax layer'sentenceMapping into categories, specifically, the functional expression of the Softmax layer is as follows:
Figure BDA0002795023230000081
where W and b are the weight parameter and the bias term.
Thus, a text classification result is obtained, and intent recognition is completed.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here.
The present embodiment provides an intention recognition system. Fig. 2 is a block diagram of an intention recognition system according to an embodiment of the present application. As shown in fig. 2, the intention recognition system includes: the system comprises a data acquisition module 10, a data preprocessing module 20, a text coding module 30, an intention identification module 40 and the like. Those skilled in the art will appreciate that the configuration of the intent recognition system illustrated in FIG. 2 is not intended to be limiting of the intent recognition system and may include more or less components than illustrated, or some components in combination, or a different arrangement of components.
The following describes the components of the intention recognition system in detail with reference to fig. 2:
the data acquisition module 10 is used for acquiring a text to be recognized;
and the data preprocessing module 20 is configured to perform word segmentation on the text to be recognized, extract keywords, convert word segmentation results and the keywords into vector representations, and obtain a character vector and a keyword vector of the text to be recognized.
The text coding module 30 is configured to respectively code the character vector and the keyword vector by using a recurrent neural network, so as to obtain a character code and a keyword code of the text to be recognized; the character codes are used for representing texts based on word vectors and increasing inter-word information in words, and the keyword codes are used for representing texts based on word vectors and are used for weakening the high ambiguity of characters and accurately expressing the current context information; optionally, the recurrent neural network is one or any combination of LSTM, GRU, Bi-LSTM and Bi-GRU.
And the intention identification module 40 is used for splicing the character codes and the keyword codes to obtain sentence codes of the text to be identified, classifying the sentence codes through a first full connection layer and a Softmax layer, outputting text classification results and completing intention identification, wherein the classification loss function of the Softmax layer is a Focal loss function. Optionally, the sentence codes are obtained by vector addition of the character codes and the keyword codes.
Through the module, the text is expressed by fusing the character vector and the keyword vector of the text, on one hand, the character vector is used for fully expressing the literal information of the text, on the other hand, the keyword vector is used for reserving the semantic characteristics of the text, and the text information is reserved to the greatest extent; by using the Focal length as a loss function of the intention identification, the problems of large difference of difficulty in identifying the samples and unbalanced samples are solved.
In some of these embodiments, the data preprocessing module 20 further comprises:
the data extraction module 201 is configured to perform word segmentation on the text to be recognized, and extract keywords of the text to be recognized to obtain characters and keywords of the text; specifically, the word segmentation is obtained by using a word segmentation tool such as Jieba or HanLP, and the keyword extraction is obtained by TF-IDF, textrank and LDA extraction methods.
The data coding module 202 is used for carrying out vector coding on the characters and the keywords by using One-Hot coding and converting the characters and the keywords into three-dimensional character vectors and three-dimensional keyword vectors through a second full connection layer; specifically, the weight of the second fully-connected layer is initialized randomly and optimized continuously in the training process. Optionally, the data encoding module may also implement vector transformation through a BERT pre-training model.
The above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
In addition, the method for recognizing the intention of the embodiment of the application described in conjunction with fig. 1 can be realized by a computer device. The computer device may comprise a memory, a processor and a computer program stored on said memory and executable on said processor.
In particular, the processor may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.
The memory may include, among other things, mass storage for data or instructions. By way of example, and not limitation, memory may include a Hard Disk Drive (Hard Disk Drive, abbreviated to HDD), a floppy Disk Drive, a Solid State Drive (SSD), flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. The memory may include removable or non-removable (or fixed) media, where appropriate. The memory may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory is a Non-Volatile (Non-Volatile) memory. In particular embodiments, the Memory includes Read-Only Memory (ROM) and Random Access Memory (RAM). The ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), Electrically rewritable ROM (earrom), or FLASH Memory (FLASH), or a combination of two or more of these, where appropriate. The RAM may be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), where the DRAM may be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended data output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), and the like.
The memory may be used to store or cache various data files for processing and/or communication use, as well as possibly computer program instructions for execution by the processor.
The processor implements any of the intent recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory.
In addition, in combination with the intention identification method in the above embodiments, the embodiments of the present application may be implemented by providing a computer-readable storage medium. The computer readable storage medium having stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement any of the intent recognition methods in the above embodiments.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. An intent recognition method, comprising:
a data acquisition step, which is used for acquiring a text to be identified;
a data preprocessing step, namely segmenting the text to be recognized, extracting keywords, and converting segmentation results and the keywords into vector representations to obtain character vectors and keyword vectors of the text to be recognized;
a text coding step, which is used for respectively coding the character vector and the keyword vector by utilizing a recurrent neural network to obtain the character code and the keyword code of the text to be recognized;
and an intention identification step, namely splicing the character codes and the keyword codes to obtain sentence codes of the text to be identified, classifying the sentence codes through a first full connection layer and a Softmax layer, outputting text classification results, and finishing intention identification.
2. The intent recognition method of claim 1, wherein the classification loss function of the Softmax layer is a Focalloss function.
3. The intention recognition method of claim 2, wherein the data preprocessing step further comprises:
a data extraction step, which is used for segmenting the text to be recognized and extracting keywords of the text to be recognized to obtain characters and keywords of the text;
and a data encoding step, which is used for carrying out vector encoding on the characters and the keywords by using One-Hot encoding and converting the characters and the keywords into three-dimensional character vectors and three-dimensional keyword vectors through a second full connection layer.
4. The intent recognition method of claim 1 or 3, wherein the recurrent neural network is one or any combination of LSTM, GRU, Bi-LSTM, and Bi-GRU.
5. An intent recognition system, comprising:
the data acquisition module is used for acquiring a text to be identified;
the data preprocessing module is used for segmenting words of the text to be recognized, extracting keywords, converting segmentation results and the keywords into vector representations, and obtaining character vectors and keyword vectors of the text to be recognized;
the text coding module is used for coding the character vector and the keyword vector respectively by utilizing a recurrent neural network to obtain a character code and a keyword code of the text to be recognized;
and the intention identification module is used for splicing the character codes and the keyword codes to obtain sentence codes of the text to be identified, classifying the sentence codes through a first full-connection layer and a Softmax layer, outputting text classification results and finishing intention identification.
6. The intent recognition system of claim 5, wherein the classification loss function of the Softmax layer is a Focalloss function.
7. The intent recognition system of claim 6, wherein said data pre-processing module further comprises:
the data extraction module is used for segmenting the text to be recognized and extracting keywords of the text to be recognized to obtain characters and keywords of the text;
and the data coding module is used for carrying out vector coding on the characters and the keywords by utilizing One-Hot coding and converting the characters and the keywords into three-dimensional character vectors and three-dimensional keyword vectors through a second full connection layer.
8. The intent recognition system of claim 5 or 7 wherein the recurrent neural network is one or any combination of LSTM, GRU, Bi-LSTM and Bi-GRU.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the intent recognition method of any of claims 1 to 4 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the intention recognition method of any one of claims 1 to 4.
CN202011328294.2A 2020-11-24 2020-11-24 Intention recognition method, system, computer device and computer-readable storage medium Pending CN112417859A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011328294.2A CN112417859A (en) 2020-11-24 2020-11-24 Intention recognition method, system, computer device and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011328294.2A CN112417859A (en) 2020-11-24 2020-11-24 Intention recognition method, system, computer device and computer-readable storage medium

Publications (1)

Publication Number Publication Date
CN112417859A true CN112417859A (en) 2021-02-26

Family

ID=74777597

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011328294.2A Pending CN112417859A (en) 2020-11-24 2020-11-24 Intention recognition method, system, computer device and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN112417859A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113297374A (en) * 2021-04-29 2021-08-24 军事科学院***工程研究院网络信息研究所 Text classification method based on BERT and word feature fusion
CN113535896A (en) * 2021-06-23 2021-10-22 北京达佳互联信息技术有限公司 Searching method, searching device, electronic equipment and storage medium
CN115310409A (en) * 2022-06-29 2022-11-08 杭州似然数据有限公司 Data encoding method, system, electronic device and storage medium
CN116580408A (en) * 2023-06-06 2023-08-11 上海任意门科技有限公司 Image generation method and device, electronic equipment and storage medium
CN117574918A (en) * 2024-01-15 2024-02-20 青岛冠成软件有限公司 Intelligent interaction method based on LSTM

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113297374A (en) * 2021-04-29 2021-08-24 军事科学院***工程研究院网络信息研究所 Text classification method based on BERT and word feature fusion
CN113297374B (en) * 2021-04-29 2023-09-12 军事科学院***工程研究院网络信息研究所 Text classification method based on BERT and word feature fusion
CN113535896A (en) * 2021-06-23 2021-10-22 北京达佳互联信息技术有限公司 Searching method, searching device, electronic equipment and storage medium
CN113535896B (en) * 2021-06-23 2024-04-19 北京达佳互联信息技术有限公司 Search method, search device, electronic equipment and storage medium
CN115310409A (en) * 2022-06-29 2022-11-08 杭州似然数据有限公司 Data encoding method, system, electronic device and storage medium
CN116580408A (en) * 2023-06-06 2023-08-11 上海任意门科技有限公司 Image generation method and device, electronic equipment and storage medium
CN116580408B (en) * 2023-06-06 2023-11-03 上海任意门科技有限公司 Image generation method and device, electronic equipment and storage medium
CN117574918A (en) * 2024-01-15 2024-02-20 青岛冠成软件有限公司 Intelligent interaction method based on LSTM
CN117574918B (en) * 2024-01-15 2024-05-03 青岛冠成软件有限公司 Intelligent interaction method based on LSTM

Similar Documents

Publication Publication Date Title
KR102304673B1 (en) Keyword extraction method, computer device, and storage medium
CN112417859A (en) Intention recognition method, system, computer device and computer-readable storage medium
CN110427461B (en) Intelligent question and answer information processing method, electronic equipment and computer readable storage medium
CN107085581B (en) Short text classification method and device
CN111709243B (en) Knowledge extraction method and device based on deep learning
CN111046179B (en) Text classification method for open network question in specific field
CN112464656B (en) Keyword extraction method, keyword extraction device, electronic equipment and storage medium
CN112417863B (en) Chinese text classification method based on pre-training word vector model and random forest algorithm
WO2022198750A1 (en) Semantic recognition method
CN112632226B (en) Semantic search method and device based on legal knowledge graph and electronic equipment
CN111539197A (en) Text matching method and device, computer system and readable storage medium
Ritu et al. Performance analysis of different word embedding models on bangla language
CN113282711B (en) Internet of vehicles text matching method and device, electronic equipment and storage medium
CN112581327B (en) Knowledge graph-based law recommendation method and device and electronic equipment
CN112115702A (en) Intention recognition method, device, dialogue robot and computer readable storage medium
CN113515632A (en) Text classification method based on graph path knowledge extraction
CN115952292B (en) Multi-label classification method, apparatus and computer readable medium
CN112183102A (en) Named entity identification method based on attention mechanism and graph attention network
CN112232070A (en) Natural language processing model construction method, system, electronic device and storage medium
CN113254637A (en) Grammar-fused aspect-level text emotion classification method and system
CN114491062B (en) Short text classification method integrating knowledge graph and topic model
CN111563378A (en) Multi-document reading understanding realization method for combined learning
Zhuang et al. An ensemble approach to conversation generation
CN111680132B (en) Noise filtering and automatic classifying method for Internet text information
CN113486143A (en) User portrait generation method based on multi-level text representation and model fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination