CN117235629B - Intention recognition method, system and computer equipment based on knowledge domain detection - Google Patents
Intention recognition method, system and computer equipment based on knowledge domain detection Download PDFInfo
- Publication number
- CN117235629B CN117235629B CN202311515015.7A CN202311515015A CN117235629B CN 117235629 B CN117235629 B CN 117235629B CN 202311515015 A CN202311515015 A CN 202311515015A CN 117235629 B CN117235629 B CN 117235629B
- Authority
- CN
- China
- Prior art keywords
- corpus
- intention
- knowledge
- semantic
- user input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000001514 detection method Methods 0.000 title claims abstract description 37
- 238000013145 classification model Methods 0.000 claims abstract description 26
- 239000013598 vector Substances 0.000 claims description 246
- 238000012545 processing Methods 0.000 claims description 27
- 238000012549 training Methods 0.000 claims description 23
- 238000013500 data storage Methods 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000005352 clarification Methods 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Machine Translation (AREA)
Abstract
The invention relates to the technical field of intention recognition, and particularly discloses an intention recognition method, an intention recognition system and computer equipment based on knowledge domain detection, which can detect the knowledge domain of a user input corpus based on a neighbor relation before entering an intention classification model, distinguish the intention in the knowledge domain from the intention outside the knowledge domain, and stop classifying the intention when judging the intention outside the knowledge domain so as to reduce the false judgment rate of the follow-up intention, improve the accuracy rate of the follow-up intention recognition and improve the experience of the user.
Description
Technical Field
The present invention relates to the field of intent recognition technologies, and in particular, to an intent recognition method, system, and computer device based on knowledge domain detection.
Background
Intent recognition refers to recognizing a user's needs by analyzing the user's language in order to provide accurate services and support. This technology continues to evolve with advances in artificial intelligence and natural language processing technology. Currently, intent recognition technology has been widely applied to various scenarios including intelligent customer service, virtual assistant, intelligent home, etc. The importance of intent recognition technology is that it can enhance user experience and enterprise efficiency. For users, the intent recognition technology can provide more personalized services, thereby improving user satisfaction and loyalty. For enterprises, the intention recognition technology can better know the demands and behaviors of users, formulate more effective marketing and sales strategies and improve sales efficiency and enterprise benefits.
Along with the continuous enrichment of service scenes and the continuous updating of service knowledge, the intentions expressed by users become various, and classification models often misjudge new intentions outside the knowledge domains, so that the user experience is poor.
Disclosure of Invention
The invention aims to solve the technical problem that in the prior art, a system cannot accurately judge whether the intention expressed by a user is an intra-knowledge domain intention or not so as to cause poor user experience, and provides an intention recognition method, system and computer equipment based on knowledge domain detection, which accurately judge whether the intention expressed by the user is an intra-knowledge domain intention or an extra-knowledge domain intention so as to improve the user experience.
In order to solve the technical problems, the invention provides the following technical scheme:
the invention discloses an intention recognition method based on knowledge domain detection, which comprises the following steps:
s1, constructing a knowledge base;
s2, mapping the corpus in the knowledge base to a high-dimensional semantic space based on a semantic embedding model to obtain semantic vectors of the corpus in the knowledge base, and storing the semantic vectors into a vector database;
s3, training an intention classification model based on a knowledge base;
s4, receiving user input corpus, and mapping the user input corpus to a high-dimensional semantic space based on a semantic embedding model to obtain a semantic vector of the user input corpus;
s5, calculating a neighbor relation between the semantic vector of the user input corpus and the semantic vector of the knowledge base, and judging whether the user input corpus belongs to the intra-knowledge domain intention or the extra-knowledge domain intention based on the neighbor relation;
s6, stopping intention recognition if the intention belongs to the intention outside the knowledge domain, starting the intention processing flow outside the knowledge domain, storing the user input corpus, and updating the knowledge base and the vector database when the stored user input corpus meets a specific condition;
and S7, if the intention belongs to the knowledge domain, entering an intention classification model, classifying the user input content, and judging the intention based on the score.
According to the intention recognition method based on the knowledge domain detection, before entering the intention classification model, the knowledge domain detection is carried out on the user input corpus based on the neighbor relation, the intention in the knowledge domain and the intention outside the knowledge domain are distinguished, and when the intention outside the knowledge domain is judged, the intention classification is stopped, so that the misjudgment rate of the follow-up intention is reduced, the accuracy rate of the follow-up intention recognition is improved, and the user experience is improved.
Further, in step S2, the corpus in the knowledge base is mapped to a high-dimensional semantic space based on the semantic embedding model, so as to obtain semantic vectors of the corpus in the knowledge base, and the semantic vectors are stored in a vector database, specifically:
inputting corpus in a knowledge base into a Roformer-sm model, a SimCSE model or a Sentence-BERT model in batches, encoding by a multi-layer encoder, selecting the output of the last layer encoder to form semantic vectors corresponding to the corpus, and storing key value pairs of the [ corpus: vectors ] into a vector database;
or in step S4, mapping the user input corpus to a high-dimensional semantic space based on the semantic embedding model to obtain a semantic vector of the user input corpus, which specifically includes:
inputting the corpus input by a user into a Roformer-smi model, a SimCSE model or a Sentence-BERT model in batches, encoding by a multi-layer encoder, and selecting the last layer encoder to output to form semantic vectors corresponding to the corpus.
Further, in step S2 or step S4, the last layer of encoder is selected to output after the encoding of the multi-layer encoder, so as to form a semantic vector corresponding to the corpus, specifically:
adopting a CLS vector method, namely selecting a starting mark [ CLS ] vector as a semantic vector of the corpus;
or adopting a mean value pooling method, namely carrying out mean value calculation on word vectors of all words in the corpus, and finally taking the mean value vectors as semantic vectors of the corpus;
or adopting a maximum value pooling method, namely carrying out maximum value solving operation on word vectors of all words in the corpus, and finally taking the maximum value vector as a semantic vector of the corpus.
Further, in step S3, training the intent classification model based on the knowledge base specifically includes:
the knowledge base is arranged to obtain { corpus: the method comprises the steps of training data, carrying out token coding, segment coding and position coding on the obtained training data in batches, sending the training data into an embedding layer to obtain token embedded vectors, segment embedded vectors and position embedded vectors, adding the three vectors into a multi-layer converter-encoder, inputting a position vector with a [ CLS ] symbol output by a last layer encoder into a linear classification head, reducing the dimension to a label dimension, calculating loss by adopting cross entropy, and carrying out gradient feedback.
Further, in step S5, a neighbor relation between the semantic vector of the user input corpus and the semantic vector of the knowledge base is calculated, and whether the user input corpus belongs to the intra-knowledge domain intention or the extra-knowledge domain intention is judged based on the neighbor relation, specifically:
the cosine distance between the semantic vector of the user input corpus and all semantic vectors in the vector database is calculated, when the cosine distance is smaller than a preset threshold value, the semantic vector of the user input corpus and all semantic vectors in the vector database are considered to be adjacent, when the number of the semantic vector of the user input corpus and all semantic vectors in the vector database is considered to be adjacent to each other to exceed the preset number, the intention of the user input corpus is considered to belong to the intention in the knowledge domain, and otherwise, the intention of the user input corpus belongs to the intention outside the knowledge domain.
Further, in step S6, the process flow of intent outside the knowledge domain is started, the user input corpus is stored, and when the stored user input corpus satisfies a specific condition, the knowledge base and the vector database are updated, specifically:
s61, storing the user input content which is judged as the out-of-knowledge domain intention into an out-of-knowledge domain intention library, and processing by adopting a strategy of repeated inquiry, clarification inquiry and spam reply;
s62, when the corpus in the knowledge domain outside-domain intent library meets a certain condition, the expert in the knowledge domain collates and generalizes the corpus in the knowledge domain outside-domain intent library, and the collated and generalized knowledge domain outside-domain intent is added into the current knowledge library to update the knowledge library;
s63, mapping the corpus in the updated knowledge base to a high-dimensional semantic space based on the semantic embedding model to obtain semantic vectors of the corpus in the updated knowledge base, and storing the semantic vectors into a vector database to update the vector database.
Further, in step S7, if the intention belongs to the knowledge domain, the intention classification model is entered, the user input content is classified, and the intention is determined based on the score, specifically:
and inputting the user input corpus which is judged to be the intention in the knowledge domain into an intention classification model, obtaining the probability value of each intention, and selecting a label with the maximum probability value as the intention of the user input corpus.
Further, in step S7, a probability value of each intention is obtained, specifically:
firstly, coding by a multi-layer coder, selecting hidden layer vectors at [ CLS ] head positions, inputting the hidden layer vectors into a linear layer, carrying out dimension reduction on the hidden layer vectors through the linear layer to obtain dimension reduction vectors, and then carrying out normalization on the dimension reduction vectors by using a sotmax layer to obtain probability values of each intention.
The invention discloses an intention recognition system based on knowledge domain detection, which comprises an intention recognition system based on knowledge domain detection, wherein the intention recognition system comprises a knowledge domain detection judgment module, a corpus acquisition module, a knowledge storage module, a vector data processing module, a vector data storage module, an intention recognition module and a knowledge domain outside intention storage module;
the corpus acquisition module is used for acquiring user input corpus and sending the acquired user input corpus to the vector data processing module;
the knowledge storage module is used for storing the corpus of the knowledge in the field and sending the stored corpus of the knowledge in the field to the vector data processing module and the intention recognition module;
the vector data processing module receives the linguistic data of the knowledge in the field sent by the knowledge storage module and the user input linguistic data collected by the linguistic data collection module, maps the linguistic data of the knowledge in the field or the user input linguistic data to a high-dimensional semantic space based on the semantic embedding model to obtain a semantic vector of the linguistic data of the knowledge in the field or a semantic vector of the user input linguistic data, wherein the semantic vector of the linguistic data of the knowledge in the field is stored in the vector data storage module, and the semantic vector of the user input linguistic data is sent to the knowledge field detection judgment module;
the vector data storage module is used for receiving and storing semantic vectors of the corpus of knowledge in the field;
the knowledge domain detection judging module is used for receiving semantic vectors of the user input corpus, comparing the semantic vectors of the user input corpus with semantic vectors of the intra-domain knowledge corpus in the vector data storage module, calculating a neighbor relation between the semantic vectors of the user input corpus and the semantic vectors of the intra-domain knowledge corpus in the vector data storage module, and judging whether the user input corpus belongs to the intra-domain intention or the extra-domain intention based on the neighbor relation; if the intention is the intention outside the knowledge domain, the input corpus of the user is sent to the intention storage module outside the knowledge domain, and if the intention is the intention inside the knowledge domain, the input corpus of the user is sent to the intention recognition module;
the intention recognition module is used for receiving the corpus of the knowledge in the field sent by the knowledge storage module so as to train the intention classification model, and simultaneously, is used for receiving the user input corpus which is sent by the knowledge domain detection judgment module and is judged to be the intention in the knowledge domain, inputting the user input corpus into the trained intention classification model, and judging the intention based on the score;
the knowledge domain external intention storage module is used for receiving and storing the user input corpus which is sent by the knowledge domain detection and judgment module and judged to be the knowledge domain external intention.
The computer equipment comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the intention recognition method based on the knowledge domain detection when executing the computer program.
The computer equipment can realize the intention recognition method based on the knowledge domain detection, so that the false judgment rate of the follow-up intention can be reduced, the accuracy rate of the follow-up intention recognition can be improved, and the experience of a user can be improved.
Drawings
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings. Like reference numerals refer to like parts throughout the drawings, and the drawings are not intentionally drawn to scale on actual size or the like, with emphasis on illustrating the principles of the invention.
FIG. 1 is a flow chart of the present invention.
Detailed Description
In order that the invention may be readily understood, a more complete description of the invention will be rendered by reference to the appended drawings.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
According to the specific implementation mode of the intention recognition method based on knowledge domain detection, the intention recognition method can detect the knowledge domain of the user input corpus based on the neighbor relation before entering the intention classification model, distinguish the intention in the knowledge domain from the intention outside the knowledge domain, and stop classifying the intention when judging the intention outside the knowledge domain, so that the false judgment rate of the follow-up intention is reduced, the accuracy rate of the follow-up intention recognition is improved, and the user experience is improved.
Referring to fig. 1, the method specifically comprises the following steps:
s1, constructing a knowledge base. The knowledge base is a knowledge storage and management system, which comprises various information, data and documents, specifically, related domain knowledge is extracted from various sources by related domain experts, the extracted related domain knowledge is sorted and classified (namely, the related domain experts label the knowledge corpus data and associate each sample with the corresponding semantics and intentions of the knowledge corpus data, so as to obtain the process of { corpus: intent } format data), and the sorted and classified knowledge is converted into a formalized model or chart (such as a knowledge graph, etc.), so that the relation and characteristics of the knowledge can be better understood, and the following { corpus in the knowledge base can be facilitated: intent format data is processed.
And S2, mapping the corpus in the knowledge base to a high-dimensional semantic space based on the semantic embedding model to obtain semantic vectors of the corpus in the knowledge base, and storing the semantic vectors into a vector database. Specifically, corpus in a knowledge base is input into a Roformer-smi model (namely a semantic understanding model), a SimCSE model or a Sentence-BERT model in batches, the corpus is encoded by a multi-layer encoder, the last layer of encoder is selected and output to form a semantic vector corresponding to the corpus, and { corpus: vector } key value pairs are stored in a vector database. The Roformer-smi model, the SimCSE model and the Sentence-BERT model are semantic embedded models, wherein the Roformer-smi takes Roformer as a network structure, combines natural language understanding and natural language generating two types of pre-training tasks, adopts a contrast learning and distilling method, and based on a large-scale similar Sentence pair training semantic understanding model, the Roformer introduces rotary position coding (RoPE) compared with a transducer, and the rotary position coding has the advantages of remote attenuation and adaptation line property attribute and is widely applied to various scenes; the SimCSE model is mainly characterized in that compared with a learning strategy, the transmission of semantic information can be enhanced, the quality of vectors can be improved, and meanwhile, a large amount of marking data is not needed, so that the SimCSE model is suitable for an unsupervised learning scene and has high data processing efficiency; the Sentence-BERT model is mainly suitable for processing tasks of a sense level, such as obtaining a vector representation of a Sentence, calculating text semantic similarity, and the like, and compared with a mode of directly using BERT to code each token and then summing or averaging, the Sentence-BERT can capture semantic information of the Sentence more accurately, and in addition, the Sentence-BERT can obtain a semantic vector with higher quality in a manner of fine tuning based on the BERT.
In the step S2, the final layer of encoder output is selected through the encoding of the multi-layer encoder, and semantic vectors corresponding to the corpus are formed; the method comprises the following steps: adopting a CLS vector method, namely selecting a starting mark [ CLS ] vector as a semantic vector of the corpus; or adopting a mean value pooling method, namely carrying out mean value calculation on word vectors of all words in the corpus, and finally taking the mean value vectors as semantic vectors of the corpus; or adopting a maximum value pooling method, namely carrying out maximum value solving operation on word vectors of all words in the corpus, and finally taking the maximum value vector as a semantic vector of the corpus.
And step S3, training an intention classification model based on a knowledge base. Training an intention classification model by combining a pre-training model with a classification head, wherein an encoder loads a self-pre-training language model, parameters of the self-training language model can participate in fine adjustment or freezing, and the classification head can select a linear classifier, CNN, LSTM/GRU, transformer and the like; specifically, for example: the knowledge base is arranged to obtain { corpus: intent training data, namely, carrying out token coding, segment coding and position coding on the obtained training data in batches, then sending the training data into an embedding layer to obtain token embedded vectors, segment embedded vectors and position embedded vectors, adding the three vectors into a multi-layer converter, inputting a [ CLS ] position vector output by a last layer of encoder into a linear classification head, reducing the dimension to a label dimension, calculating loss by adopting cross entropy, and carrying out gradient feedback. The knowledge base is arranged to obtain { corpus: intent training data, specifically, for { corpus in knowledge base: intent format data is cleaned and preprocessed to ensure quality and accuracy of the data, including removing noise, correcting spelling errors, processing punctuation, etc. In addition, the formula of the cross entropy calculation loss is as follows
，
Wherein x represents the value of a random variable, and p (x) and q (x) respectively represent the probabilities of the real label and the prediction result at the x position; after the loss is calculated using cross entropy, an optimization algorithm (e.g., random gradient descent SGD, adam, etc.) may be used to update the model parameters in a direction that minimizes the loss function using the gradient pass-back process.
And S4, receiving the user input corpus, and mapping the user input corpus to a high-dimensional semantic space based on a semantic embedding model to obtain a semantic vector of the user input corpus. Specifically, the corpus input by a user is input into a Roformer-smi model, a SimCSE model or a Sentence-BERT model in batches, the corpus is encoded by a multi-layer encoder, and the last layer encoder is selected for output to form a semantic vector corresponding to the corpus. The Roformer-smi model, the SimCSE model and the Sentence-BERT model are semantic embedded models, wherein the Roformer-smi model takes Roformer as a network structure, combines natural language understanding and natural language generating two types of pre-training tasks, adopts a contrast learning and distillation method, and is based on a large-scale similar Sentence pair training semantic understanding model, and compared with a transducer, the Roformer introduces a rotary position code (ROPE) which has the advantages of remote attenuation and adaptation line property attribute and is widely applied to name scenes; the semCSE model is mainly used for processing tasks of a sense level such as obtaining a vector representation of a Sentence, calculating text semantic similarity and the like, and compared with a mode of directly using BERT to code each token and then summing or averaging, the semanteme-BERT can more accurately capture the semantic information of the Sentence, and in addition, the semanteme-BERT also obtains a semantic vector with higher quality in a mode of fine tuning based on the BERT.
In the step S4, the final layer of encoder output is selected through the encoding of the multi-layer encoder, and semantic vectors corresponding to the corpus are formed; the method comprises the following steps: adopting a CLS vector method, namely selecting a starting mark [ CLS ] vector as a semantic vector of the corpus; or adopting a mean value pooling method, namely carrying out mean value calculation on word vectors of all words in the corpus, and finally taking the mean value vectors as semantic vectors of the corpus; or adopting a maximum value pooling method, namely carrying out maximum value solving operation on word vectors of all words in the corpus, and finally taking the maximum value vector as a semantic vector of the corpus.
S5, calculating a neighbor relation between the semantic vector of the user input corpus and the semantic vector of the knowledge base, and judging the user input based on the neighbor relationThe input corpus belongs to the intra-knowledge domain intention or the extra-knowledge domain intention. The calculation of the neighborhood relationship has various embodiments, the first embodiment: calculating cosine distances between semantic vectors of the user input corpus and all semantic vectors in the vector database, when the cosine distances are smaller than a preset threshold value, considering that the semantic vectors are adjacent to each other, and when the number of the semantic vectors of the user input corpus and all semantic vectors in the vector database, which are considered to be adjacent to each other, exceeds the preset number, considering that the intention of the user input corpus belongs to the intention in the knowledge domain, otherwise, the intention belongs to the intention outside the knowledge domain; wherein the cosine distance is calculated by the formulaA and B represent two vectors, I A I 2 And B 2 Representing the modular length of vector a and vector B, respectively. Second embodiment: calculating the Euclidean distance between the semantic vector of the user input corpus and all semantic vectors in the vector database, when the Euclidean distance is smaller than a preset threshold value, considering that the semantic vector of the user input corpus and all semantic vectors in the vector database are adjacent, and when the number of the semantic vector of the user input corpus and all semantic vectors in the vector database which are considered to be adjacent exceeds the preset number, considering that the intention of the user input corpus belongs to the intention in the knowledge domain, otherwise, the intention of the user input corpus belongs to the intention outside the knowledge domain; wherein the calculation formula of the Euclidean distance is
，
Wherein d (x, y) is the point (x 1 ，x 2 ……x n ) And point (y) 1 ，y 2 ……y n ) Euclidean distance between them.
And S6, stopping intention recognition if the intention belongs to the intention outside the knowledge domain, starting the intention processing flow outside the knowledge domain, storing the user input corpus, and updating the knowledge base and the vector database when the stored user input corpus meets a specific condition. Specifically, S61, storing the user input content determined as the intention outside the knowledge domain into the knowledge domain intention library, and processing by adopting a strategy of repeated inquiry, clarification inquiry and spam reply, so as to avoid influencing the user experience due to wrong answers; s62, when the corpus in the knowledge domain outside-meaning gallery meets a certain condition (such as accumulating for a specific time period or accumulating for a specific number, or adopting a data clustering algorithm for data mining, and when the corpus of a certain category reaches a preset number), the expert in the knowledge domain collates and generalizes the corpus in the knowledge domain outside-meaning gallery, and the collated and generalized knowledge domain outside-meaning is added into the current knowledge gallery to update the knowledge gallery, wherein the processing method for updating the knowledge gallery can be referred to the processing method for constructing the knowledge gallery in the step S1; s63, mapping the corpus in the updated knowledge base to a high-dimensional semantic space based on the semantic embedding model to obtain semantic vectors of the corpus in the updated knowledge base, and storing the semantic vectors into a vector database to update the vector database.
And S7, entering an intention classification model if the intention belongs to the knowledge domain, classifying the user input content, and judging the intention based on the score. Specifically, a user input corpus which is judged as an intention in a knowledge domain is input into an intention classification model, a probability value of each intention is obtained, a label with the maximum probability value is selected as the intention of the user input corpus, and a subsequent dialog flow is advanced based on the intention. The method for obtaining the probability value of each intention is as follows: firstly, coding by a multi-layer coder, selecting hidden layer vectors at the [ CLS ] head position, inputting the hidden layer vectors into a linear layer, and carrying out dimension reduction treatment on the hidden layer vectors through the linear layer to obtain dimension reduction vectors so as to reduce calculated amount and model parameters and keep main semantic information; then, normalizing the dimension reduction vector by using a softmax layer to obtain a probability value of each intention; the softmax function may translate each element of the vector into a probability value between 0 and 1, and the sum of all probability values is 1; through this processing, probability distribution of each intention can be obtained, so that the intention of the user input corpus can be judged.
Specific examples of how the intention recognition method in the above-described embodiment makes a judgment and processes of an intra-knowledge intention and an out-of-knowledge intention are as follows:
step S1, constructing a knowledge base, and storing corpus { < me complaints, complaints >, < me service is too bad, me is very angry, complaints > … … };
step S2, mapping the corpus in the knowledge base in the step S1 into a high-dimensional semantic space based on a Roformer-smi model to obtain semantic vectors [0.99, ] of corpus 'I complaint', and semantic vectors [0.97,.,. 0.97] of 'your service is too bad', and storing the semantic vectors into a vector database;
step S3, training an intention classification model based on corpus { < me complaint, complaint >, < your service is too bad, me is very angry, complaint > … … };
step S4, if the user input corpus I complaint is received, mapping the user input corpus I complaint into a high-dimensional semantic space based on a Roformer-smi model to obtain a semantic vector [ 0.98..0.98 ] of the corpus I complaint;
s5, calculating that cosine similarity of semantic vectors of the user input corpus and the semantic vectors [0.99, & gt, 0.99] and [0.97, & gt, 0.97] in a vector database are respectively 0.99 and 0.99, wherein cosine distances are 0.01 and 0.01, a preset threshold value is 0.1, meanwhile, the number of neighbor relations between the semantic vectors of the user input corpus and the semantic vectors in the vector database is preset to be more than one, at the moment, the two cosine distances are smaller than the threshold value, and the number of neighbor relations between the semantic vectors of the user input corpus and the semantic vectors in the vector database is 2 (more than 1), so that the condition is met, and the intention of the user input corpus is judged to belong to the intention of the knowledge domain;
step S7, based on the intention classification model obtained in the step S3, outputting a label probability value [0.01,0.02 ] of the corpus I complaint input by the user, the probability value of the complaint intent is 0.9 at maximum, and the intent of the user to input the corpus "i want to complain" is judged as complaint.
Step S4, if the user input corpus I want to learn is received, mapping the user input corpus I want to learn into a high-dimensional semantic space based on a Roformer-smi model to obtain a semantic vector [0.01, ] 0.01 of the corpus I want to learn;
s5, calculating that cosine similarity between semantic vectors of the user input corpus and the semantic vectors [0.99, & gt, 0.99] and [0.97, & gt, 0.97] in a vector database are respectively 0.01 and 0.01, wherein cosine distances are 0.99 and 0.99, a preset threshold value is 0.1, and meanwhile, the number of neighbor relations between the semantic vectors of the user input corpus and the semantic vectors in the vector database is preset to be more than one, at the moment, the two cosine distances are larger than the threshold value, and the number of neighbor relations between the semantic vectors of the user input corpus and the semantic vectors in the vector database is 0, so that the condition is not satisfied, and the intention of the user input corpus is judged to belong to the intention outside the knowledge domain;
and S6, stopping entering the intention classification model to identify, outputting the intention outside the knowledge domain, starting the intention processing flow outside the knowledge domain, storing the user input corpus, and updating the knowledge base and the vector database when the stored user input corpus meets a specific condition.
The invention also provides a specific implementation method of the intention recognition system based on knowledge domain detection, which comprises a knowledge domain detection judging module, a corpus acquisition module, a knowledge storage module, a vector data processing module, a vector data storage module, an intention recognition module and a knowledge domain external intention storage module.
The corpus acquisition module is used for acquiring user input corpus and sending the acquired user input corpus to the vector data processing module; the knowledge storage module is used for storing the corpus of the knowledge in the field and sending the stored corpus of the knowledge in the field to the vector data processing module and the intention recognition module.
The vector data processing module receives the corpus of the knowledge in the field sent by the knowledge storage module and the user input corpus collected by the corpus collection module, maps the corpus of the knowledge in the field or the user input corpus to a high-dimensional semantic space based on the semantic embedding model to obtain a semantic vector of the corpus of the knowledge in the field or a semantic vector of the user input corpus, wherein the semantic vector of the corpus of the knowledge in the field is stored in the vector data storage module, and the semantic vector of the corpus of the user input corpus is sent to the knowledge field detection judgment module.
The vector data storage module is used for receiving and storing semantic vectors of the corpus of knowledge in the field; the knowledge domain detection judging module is used for receiving semantic vectors of the user input corpus, comparing the semantic vectors of the user input corpus with semantic vectors of the intra-domain knowledge corpus in the vector data storage module, calculating a neighbor relation between the semantic vectors of the user input corpus and the semantic vectors of the intra-domain knowledge corpus in the vector data storage module, and judging whether the user input corpus belongs to the intra-domain intention or the extra-domain intention based on the neighbor relation; if the intention is the intention outside the knowledge domain, the user input corpus is sent to the intention storage module outside the knowledge domain, and if the intention is the intention inside the knowledge domain, the user input corpus is sent to the intention recognition module.
The intention recognition module is used for receiving the corpus of the knowledge in the field sent by the knowledge storage module so as to train the intention classification model, and simultaneously, is used for receiving the user input corpus which is sent by the knowledge field detection judgment module and is judged to be the intention in the knowledge field, inputting the user input corpus into the trained intention classification model, and judging the intention based on the score.
The knowledge domain external intention storage module is used for receiving and storing the user input corpus which is sent by the knowledge domain detection and judgment module and judged to be the knowledge domain external intention.
The invention also provides a concrete implementation mode of the computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the intention recognition method based on the knowledge domain detection when executing the computer program. The computer device may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack-mounted server, a blade server, a tower server, or a rack-mounted server (including a stand-alone server, or a server cluster composed of multiple servers) that executes a program, or the like. The computer device of the present embodiment includes at least, but is not limited to: a memory, a processor, and the like, which may be communicatively coupled to each other via a system bus.
In this embodiment, the memory (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of a computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD), a Flash Card (Flash Card), etc. that are provided on the computer device. Of course, the memory may also include both internal storage units of the computer device and external storage devices. In this embodiment, the memory is typically used to store an operating system and various application software installed on the computer device. In addition, the memory can be used to temporarily store various types of data that have been output or are to be output.
The processor may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to execute the program code stored in the memory or process the data to implement the intention recognition method based on knowledge domain detection in the foregoing embodiment.
According to the intention recognition method, system and computer equipment based on the knowledge domain detection, the knowledge domain detection can be firstly carried out on the input corpus of the user based on the neighbor relation before the intention classification model is entered, the intra-knowledge domain intention and the extra-knowledge domain intention are distinguished, and the intention classification is stopped when the intra-knowledge domain intention and the extra-knowledge domain intention are judged, so that the misjudgment rate of the follow-up intention is reduced, the accuracy rate of the follow-up intention recognition is improved, and the user experience is improved.
In the description of the present specification, a description referring to the terms "preferred embodiment," "further embodiment," "other embodiments," or "specific examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Although embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (4)
1. The intention recognition method based on knowledge domain detection is characterized by comprising the following steps:
s1, constructing a knowledge base;
s2, mapping the corpus in the knowledge base to a high-dimensional semantic space based on a semantic embedding model to obtain semantic vectors of the corpus in the knowledge base, and storing the semantic vectors into a vector database; the method comprises the following steps: inputting corpus in a knowledge base into a Roformer-smi model, a SimCSE model or a Sentence-BERT model in batches, encoding by a multi-layer encoder, selecting the output of the last layer encoder to form semantic vectors corresponding to the corpus, and storing key value pairs of the [ corpus: vectors ] into a vector database;
s3, training an intention classification model based on a knowledge base; the method comprises the following steps: the knowledge base is arranged to obtain { corpus: intent training data, namely, carrying out token coding, segment coding and position coding on the obtained training data in batches, then sending the training data into an embedding layer to obtain token embedded vectors, segment embedded vectors and position embedded vectors, adding the three vectors into a multi-layer converter-encoder, inputting a position vector with a [ CLS ] symbol output by a last layer encoder into a linear classification head, reducing the dimension to a label dimension, calculating loss by adopting cross entropy, and carrying out gradient feedback;
s4, receiving user input corpus, and mapping the user input corpus to a high-dimensional semantic space based on a semantic embedding model to obtain a semantic vector of the user input corpus; the method comprises the following steps: inputting the corpus input by a user into a Roformer-smi model, a SimCSE model or a Sentence-BERT model in batches, encoding by a multi-layer encoder, and selecting the last layer encoder to output to form a semantic vector corresponding to the corpus;
s5, calculating a neighbor relation between the semantic vector of the user input corpus and the semantic vector of the knowledge base, and judging whether the user input corpus belongs to the intra-knowledge domain intention or the extra-knowledge domain intention based on the neighbor relation; the method comprises the following steps: calculating cosine distances between semantic vectors of the user input corpus and all semantic vectors in the vector database, when the cosine distances are smaller than a preset threshold value, considering that the semantic vectors of the user input corpus and all semantic vectors in the vector database are adjacent, when the number of the semantic vectors of the user input corpus and all semantic vectors which are considered to be adjacent exceeds the preset number, considering that the intention of the user input corpus belongs to the intention in the knowledge domain, otherwise, the intention belongs to the intention outside the knowledge domain;
s6, stopping intention recognition if the intention belongs to the intention outside the knowledge domain, starting the intention processing flow outside the knowledge domain, storing the user input corpus, and updating the knowledge base and the vector database when the stored user input corpus meets a specific condition; the method comprises the following steps: s61, storing the user input content which is judged as the out-of-knowledge domain intention into an out-of-knowledge domain intention library, and processing by adopting a strategy of repeated inquiry, clarification inquiry and spam reply; s62, when the corpus in the knowledge domain outside-domain intent library meets a certain condition, the expert in the knowledge domain collates and generalizes the corpus in the knowledge domain outside-domain intent library, and the collated and generalized knowledge domain outside-domain intent is added into the current knowledge library to update the knowledge library; s63, mapping the corpus in the updated knowledge base to a high-dimensional semantic space based on a semantic embedding model to obtain semantic vectors of the corpus in the updated knowledge base, and storing the semantic vectors into a vector database to update the vector database;
s7, if the intention belongs to the knowledge domain, entering an intention classification model, classifying the user input content, and judging the intention based on the score; the method comprises the following steps: the method comprises the steps of inputting corpus input intent classification models which are judged to be intent in a knowledge domain by a user, firstly encoding by a multi-layer encoder, selecting hidden layer vectors at [ CLS ] head positions, inputting the hidden layer vectors into a linear layer, performing dimension reduction on the hidden layer vectors through the linear layer to obtain dimension reduction vectors, performing normalization on the dimension reduction vectors by using a sotmax layer to obtain probability values of each intent, and selecting labels with the maximum probability values as the intent of the user input corpus.
2. The method for identifying intention based on knowledge domain detection according to claim 1, wherein in step S2 or step S4, the last layer of encoder is selected for outputting through multi-layer encoder, so as to form semantic vectors corresponding to corpus, specifically:
adopting a CLS vector method, namely selecting a starting mark [ CLS ] vector as a semantic vector of the corpus;
or adopting a mean value pooling method, namely carrying out mean value calculation on word vectors of all words in the corpus, and finally taking the mean value vectors as semantic vectors of the corpus;
or adopting a maximum value pooling method, namely carrying out maximum value solving operation on word vectors of all words in the corpus, and finally taking the maximum value vector as a semantic vector of the corpus.
3. An intention recognition system for implementing the knowledge domain detection-based intention recognition method of any one of claims 1-2, characterized in that: the system comprises a knowledge domain detection and judgment module, a corpus acquisition module, a knowledge storage module, a vector data processing module, a vector data storage module, an intention recognition module and a knowledge domain external intention storage module;
the corpus acquisition module is used for acquiring user input corpus and sending the acquired user input corpus to the vector data processing module;
the knowledge storage module is used for storing the corpus of the knowledge in the field and sending the stored corpus of the knowledge in the field to the vector data processing module and the intention recognition module;
the vector data processing module receives the linguistic data of the knowledge in the field sent by the knowledge storage module and the user input linguistic data collected by the linguistic data collection module, maps the linguistic data of the knowledge in the field or the user input linguistic data to a high-dimensional semantic space based on the semantic embedding model to obtain a semantic vector of the linguistic data of the knowledge in the field or a semantic vector of the user input linguistic data, wherein the semantic vector of the linguistic data of the knowledge in the field is stored in the vector data storage module, and the semantic vector of the user input linguistic data is sent to the knowledge field detection judgment module;
the vector data storage module is used for receiving and storing semantic vectors of the corpus of knowledge in the field;
the knowledge domain detection judging module is used for receiving semantic vectors of the user input corpus, comparing the semantic vectors of the user input corpus with semantic vectors of the intra-domain knowledge corpus in the vector data storage module, calculating a neighbor relation between the semantic vectors of the user input corpus and the semantic vectors of the intra-domain knowledge corpus in the vector data storage module, and judging whether the user input corpus belongs to the intra-domain intention or the extra-domain intention based on the neighbor relation; if the intention is the intention outside the knowledge domain, the input corpus of the user is sent to the intention storage module outside the knowledge domain, and if the intention is the intention inside the knowledge domain, the input corpus of the user is sent to the intention recognition module;
the intention recognition module is used for receiving the corpus of the knowledge in the field sent by the knowledge storage module so as to train the intention classification model, and simultaneously, is used for receiving the user input corpus which is sent by the knowledge domain detection judgment module and is judged to be the intention in the knowledge domain, inputting the user input corpus into the trained intention classification model, and judging the intention based on the score;
the knowledge domain external intention storage module is used for receiving and storing the user input corpus which is sent by the knowledge domain detection and judgment module and judged to be the knowledge domain external intention.
4. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the knowledge domain detection based intent recognition method as claimed in any one of claims 1-2 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311515015.7A CN117235629B (en) | 2023-11-15 | 2023-11-15 | Intention recognition method, system and computer equipment based on knowledge domain detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311515015.7A CN117235629B (en) | 2023-11-15 | 2023-11-15 | Intention recognition method, system and computer equipment based on knowledge domain detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117235629A CN117235629A (en) | 2023-12-15 |
| CN117235629B true CN117235629B (en) | 2024-04-12 |
Family
ID=89084659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311515015.7A Active CN117235629B (en) | 2023-11-15 | 2023-11-15 | Intention recognition method, system and computer equipment based on knowledge domain detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117235629B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020163627A1 (en) * | 2019-02-07 | 2020-08-13 | Clinc, Inc. | Systems and methods for machine learning-based multi-intent segmentation and classification |
| CN111625641A (en) * | 2020-07-30 | 2020-09-04 | 浙江大学 | Dialog intention recognition method and system based on multi-dimensional semantic interaction representation model |
| CN114077666A (en) * | 2020-08-21 | 2022-02-22 | 丰田自动车株式会社 | Dialog intention classification method, apparatus and non-volatile computer storage medium |
| CN114528844A (en) * | 2022-01-14 | 2022-05-24 | 中国平安人寿保险股份有限公司 | Intention recognition method and device, computer equipment and storage medium |
| CN114564964A (en) * | 2022-02-24 | 2022-05-31 | 杭州中软安人网络通信股份有限公司 | Unknown intention detection method based on k-nearest neighbor comparison learning |
| CN114925163A (en) * | 2022-04-29 | 2022-08-19 | 海信电子科技(武汉)有限公司 | Intelligent equipment and intention recognition model training method |
| CN115098681A (en) * | 2022-06-29 | 2022-09-23 | 浙江工业大学 | Open service intention detection method based on supervised contrast learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9870356B2 (en) * | 2014-02-13 | 2018-01-16 | Microsoft Technology Licensing, Llc | Techniques for inferring the unknown intents of linguistic items |
| US12019984B2 (en) * | 2021-09-20 | 2024-06-25 | Salesforce, Inc. | Multi-lingual intent model with out-of-domain detection |
-
2023
- 2023-11-15 CN CN202311515015.7A patent/CN117235629B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020163627A1 (en) * | 2019-02-07 | 2020-08-13 | Clinc, Inc. | Systems and methods for machine learning-based multi-intent segmentation and classification |
| CN111625641A (en) * | 2020-07-30 | 2020-09-04 | 浙江大学 | Dialog intention recognition method and system based on multi-dimensional semantic interaction representation model |
| CN114077666A (en) * | 2020-08-21 | 2022-02-22 | 丰田自动车株式会社 | Dialog intention classification method, apparatus and non-volatile computer storage medium |
| CN114528844A (en) * | 2022-01-14 | 2022-05-24 | 中国平安人寿保险股份有限公司 | Intention recognition method and device, computer equipment and storage medium |
| CN114564964A (en) * | 2022-02-24 | 2022-05-31 | 杭州中软安人网络通信股份有限公司 | Unknown intention detection method based on k-nearest neighbor comparison learning |
| CN114925163A (en) * | 2022-04-29 | 2022-08-19 | 海信电子科技(武汉)有限公司 | Intelligent equipment and intention recognition model training method |
| CN115098681A (en) * | 2022-06-29 | 2022-09-23 | 浙江工业大学 | Open service intention detection method based on supervised contrast learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于自适应神经网络模糊***(ANFIS)的空中目标意图分类;贾苏元;徐金钰;王钰;;电子测量技术;20161215(第12期);第62-66页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117235629A (en) | 2023-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108304372B (en) | Entity extraction method and device, computer equipment and storage medium | |
| CN112084337A (en) | Training method of text classification model, and text classification method and equipment | |
| CN112800170A (en) | Question matching method and device and question reply method and device | |
| CN113392209B (en) | Text clustering method based on artificial intelligence, related equipment and storage medium | |
| CN111221936B (en) | Information matching method and device, electronic equipment and storage medium | |
| CN114647741A (en) | Process automatic decision and reasoning method, device, computer equipment and storage medium | |
| CN112035645A (en) | Data query method and system | |
| CN116089873A (en) | Model training method, data classification and classification method, device, equipment and medium | |
| CN114780746A (en) | Knowledge graph-based document retrieval method and related equipment thereof | |
| CN116991875B (en) | SQL sentence generation and alias mapping method and device based on big model | |
| CN112446209A (en) | Method, equipment and device for setting intention label and storage medium | |
| CN116416480A (en) | Visual classification method and device based on multi-template prompt learning | |
| CN114691866A (en) | Multilevel label-oriented text classification method, device, equipment and storage medium | |
| CN115730597A (en) | Multi-level semantic intention recognition method and related equipment thereof | |
| CN117807482B (en) | Method, device, equipment and storage medium for classifying customs clearance notes | |
| CN113837307A (en) | Data similarity calculation method and device, readable medium and electronic equipment | |
| CN117828024A (en) | Plug-in retrieval method, device, storage medium and equipment | |
| CN110633468B (en) | Information processing method and device for object feature extraction | |
| CN117235629B (en) | Intention recognition method, system and computer equipment based on knowledge domain detection | |
| CN113177164B (en) | Multi-platform collaborative new media content monitoring and management system based on big data | |
| CN114513578A (en) | Outbound method, device, computer equipment and storage medium | |
| CN114254622A (en) | Intention identification method and device | |
| CN113469237A (en) | User intention identification method and device, electronic equipment and storage medium | |
| CN112784033B (en) | Aging grade identification model training and application method and electronic equipment | |
| CN117875424B (en) | Knowledge graph completion method and system based on entity description and symmetry relation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |