CN115168564A

CN115168564A - Dialogue mining method and device, electronic equipment and medium

Info

Publication number: CN115168564A
Application number: CN202211086696.5A
Authority: CN
Inventors: 周娟
Original assignee: Ping An Bank Co Ltd
Current assignee: Ping An Bank Co Ltd
Priority date: 2022-09-07
Filing date: 2022-09-07
Publication date: 2022-10-11

Abstract

The application provides a conversation mining method, a conversation mining device, electronic equipment and a medium, wherein the method is used for acquiring a conversation text in a conversation between a client side and a server side in the field of target business, and the conversation text comprises a client text and a reply text; extracting a plurality of effective question features of the client text and a plurality of effective answer features of the answer text from a plurality of dimensions aiming at the conversation text; judging whether the valid question condition and the valid answer condition are met or not according to the valid question features and the valid answer features; when the client text is judged to accord with the effective question conditions and the reply text is judged to accord with the effective answer conditions, the conversation text is determined to be the target conversation, the target conversation is added to a question-answer database in the target business field, so that high-quality question answers are extracted from historical conversation data of the client side and the server side, and the high-quality question answers are pushed to seats receiving the same questions as references.

Description

Dialogue mining method and device, electronic equipment and medium

Technical Field

The present application relates to the field of natural language processing technologies, and in particular, to a dialog mining method, apparatus, electronic device, and medium.

Background

In a human-human interaction scene, the question-answering capability of the service side agent directly determines the fluency of the whole conversation and the satisfaction degree of customers. Professional seats can memorize business rules and freely answer and solve various business questions of customers under the guidance of own experiences, but the professional seats are difficult to enter the seats. For the current situation of the uneven service level, the agent can search the keywords of the customer questions to find answers from the service rules. However, this method affects the instantaneity of the seat response, and meanwhile, the business rules cannot be directly used as answers, and not high-quality answers, only limited reference opinions can be given to the seat.

Disclosure of Invention

In view of the above, an object of the present application is to provide a dialog mining method, apparatus, electronic device and medium, which can mine high-quality questions and answers from historical session data of a client and a server, and push the high-quality questions and answers as a reference to an agent receiving the same questions.

The conversation mining method provided by the embodiment of the application comprises the following steps:

obtaining a dialog text in a conversation between a client side and a server side in a target service field, wherein the dialog text comprises a client text and a reply text;

extracting, for the dialog text, a plurality of question features of the client text and a plurality of answer features of the answer text from a plurality of non-homogenous dimensions;

judging whether the client text meets the effective question condition or not according to the effective question characteristics, and judging whether the reply text meets the effective answer condition or not according to the effective answer characteristics;

and when the client text is judged to accord with the effective question condition and the reply text is judged to accord with the effective answer condition, determining the conversation text as a target conversation, and adding the target conversation to a question-answer database in the target business field.

In some embodiments, in the dialog mining method, the plurality of question features includes: text type characteristics, text length characteristics, whether the grammar of the client text is complete or not, and client text vector characteristics; the text type feature represents whether the client text has a question intention;

the plurality of valid answer features comprises: the answer text vector feature, the answer text grammar complete feature, the answer and question business vocabulary belong to the same business scope feature, the answer and question keyword repetition feature and the guide term intensity feature.

In some embodiments, in the dialog mining method, the text type feature is extracted by:

judging whether the client text is a question sentence according to a preset judgment rule, and if so, determining that the text type feature is a question type feature;

if not, the client text is input into the trained intention recognition model so as to recognize whether the client text has a question intention or not and determine the text type characteristic.

In some embodiments, in the dialog mining method, the client text vector feature and the reply text feature vector are extracted by:

according to a first preprocessing rule, preprocessing the client text and the reply text, and reserving target words of the client text and the reply text;

determining the word frequency of each target vocabulary in the customer text based on a service dictionary of the target service field to obtain a customer text feature vector; and determining the word frequency of each target word in the reply text to obtain a feature vector of the reply text.

In some embodiments, the dialog mining method, determining whether the client text meets the question validity condition according to the question validity characteristics and determining whether the answer text meets the answer validity condition according to the answer validity characteristics, includes:

inputting the plurality of effective question features into a trained effective question recognition model, and judging whether the client text meets an effective question condition according to an output result of the effective question recognition model; different effective question features have different weight indexes in the effective question identification model;

inputting the plurality of effective answer features into a trained effective answer recognition model, and judging whether the answer text meets an effective answer condition according to an output result of the effective answer recognition model; different effective answer characteristics have different weight indexes in the effective answer identification model;

the effective question recognition model and the effective answer recognition model are obtained by adopting an XGboost model.

In some embodiments, after the dialog mining method adds the target dialog to the question-answer database of the target business domain, the method further includes:

in response to receiving a customer question of a target service field, determining at least one target customer text with the intention similarity meeting a preset similarity condition with the customer question from a target dialogue stored in the question-answer database;

and determining a target question answer corresponding to the target customer text, and recommending the target customer text and the target answer text to a conversation interface of a service party.

In some embodiments, after the target dialog is added to the question and answer database of the target business domain, the method further includes:

and in response to receiving the evaluation operation of the server on the target conversation, updating the quality score of the target conversation in the question-answer database so as to determine the priority of the target conversation recommended for the customer question according to the quality score of the target conversation.

In some embodiments, there is also provided a conversation mining apparatus, the apparatus comprising:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a dialog text in a conversation between a client side and a server side in a target service field, and the dialog text comprises a client text and a reply text;

the extraction module is used for extracting a plurality of question features of the client text and a plurality of answer features of the answer text from a plurality of non-homogeneous dimensions aiming at the dialog text;

the judging module is used for judging whether the client text meets the effective question conditions or not according to the effective question features and judging whether the reply text meets the effective answer conditions or not according to the effective answer features;

and the determining module is used for determining the dialog text as the target dialog and adding the target dialog to a question-answer database in the target business field when the client text is judged to accord with the effective question condition and the reply text is judged to accord with the effective answer condition.

In some embodiments, there is also provided an electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is operating, the machine-readable instructions being executable by the processor to perform the steps of the dialog mining method.

In some embodiments, a computer-readable storage medium is also provided, having stored thereon a computer program which, when executed by a processor, performs the steps of the dialog mining method described.

The embodiment of the application provides a conversation mining method, a device, electronic equipment and a medium, wherein a high-quality conversation text is mined from conversation texts in conversation between a client and a server in a target business field, the high-quality conversation text is added to a question-answer database in the target business field, when a question belonging to the target business field is received from the client, the intention of the question is directly identified, similar questions are identified from the question-answer database, the identified similar questions and high-quality answers of the similar questions are pushed to a conversation interface of a seat, the seat does not need to extract keywords and conduct retrieval operation by itself, and the pushed high-quality answers not only contain business rules but also contain high-quality expressions after excellent seats are found, so that the difficulty of the self-organization language of the seat is reduced, the timeliness and the answering efficiency of the seat are improved, and the answering quality of the seat is also improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

FIG. 1 is a flow chart illustrating a method of a dialog mining method according to an embodiment of the present application;

FIG. 2 is a flowchart illustrating a method for extracting text type features according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating a method for extracting a client text vector feature and a reply text feature vector according to an embodiment of the present application;

FIG. 4 is a flow chart of a method of another dialog mining method according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a dialog mining device according to an embodiment of the present application;

fig. 6 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. In addition, one skilled in the art, under the guidance of the present disclosure, may add one or more other operations to the flowchart, or may remove one or more operations from the flowchart.

In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.

In a human-human interaction scene, the question and answer capability of a service side agent directly determines the fluency of the whole conversation and the satisfaction degree of a client. Professional seats can memorize business rules and freely answer and solve various business questions of customers under the guidance of own experiences, but the professional seats are difficult to enter the seats. For the current situation of the uneven service level, the agent can search the keywords of the customer questions to find answers from the service rules.

However, in this method, after receiving a customer question, the terminal device of the agent needs to determine a keyword in the question, and then search according to the keyword to obtain a relevant business rule, so that the agent with little experience cannot extract the keyword accurately, and the steps will take time, which affects the instantaneity of response of the agent.

Meanwhile, after the seat retrieves the relevant business rules, the hard business rules cannot be directly used as answers, and certain processing is needed to be carried out to convert the hard business rules into more emotional answers, that is, the retrieved business rules only can give limited reference opinions to the seat, and a new seat with limited experience not only needs to know the business rules but also needs communication skills.

Here, the agent may be a person who is responsible for answering a question from a client, such as a customer service, a client manager, or a consultant.

Based on this, the embodiment of the present application provides a dialog mining method, which excavates a high-quality dialog text from dialog texts in a session between a client and a server in a target business field, adds the high-quality dialog text to a question and answer database in the target business field, directly identifies an intention of a question when receiving a question belonging to the target business field from the client, identifies similar questions from the question and answer database, and pushes the identified similar questions and high-quality answers to the conversation interface of a seat, without the need of the seat to extract keywords and perform retrieval operations by itself, and the pushed high-quality answers include not only business rules but also high-quality expressions after excellent seats are found, thereby reducing the difficulty of the seat organizing languages by itself, improving the timeliness and the response efficiency of the seat, and improving the response quality of the seat.

Referring to fig. 1, fig. 1 shows a flowchart of a method of a dialog mining method according to an embodiment of the present application, and specifically, the dialog mining method includes the following steps S101 to S104:

s101, obtaining a dialog text in a conversation between a client and a server in the field of target services, wherein the dialog text comprises a client text and a reply text;

s102, aiming at the dialog text, extracting a plurality of effective question features of the client text from a plurality of non-homogeneous dimensions, and extracting a plurality of effective answer features of the answer text;

s103, judging whether the client text meets the effective question condition or not according to the effective question characteristics, and judging whether the reply text meets the effective answer condition or not according to the effective answer characteristics;

s104, when the client text is judged to accord with the effective question condition and the reply text accords with the effective answer condition, determining the conversation text as the target conversation, and adding the target conversation to a question-answer database in the target business field.

In the embodiment of the application, the dialogue mining method can be operated on terminal equipment or a server; when the session mining method is executed on a server, the entity location method may be implemented and executed based on a cloud interaction system, where the cloud interaction system at least includes the server and a client device (i.e., the terminal device).

Specifically, taking the application to a server as an example, when the entity location method is run on the server, the dialogue mining method is used for screening out high-quality dialogue texts with quality meeting requirements from historical conversation data.

In step S101, the obtaining of the dialog text in the session between the client and the server in the target service domain includes:

and acquiring dialogue data in the target business field, and cleaning the dialogue data to obtain a dialogue text.

And cleaning the dialogue data by using methods such as regular method, dialogue block splicing and the like to clean the literal noise of the text.

The literal noise, such as some cluttered strings, repeated words, etc. For example, "when shijian is asked to arrive at the customer text, such as the account boisyanrfq #", shijian may be a stroke error in the customer input, and the character string boisyanrfq # may be an unrecognizable emoticon.

The literal noise affects the feature extraction effect of the dialog text, for example, affects the semantics of the judgment text, so that it is impossible to accurately judge whether the dialog text belongs to a high-quality dialog, and therefore, the dialog data needs to be cleaned.

The target business fields, such as the insurance sales field, the automobile recommendation field, the financial service field and the like, have different business rules corresponding to each target business field. A company may have multiple target business domains and develop different business rules for each domain, including both insurance sales domains and financial services domains, for example.

Here, when a preset acquisition condition is satisfied, historical dialogue data in a session between a client and a server in the acquisition target business field is acquired from the historical dialogue data.

The preset acquisition condition comprises: and when the preset acquisition time is reached, receiving a conversation acquisition signal and the like.

The preset acquisition duration is reached, namely, the high-quality dialogue text is screened out from the historical conversation data generated in the time period at intervals, so that the question and answer database in the target business field is continuously updated, the data in the question and answer database is richer and richer, and the requirements of customer service managers can be met more and more.

In step S102, a plurality of valid question features of the client text are extracted from a plurality of non-homogeneous dimensions, that is, the extracted valid question features are non-homogeneous features.

The plurality of question features includes: text type characteristics, text length characteristics, whether the grammar of the client text is complete or not, and client text vector characteristics; the text type feature characterizes whether the customer text has a questionable intent.

Whether questions are asked or not, which may constitute a question, is really a core feature in constructing features of the customer's text. In the embodiment of the present application, if the customer text has a query intention, the text type feature is 1, whereas if the customer text has no query intention, the text type is 0.

In natural language, it is difficult to identify whether or not a query is intended in text. Some of the texts sent by the client belong to obvious question sentences, and some are implicit questions. For example, a customer who wants to consult the rules for a reserve fund to bring up a savings card may ask questions directly, the customer text being: asking how to withdraw the reserve money to the savings card; it is also possible to state, for example: my spare money wants to cash out to the savings card.

For the embodiment of the application, the client text contains the query intention and can be used as high-quality dialog text, so that the text type feature is a core feature in the question-asking feature.

Based on this, in the embodiment of the present application, please refer to fig. 2, and fig. 2 shows a flowchart of a method for extracting text type features in the embodiment of the present application; specifically, the text type feature is extracted by the following method:

s201, judging whether the client text is a question sentence or not according to a preset judgment rule, and if so, determining that the text type feature is a question type feature;

s202, if not, inputting the client text into the trained intention recognition model to recognize whether the client text has a question intention or not and determine text type characteristics.

Here, in the application, firstly, it is determined through some regular expression matching that whether the client text is an obvious question sentence, for example, a client text such as "ask how to bring a reserve fund to a savings card" is determined once by the user, and the text type feature can be directly determined as a question type feature, and the text type feature is determined as 1. For the implicit question intention of 'my spare money wants to bring up a savings card', a judgment cannot be concluded, and text type features are determined through a trained intention recognition model.

Compared with regular expression matching, the calculation process of the intention recognition model is more complex, more computing resources are occupied, when conversation mining is actually carried out, a large number of conversations need to be processed, obvious questions can be screened out from the large number of conversations through primary judgment, only the unobvious questions are subjected to secondary recognition, and therefore the calculation amount of the intention recognition model is reduced, the feature extraction precision of text type features is considered, and the computing resources are saved.

The text length features extracted in the embodiment of the application represent the character number features of the client text.

In some embodiments, the text length feature directly characterizes the number of words, using the number of words as the text length feature, such as "my reserve wants to cash out to a savings card" and the text length feature is 12.

In some embodiments, the text length feature characterizes whether the number of words meets a preset length condition; here, the text length characteristic is represented by 0 or 1, and when the number of words of the client text is less than or equal to a preset maximum number and is greater than or equal to a preset minimum number, the text length characteristic is 1; otherwise, if the number of words in the client text is greater than the preset maximum number or less than the preset minimum number, the text length characteristic is 0.

The embodiment of the application aims to dig out high-quality conversations from a large number of conversations, the high-quality conversations must contain enough abundant information to clearly express the business problem to be consulted, and therefore the number of words of a text must be greater than or equal to a preset minimum number; meanwhile, high-quality conversation also requires that a customer ask a question more succinctly and not much useless information, for example, when some people consult customer service problems, some internet barriers are used, such as "i am complaining about what you are doing again", i am only a worker, and you can not decide what, i am just to bring a spare money to a deposit card and strike, i am only one set of words, i am out of nothing, i can not understand what you are, and i can avoid my topics lightly, i just feel that i is troublesome, i am just to catch up with me ending topics, i am out of everything. Such user questions are not only used as high quality dialogs for agent reference.

Therefore, the method and the device select the user text with the proper length by limiting the length of the user text, so that the mined high-quality dialog not only contains enough rich information, but also can express the business problem of consulting concisely and clearly.

Whether the client text grammar has complete characteristics is completed through sentence component analysis of traditional linguistics in the embodiment of the application, the missing sentence becomes 0, and the complete sentence becomes 1.

For the user problems of subject missing, object missing and the like which do not conform to the syntactic structure tree, the user problems are selected to be discarded in the embodiment of the application, because the information of the user problems is scattered, and the user problems can clearly express the purpose of consulting by combining the context information and do not belong to high-quality conversations which can give seat reference and know guidance.

Because the characteristics of the constructed client text and the characteristics of the reply text belong to natural language processing, and the aim is to screen out high-quality conversations, the task of constructing the characteristics of the client text and the task of constructing the characteristics of the reply text are overlapped. The client text vector feature and the reply text feature vector are constructed in the same way. Whether the client text grammar is complete in characteristic and whether the reply text grammar is complete in characteristic are similar in meaning and are also constructed through the same method.

Referring to fig. 3 for the client text vector feature and the reply text feature vector, fig. 3 is a flowchart illustrating a method for extracting a client text vector feature and a reply text feature vector according to an embodiment of the present application; specifically, the method for extracting the feature vector of the client text and the feature vector of the reply text comprises the following steps S301-S302;

s301, preprocessing the client text and the reply text according to a first preprocessing rule, and reserving target words of the client text and the reply text;

s302, determining the word frequency of each target vocabulary in the customer text based on a service dictionary of the target service field to obtain a customer text feature vector; and determining the word frequency of each target word in the reply text to obtain a feature vector of the reply text.

The business dictionary of the target business field comprises business vocabularies of the target business field and the weight of each business vocabulary. Different target business fields correspond to different business dictionaries.

The service vocabulary in the service dictionary is obtained from the conversation high-frequency vocabulary by a manual labeling method; the weight of the traffic vocabulary may be derived by calculating the TF-IDF value of the vocabulary in the session.

For the client text and the reply text, preprocessing the client text and the reply text according to a first preprocessing rule, wherein the preprocessing rule mainly comprises word segmentation and part-of-speech tagging; and then filtering the stop words of the obtained result, and reserving vocabularies with specified parts of speech, such as common nouns, bank-specific named entities, verbs, name verbs and the like.

After preprocessing, obtaining a client text characteristic vector S or a reply text characteristic vector S based on a service dictionary;

，

reserved for customer text or answer text and belonging to the purpose of service dictionaryWord frequency of the vocabulary; if the ith word is not in the sentence according to the dictionary order, then

Is 0.

For example, if the service dictionary of the target service domain is 1000-dimensional, then

(ii) a If the third service vocabulary in the service dictionary does not appear in the client text, in the feature vector S of the client text,

is 0.

For the complete feature of the reply text grammar, the same is true, in the embodiment of the application, the sentence component analysis of the traditional linguistics is completed, the missing sentence becomes 0, and the complete sentence becomes 1.

For the reply texts with the missing subjects, the missing objects and the like which do not conform to the syntactic structure tree, the reply texts are selected to be discarded in the embodiment of the application, and the reply texts do not belong to high-quality replies which can give seat reference and know instructions.

However, the reply text and the client text have different characteristics in question and answer, and therefore, when the characteristics of the reply text are constructed, it is also necessary to construct some characteristics different from those of the client text based on the characteristics of the reply text. In the embodiment of the application, whether the service vocabularies in the answers and the questions belong to the same service range characteristic, the keyword repetition degree characteristic and the guiding term intensity characteristic in the answers and the questions are all characteristics with different dimensions from the text characteristic of the client.

When a client proposes a question, different customer services of a service party often make different responses, and the problem is guaranteed to be really solved by the responses to screen high-quality conversations, so that the problems that a chicken speaks with a duck, answers are not asked, or the problems are not answered from a professional perspective in some low-quality questions and answers are avoided.

Based on the method, whether the service vocabularies in the answer and the question belong to the same service range characteristic is constructed aiming at the answer text so as to ensure that the answer text solves the customer question.

Here, the service scope is a smaller scope in a target service domain, and a target service domain includes a plurality of service scopes; for example: the financial service field includes: deposit business scope, withdraw business scope, fund purchase business scope, fund contract business scope, and the like.

In some embodiments, the service scope is specifically a service vocabulary group of the target service domain; for example, account, redemption, purchase and fund are a service vocabulary group and belong to the fund purchase service range. If the customer question contains "account to go" and the response text contains "redemption", then the response text is said to have a high probability of effectively responding to the customer question. On the other hand, if the customer question contains "account to be paid" and the reply text does not contain any vocabulary in "redemption, purchase, fund and account to be paid", the reply text may not effectively reply to the customer question.

Furthermore, from a business perspective, it is desirable to include professional business vocabulary in the response, so as to give a little-experienced agent some guidance, and therefore, in the embodiment of the present application, whether the business vocabulary in constructing the response and the question belongs to the same business scope feature is constructed.

In some embodiments, if the business vocabulary in the response and question belong to the same business scope, e.g., the customer question contains "ledger" and the response text contains "redemption," then whether the business vocabulary in the response and question belong to the same business scope is characterized as 1; on the contrary, if the service vocabulary in the answer and the question do not belong to the same service range, the characteristic of whether the service vocabulary in the answer and the question belong to the same service range is 0.

In natural language, a question and its answer are semantically similar in some appropriate way, and in the embodiment of the present application, how many keywords of the customer question are included in the answer is determined, and the answer and the keyword repetition degree of the question are taken as an important feature.

Here, the keyword repetition degree may be the number of words in the reply text and the customer text.

In some embodiments, the answer text and the customer text may be filtered first, and stop words that are extremely common and rarely express text information separately are filtered, for example, words such as "like", and the like that are not helpful to understanding semantics; and the number of the same words in the reply text and the reserved text in the client text is used as the keyword repetition degree.

Here, the similarity between two texts can be calculated through semantic recognition in the prior art, but for the present application, the customer question text and the answer text are essentially texts in two different scenarios, and the similarity between the two texts in a high-quality dialog is not necessarily high, and may even be low, for example: the question "will be shrinked in this piece of clothing", and the answer text may be "this piece of clothing was predicted to shrink, so this will not happen, please feel relieved". If the semantics of two texts are analyzed, the similarity is low. But the presence of "clothes" or "shrink" in the answer text indicates that the answer did answer the question.

Therefore, the semantic similarity between the question and the answer thereof is judged through the keyword repetition degree, and the semantic similarity is more suitable to be used as a reference for high-quality conversation.

In the customer service question-answering, the answer of a question may appear in many different answer forms, and the information amount and the specific information contained in each answer are different, so that the question-answering containing more knowledge and operation rules may have more guiding significance from the perspective of business expectation.

Based on the above, in the embodiment of the application, the guidance term intensity feature is constructed to be used for representing the richness of knowledge and operation rules contained in the answer.

Specifically, the density of the guide terms is characterized by the number of guide terms. The greater the number of guide terms, the higher the answer quality, and the fewer the number of guide terms, the lower the answer quality.

In step S103, determining whether the client text meets a valid question condition according to the multiple valid question features, and determining whether the answer text meets a valid answer condition according to the multiple valid answer features includes:

inputting the plurality of effective question features into a trained effective question recognition model, and judging whether the client text meets an effective question condition according to an output result of the effective question recognition model; different question features have different weight indexes in the question identification model;

inputting the multiple effective answer characteristics to a trained effective answer recognition model, and judging whether the answer text meets an effective answer condition or not according to an output result of the effective answer recognition model; different effective answer features have different weight indexes in the effective answer identification model;

The effective question recognition model and the effective answer recognition model which are constructed based on the XGboost model have good learning effect on non-homogeneous characteristics.

The XGboost in machine learning is named eXtreme Gradient Boosting, is an optimized distributed Gradient Boosting algorithm, and can directly support decision trees compared with GBDT on weak learner model selection of the algorithm, and can also directly support a plurality of other weak learners. In addition to the loss itself, a regularization component is added to the loss function of the algorithm. In the optimization mode of the algorithm, the loss function of the GBDT only performs negative gradient (first-order Taylor) expansion on the error part, and the XGboost loss function performs second-order Taylor expansion on the error part, so that the algorithm is more accurate. In the model training process, a grid searching method is used for parameter searching.

For the valid question recognition model, the weight indexes of different valid question features in the valid question recognition model are different, that is, the importance degrees of the parameters corresponding to different valid question features in the training process are different, and the influence degrees of different valid question features on the valid question recognition model are different.

Similarly, for the effective answer recognition models, the weight indexes of different effective answer features in the effective answer recognition models are different, that is, the importance degrees of the parameters corresponding to different effective answer features in the training process are different, and the influence degrees of different effective answer features on the effective answer recognition models are different.

In the embodiment of the present application, the importance degrees of the valid question features are ranked from low to high as follows: text type feature, text length feature, whether the grammar of the client text is complete feature, and client text vector feature.

The significance of the valid answer features is ranked from low to high as follows: whether the service vocabularies in the answers and the questions belong to the same service range characteristic, the keyword repetition degree characteristic in the answers and the questions, the complete characteristic of the answer text grammar, the answer text vector characteristic and the guide term intensity characteristic.

Referring to fig. 4, fig. 4 is a flowchart illustrating a method of another dialog mining method according to an embodiment of the present application; in the embodiment of the present application, after the target dialog is added to the question-answer database of the target business field, the method further includes the following steps S401-S402;

s401, responding to a received customer question in a target service field, and determining at least one target customer text of which the intention similarity with the customer question meets a preset similarity condition from a target dialogue stored in the question-answer database;

s402, determining a target question answer corresponding to the target customer text, and recommending the target customer text and the target answer text to a conversation interface of a service party.

Determining at least one target customer text with the intention similarity meeting a preset similarity condition with the customer question from the target dialogue stored in the question-answer database; specifically, the customer questions are input into a trained answer recommendation model, and customer questions with the same semantics as the customer questions are searched from target conversations in a target service field in a question-answer database through the answer recommendation model.

After the customer questions with the same semantics are determined, the high-quality answer texts corresponding to the customer questions with the same semantics are recommended to a session interface of a service party, namely pushed to a family interaction interface of terminal equipment of a service party seat, so that the entering seat can better answer various questions of the user by referring to the high-quality answer texts.

And when the target client text and the target reply text are recommended to a session interface of a service party, sequentially displaying the target dialogues according to the quality scores of the target dialogues from high to low so that the seat preferentially refers to the target dialogues with higher quality.

Here, the quality score of the target dialog is initially determined by the output results of the question recognition model and the answer recognition model.

In an embodiment of the present application, after the target dialog is added to a question-answer database in the target business field, the method further includes:

That is, after the target dialog is pushed to the dialog interface, the quality score of the target dialog is updated according to the feedback of the service provider's seat. For example, the evaluation operation of the receiving server on the target dialog may be an evaluation operation for each target dialog evaluation control.

Illustratively, each target dialog corresponds to an evaluation control, the evaluation controls respectively display rating scores-2, -1, 0, 1 and 2, and each rating score corresponds to a selected button; the evaluation operation is a selection operation for the selection button.

The conversational interface may display a limited number of conversations, and thus, in some embodiments, a preset number of target conversations having quality scores that are ordered first are presented in the conversational interface.

When the dialogue mining method is operated on a server, the target client text and the target reply text are recommended to a conversation interface of a service party, the server sends the target client text and the target reply text to terminal equipment of the service party, and the terminal equipment displays the target client text and the target reply text in the conversation interface.

Here, the server and the terminal device of the service side may perform data transmission and interaction in a wired network/wireless network manner according to a preset communication Protocol (such as a Real Time Streaming Protocol (RTSP)) Protocol; in the data interaction process, the terminal device can receive a target client text and a target reply text sent by the server, display the received target client text and the target reply text on a session interface for reference of a client manager, receive evaluation operation of a user on the target reply text, generate an evaluation instruction, and send the evaluation instruction to the server, so that the server updates the quality score of the target reply text according to the evaluation instruction.

After the target client text and the target reply text are recommended to a session interface of a service party, the target reply text is filled in an answer input box of the session interface in response to the copying operation aiming at the target reply text, and a client manager can send the target reply text to the client through simple modification without inputting the target reply text by self, so that the reply difficulty of the client manager is further reduced, and the reply efficiency is improved.

In the embodiment of the application, the high-quality dialogue mining results are synchronized to the operation platform according to the day, the high-quality available QA passes the auditing rate to reach 76.3%, meanwhile, the original construction of 1000 effective knowledge questions and answers needs to consume almost 2 weeks of manpower, the mining result service of the model can complete the auditing of 1000Q groups of dialogs within half a day, and the time of manual extraction is greatly saved.

In some embodiments, there is also provided a conversation mining apparatus; referring to fig. 5, fig. 5 is a schematic structural diagram illustrating a dialog mining device according to an embodiment of the present application; specifically, the dialogue mining device includes:

an obtaining module 501, configured to obtain a dialog text in a session between a client and a server in a target service field, where the dialog text includes a client text and a reply text;

an extracting module 502, configured to extract, for the dialog text, a plurality of question features of the client text from a plurality of non-homogeneous dimensions, and a plurality of answer features of the answer text;

a judging module 503, configured to judge whether the client text meets the question validity condition according to the question validity characteristics, and judge whether the answer text meets the answer validity condition according to the answer validity characteristics;

and the determining module 504 is configured to determine the dialog text as a target dialog and add the target dialog to a question and answer database in the target business field when it is determined that the client text meets the valid question condition and the reply text meets the valid answer condition.

The embodiment of the application provides a conversation mining device, which is used for mining a high-quality conversation text from the conversation text in conversation between a client and a server in a target business field, adding the high-quality conversation text into a question-answer database in the target business field, directly identifying the intention of a question when the client belongs to the question in the target business field, identifying similar questions from the question-answer database, pushing the identified similar questions and high-quality answers of the similar questions to a conversation interface of a seat, and avoiding the need of the seat to extract keywords and perform retrieval operation by itself.

In some embodiments, the plurality of question features in the conversation mining apparatus comprises: text type characteristics, text length characteristics, whether the grammar of the client text is complete or not, and client text vector characteristics; the text type feature represents whether the client text has a question intention;

That is to say, the extraction module in the dialog mining device is specifically configured to:

extracting text type characteristics, text length characteristics, whether the grammar of the client text is complete or not and client text vector characteristics from the client text; the text type feature represents whether the client text has a question intention;

and extracting the vector characteristics of the reply text, the complete characteristics of the grammar of the reply text, the characteristics of the business vocabularies in the reply and the question belonging to the same business range, the keyword repetition characteristics in the answer and the question and the guide term intensity characteristics from the reply text.

In some embodiments, when the extraction module in the dialog mining device extracts the text type feature from the client text, the extraction module is specifically configured to:

In some embodiments, the extracting module in the dialog mining device extracts the client text vector feature and the reply text feature vector, and is specifically configured to:

In some embodiments, the determining module in the dialog mining device, when determining whether the client text meets the question validity condition according to the question validity characteristics and determining whether the answer text meets the answer validity condition according to the answer validity characteristics, is specifically configured to:

the effective question identification model and the effective answer identification model are obtained by adopting an XGboost model.

In some embodiments, the dialog mining device further includes:

the response module is used for responding to the received customer questions in the target business field after the target dialogue is added into the question-answer database in the target business field, and determining at least one target customer text of which the intention similarity with the customer questions meets a preset similarity condition from the target dialogue stored in the question-answer database;

In some embodiments, the dialog mining device further includes:

and the updating module is used for responding to the received evaluation operation of the server on the target conversation after the target conversation is added into a question and answer database of the target business field, updating the quality score of the target conversation in the question and answer database, and determining the priority of the target conversation recommended for the customer question according to the quality score of the target conversation.

An electronic device is further provided in the embodiments of the present application, please refer to fig. 6, where fig. 6 shows a schematic structural diagram of the electronic device in the embodiments of the present application; the electronic device 600 includes: a processor 602, a memory 601 and a bus, wherein the memory 601 stores machine-readable instructions executable by the processor 602, the processor 602 and the memory 601 communicate via the bus when the electronic device 600 is running, and the machine-readable instructions are executed by the processor 602 to perform the steps of the dialog mining method.

Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the dialog mining method.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some communication interfaces, indirect coupling or communication connection between devices or modules, and may be in an electrical, mechanical or other form.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in software functional units and sold or used as a stand-alone product, may be stored in a non-transitory computer-readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a platform server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of dialogue mining, the method comprising the steps of:

extracting, for the dialog text, a plurality of question features of the client text from a plurality of non-homogenous dimensions and a plurality of answer features of the answer text;

and when the client text is judged to accord with the effective question condition and the reply text accords with the effective answer condition, determining the dialog text as a target dialog, and adding the target dialog into a question-answer database of the target business field.

2. The dialog mining method of claim 1 wherein the plurality of question features comprises: text type characteristics, text length characteristics, whether the grammar of the client text is complete or not, and client text vector characteristics; the text type feature represents whether the client text has a question intention;

the plurality of valid answer features comprises: the method comprises the steps of answering text vector characteristics, whether answering text grammars are complete or not, whether service vocabularies in answers and questions belong to the same service range characteristics, keyword repetition characteristics in answers and questions and guide term intensity characteristics.

3. The method of claim 2, wherein the text type feature is extracted by:

if not, the client text is input to the trained intention recognition model so as to recognize whether the client text has a question intention or not and determine the text type characteristics.

4. The method of claim 2, wherein the client text vector features and reply text feature vectors are extracted by:

5. The dialog mining method of claim 1, wherein determining whether the client text meets a valid question condition based on the plurality of valid question features and determining whether the answer text meets a valid answer condition based on a plurality of valid answer features comprises:

6. The dialogue mining method according to claim 1, wherein after the target dialogue is added to a question-answer database of a target business domain, the method further comprises:

7. The dialogue mining method according to claim 1, wherein after the target dialogue is added to a question-answer database of a target business domain, the method further comprises:

8. A conversation mining apparatus, characterized in that the apparatus comprises:

the judging module is used for judging whether the client text meets the effective question condition or not according to the effective question characteristics and judging whether the reply text meets the effective answer condition or not according to the effective answer characteristics;

9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the dialog mining method of any of claims 1 to 7.

10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, is adapted to carry out the steps of the dialog mining method according to any one of claims 1 to 7.