CN112182186A

CN112182186A - Intelligent customer service operation method, device and system

Info

Publication number: CN112182186A
Application number: CN202011064957.4A
Authority: CN
Inventors: 刘炎; 覃建策; 田本真; 陈邦忠
Original assignee: Perfect World Beijing Software Technology Development Co Ltd
Current assignee: Perfect World Beijing Software Technology Development Co Ltd; Perfect World Co Ltd
Priority date: 2020-09-30
Filing date: 2020-09-30
Publication date: 2021-01-05

Abstract

The application discloses an operation method, device and system of an intelligent customer service. Wherein, the method comprises the following steps: receiving first questioning content sent by an intelligent terminal; identifying an intention identification result matched with the first question content through an intention identification model, wherein the intention identification model is obtained by adopting a plurality of groups of question contents for training, and each question content in the same group of question contents is marked to be matched with the same answer content; and returning the first reply content determined according to the intention recognition result to the intelligent terminal. The method and the device solve the technical problem that the effective answer rate of the intelligent customer service is low in the related technology.

Description

Intelligent customer service operation method, device and system

Technical Field

The application relates to the field of artificial intelligence, in particular to an operation method, device and system of intelligent customer service.

Background

In the internet era, artificial customer service is a common work in daily life, and is a bridge for communication between each enterprise and vast users, and product consultation, activity problems, after-sale maintenance, complaint suggestions and the like can not be served, but a large number of repeated or modeled problems exist in the customer service work, and occupy a large amount of time of customer service personnel, so that the efficiency is influenced. Therefore, the customer service robot slowly enters the customer service post, the enterprise is helped to reduce labor cost, and the working efficiency is improved.

At present, a customer service robot is mainly realized by adding search to a document knowledge base, user answers are given by searching keywords asked by users in a question-answer knowledge base, however, the questions of different users for the same question are different, so that most of the questions cannot be directly searched.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the application provides an operation method, device and system of an intelligent customer service, and aims to at least solve the technical problem that the effective answer rate of the intelligent customer service is low in the related technology.

According to an aspect of an embodiment of the present application, there is provided an operation method of an intelligent customer service, applied to a customer service robot, including: receiving first questioning content sent by an intelligent terminal; identifying an intention identification result matched with the first question content through an intention identification model, wherein the intention identification model is obtained by adopting a plurality of groups of question contents for training, and each question content in the same group of question contents is marked to be matched with the same answer content; and returning the first reply content determined according to the intention recognition result to the intelligent terminal.

Optionally, returning the first reply content determined according to the intention recognition result to the intelligent terminal includes: under the condition that the confidence of the intention recognition result is greater than the target threshold, returning first reply content matched with the first question content to the intelligent terminal; and under the condition that the confidence of the intention recognition result is not greater than the target threshold, searching first reply content matched with the first question content from the target knowledge base, and returning the first reply content to the intelligent terminal.

Optionally, returning the first reply content determined according to the intention recognition result to the intelligent terminal includes: acquiring a plurality of candidate reply contents matched with the intention recognition result; and selecting the first reply content with the highest matching degree with the first question content from the candidate reply contents as the first reply content and returning the first reply content to the intelligent terminal.

Optionally, the target knowledge base stores a plurality of reference questions and response content corresponding to each reference question, wherein searching for the first response content matching the first question content from the target knowledge base includes: searching candidate reference questions matched with the first question content from a target knowledge base; feeding back the candidate reference problem to the intelligent terminal; determining a target reference problem selected by the intelligent terminal in the candidate reference problems; and taking the reply content corresponding to the target reference question in the target knowledge base as the first reply content.

Optionally, before searching the target knowledge base for the first response content matching the first question content, selecting the target knowledge base from a plurality of knowledge bases for the customer service robot, wherein each of the plurality of knowledge bases stores a question in a field and a corresponding response content.

Optionally, after receiving first question content sent by the intelligent terminal, obtaining historical question content, wherein the historical question content comprises the first question content; clustering and analyzing the historical questioning contents to obtain multiple questioning topics; and supplementing the target knowledge base with response contents matched with the various question topics.

Optionally, after receiving first question content sent by the intelligent terminal, obtaining historical question content, wherein the historical question content comprises the first question content; performing word frequency statistics on the historical question content to obtain the word frequency of words in the historical question content; and supplementing the reply content associated with the target word in the target knowledge base, wherein the word frequency of the target word is greater than the word frequency of the words except the target word in the historical questioning content.

Optionally, before receiving first question content sent by the intelligent terminal, training the initial recognition model by using data in a first training set, wherein the first training set comprises a part of reference questions in the target knowledge base and corresponding reply content; under the condition that the recognition accuracy of the trained initial recognition model on the data in a first verification set reaches a first threshold value, taking the trained initial recognition model as an intention recognition model, wherein the first verification set comprises another part of reference questions in a target knowledge base and corresponding reply contents; under the condition that the recognition accuracy of the trained initial recognition model to the data in the first verification set does not reach the first threshold, the data in the first training set is continuously used for training the initial recognition model until the recognition accuracy of the trained initial recognition model to the data in the first verification set reaches the first threshold.

Optionally, after the first response content determined according to the intention recognition result is returned to the intelligent terminal, labeling the first question content, wherein the label of the first question content is used for indicating whether the customer service robot takes the first response content as the response content of the first question content to pass verification; and storing the labeled first question content and the labeled first answer content into a historical question-answer database.

Optionally, after the labeled first question content and the labeled first answer content are saved in a historical question-answer database, training an intention recognition model by using data in a second training set, wherein the second training set comprises a part of questions in the historical question-answer database and corresponding answer content; stopping training under the condition that the recognition accuracy of the trained intention recognition model on the data in a second verification set reaches a second threshold value, wherein the second verification set comprises another part of questions in the historical question-answer database and corresponding answer contents; under the condition that the recognition accuracy of the trained intention recognition model to the data in the second verification set does not reach the second threshold, the intention recognition model continues to be trained by using the data in the second training set until the recognition accuracy of the trained intention recognition model to the data in the second verification set reaches the second threshold.

Optionally, before receiving first questioning content sent by the intelligent terminal, prompting a plurality of interaction modes on the intelligent terminal, wherein the plurality of interaction modes include a task mode, a chatting mode and a manual mode; under the condition that a response message of the intelligent terminal for selecting the task mode is received, first questioning content sent by the intelligent terminal is received.

Optionally, after the intelligent terminal is prompted with a plurality of interaction modes, receiving second questioning content sent by the intelligent terminal under the condition of receiving a response message of the intelligent terminal selecting the chatting mode; encoding the semantic vector of the second question content through an encoder to obtain the semantic vector for expressing the second question content; decoding the semantic vector of the second question content by using a decoder to obtain second answer content; and returning the second reply content to the intelligent terminal.

Optionally, the receiving the first question content sent by the intelligent terminal includes: and receiving first question content transmitted by the intelligent terminal by adopting a hypertext transfer protocol (HTTP).

According to another aspect of the embodiments of the present application, there is also provided an operation device for an intelligent customer service, including: the receiving unit is used for receiving first questioning content sent by the intelligent terminal; the identification unit is used for identifying an intention identification result matched with the first question content through an intention identification model, wherein the intention identification model is obtained by adopting a plurality of groups of question contents for training, and each question content in the same group of question contents is marked to be matched with the same answer content; and the returning unit is used for returning the first reply content determined according to the intention recognition result to the intelligent terminal.

Optionally, the return unit may be further operable to: under the condition that the confidence of the intention recognition result is greater than the target threshold, returning first reply content matched with the first question content to the intelligent terminal; and under the condition that the confidence of the intention recognition result is not greater than the target threshold, searching first reply content matched with the first question content from the target knowledge base, and returning the first reply content to the intelligent terminal.

Optionally, the return unit may be further operable to: the returning of the first reply content determined according to the intention recognition result to the intelligent terminal includes: acquiring a plurality of candidate reply contents matched with the intention recognition result; and selecting the first reply content with the highest matching degree with the first question content from the candidate reply contents as the first reply content and returning the first reply content to the intelligent terminal.

Optionally, the target knowledge base stores a plurality of reference questions and the reply content corresponding to each reference question, and the returning unit is further configured to: searching candidate reference questions matched with the first question content from a target knowledge base; feeding back the candidate reference problem to the intelligent terminal; determining a target reference problem selected by the intelligent terminal in the candidate reference problems; and taking the reply content corresponding to the target reference question in the target knowledge base as the first reply content.

Optionally, the apparatus may further include a configuration unit configured to select a target knowledge base for the customer service robot from a plurality of knowledge bases before searching the target knowledge base for first response content matching the first question content, wherein each of the plurality of knowledge bases stores a question in a field and corresponding response content.

Optionally, the apparatus may further include an expanding unit, configured to obtain historical question content after receiving the first question content sent by the intelligent terminal, where the historical question content includes the first question content; clustering and analyzing the historical questioning contents to obtain multiple questioning topics; and supplementing the target knowledge base with response contents matched with the various question topics.

Optionally, the extension unit may be further configured to obtain historical question content after receiving the first question content sent by the intelligent terminal, where the historical question content includes the first question content; performing word frequency statistics on the historical question content to obtain the word frequency of words in the historical question content; and supplementing the reply content associated with the target word in the target knowledge base, wherein the word frequency of the target word is greater than the word frequency of the words except the target word in the historical questioning content.

Optionally, the apparatus may further include a model training unit, configured to train the initial recognition model with data in a first training set before receiving first question content sent by the intelligent terminal, where the first training set includes a part of reference questions in the target knowledge base and corresponding reply content; under the condition that the recognition accuracy of the trained initial recognition model on the data in a first verification set reaches a first threshold value, taking the trained initial recognition model as an intention recognition model, wherein the first verification set comprises another part of reference questions in a target knowledge base and corresponding reply contents; under the condition that the recognition accuracy of the trained initial recognition model to the data in the first verification set does not reach the first threshold, the data in the first training set is continuously used for training the initial recognition model until the recognition accuracy of the trained initial recognition model to the data in the first verification set reaches the first threshold.

Optionally, the apparatus may further include a labeling unit, configured to label the first question content after returning the first response content determined according to the intention recognition result to the intelligent terminal, where the label of the first question content is used to indicate whether the customer service robot verifies the first response content as the response content of the first question content; and storing the labeled first question content and the labeled first answer content into a historical question-answer database.

Optionally, the model training unit may be further configured to train the intention recognition model by using data in a second training set after the labeled first question content and the labeled first answer content are saved in the historical question-answer database, where the second training set includes a part of questions in the historical question-answer database and corresponding answer content; stopping training under the condition that the recognition accuracy of the trained intention recognition model on the data in a second verification set reaches a second threshold value, wherein the second verification set comprises another part of questions in the historical question-answer database and corresponding answer contents; under the condition that the recognition accuracy of the trained intention recognition model to the data in the second verification set does not reach the second threshold, the intention recognition model continues to be trained by using the data in the second training set until the recognition accuracy of the trained intention recognition model to the data in the second verification set reaches the second threshold.

Optionally, the receiving unit may be further configured to prompt, before receiving the first question content sent by the intelligent terminal, a plurality of interaction modes on the intelligent terminal, where the plurality of interaction modes include a task mode, a chat mode, and a manual mode; and receiving first question content sent by the intelligent terminal under the condition of receiving a response message of the task mode selected by the intelligent terminal.

Optionally, the returning unit may be further configured to receive, after prompting the multiple interaction modes on the intelligent terminal, second question content sent by the intelligent terminal in a case of receiving a response message that the intelligent terminal selects the chatting mode; encoding the semantic vector of the second question content through an encoder to obtain the semantic vector for expressing the second question content; decoding the semantic vector of the second question content by using a decoder to obtain second answer content; and returning the second reply content to the intelligent terminal.

Optionally, the receiving unit may be further configured to receive first question content transmitted by the intelligent terminal using a hypertext transfer protocol HTTP.

According to another aspect of the embodiments of the present application, there is also provided an operation system of an intelligent customer service, including: the customer service robot is used for identifying an intention identification result matched with the first question content through an intention identification model under the condition that the first question content of the intelligent terminal is received, and returning first reply content determined according to the intention identification result to the intelligent terminal; and the model training system is used for training an intention recognition model by adopting a plurality of groups of questioning contents and providing the trained intention recognition model for the customer service robot to call, wherein each questioning content in the same group of questioning contents is marked to be matched with the same reply content.

According to another aspect of the embodiments of the present application, the operation system may further include: the management background is used for configuring a target knowledge base for the customer service robot from the plurality of knowledge bases and configuring an intention recognition model for the customer service robot, the management background provides the robot configuration information, the adding, deleting, modifying and checking operations of knowledge bar information, model training information and the like, meanwhile, data analysis results, historical chatting data information and the like can be checked, and the knowledge bar, the robot configuration and the recognition model can be updated at any time through the management background; the system comprises a plurality of knowledge bases, a plurality of communication terminals and a plurality of communication terminals, wherein each knowledge base in the plurality of knowledge bases stores a field question and corresponding reply content, and when the configuration of the knowledge bars is carried out through a background, a plurality of robots are considered to have common knowledge bars, so that the universal configuration can be increased, and any robot can use related knowledge content by associating the universal knowledge bars through the background; the system comprises a historical question-answer database, a customer service robot and a database server, wherein the historical question-answer database is used for storing question contents and corresponding reply contents received by the customer service robot; the data annotation platform is mainly used for annotating historical session information of spot check by background staff, marking whether the questions are accurately answered or not, if the questions are answered incorrectly, judging whether the questions are accurately answered or not, and storing information after annotation in the historical question-answer database for iterative training of an identification model; the data analysis service is used for analyzing the questioning content and the corresponding reply content in the historical questioning and answering database; the model training system is mainly responsible for iterative training of an intention recognition model, when an access party starts to use, the system trains a model by using an added knowledge base, the recognition capability of the model is possibly weak when the access party starts to use, so that the model needs to be iteratively updated by manually marked data, the recognition capability of the model can be stronger and stronger, the model training system can regularly run a model training program every day and can also be manually operated to start training, meanwhile, one part of the marked data is used as a test data set, the test data set is used for evaluation after the model training is completed every time, and an evaluation result is recorded into the database for a calling party to make a decision.

According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program which, when executed, performs the above-described method.

According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.

In the embodiment of the application, in the process of asking questions by a user, the intention of the user is identified through the intention identification model, which is equivalent to the fact that the customer service robot can understand the user questions in a generalization mode and further obtain the response content matched with the question content, but not only can identify the existing fixed question method in the knowledge base, so that the technical problem that the effective answer rate of the intelligent customer service is low in the related technology can be solved, and the technical effect of improving the effective answer rate is achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of an alternative method of operation of a smart customer service according to an embodiment of the present application;

FIG. 2 is a flow chart of an alternative method of operation of a smart customer service according to an embodiment of the present application;

FIG. 3 is a flow chart of an alternative method of operation of a smart customer service according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an alternative model for intelligent customer service according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an alternative text classification according to an embodiment of the present application;

FIG. 6 is a schematic diagram of an alternative transfer-to-manual service according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an alternative word analysis in accordance with embodiments of the present application;

FIG. 8 is a schematic diagram of an alternative statistical word cloud in accordance with embodiments of the present application;

FIG. 9 is a schematic diagram of an alternative means of operation of a smart customer service in accordance with an embodiment of the present application;

and

fig. 10 is a block diagram of a terminal according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

According to an aspect of embodiments of the present application, a system embodiment of an operating system of an intelligent customer service is provided. As shown in fig. 1, the system mainly comprises a customer service robot, configured to, in a case where a first question content of an intelligent terminal is received, identify an intention recognition result matching the first question content through an intention recognition model, and return a first reply content determined according to the intention recognition result to the intelligent terminal; and the model training system is used for training the intention recognition model by adopting a plurality of groups of questioning contents and providing the trained intention recognition model for the customer service robot to call, and each questioning content in the same group of questioning contents is marked to be matched with the same reply content.

Optionally, the operating system may further include:

the management background is used for configuring a target knowledge base for the customer service robot from the plurality of knowledge bases and configuring an intention recognition model for the customer service robot, the management background provides the robot configuration information, the adding, deleting, modifying and checking operations of knowledge bar information, model training information and the like, meanwhile, data analysis results, historical chatting data information and the like can be checked, and the knowledge bar, the robot configuration and the recognition model can be updated at any time through the management background;

the system comprises a plurality of knowledge bases, a plurality of communication terminals and a plurality of communication terminals, wherein each knowledge base in the plurality of knowledge bases stores a field question and corresponding reply content, and when the configuration of the knowledge bars is carried out through a background, a plurality of robots are considered to have common knowledge bars, so that the universal configuration can be increased, and any robot can use related knowledge content by associating the universal knowledge bars through the background;

the system comprises a historical question-answer database, a customer service robot and a database server, wherein the historical question-answer database is used for storing question contents and corresponding reply contents received by the customer service robot;

the data annotation platform is mainly used for annotating historical session information of spot check by background staff, marking whether the questions are accurately answered or not, if the questions are answered incorrectly, judging whether the questions are accurately answered or not, and storing information after annotation in the historical question-answer database for iterative training of an identification model;

the data analysis service is used for analyzing the questioning content and the corresponding reply content in the historical questioning and answering database;

the model training system is mainly responsible for iterative training of an intention recognition model, when an access party starts to use, the system trains a model by using an added knowledge base, the recognition capability of the model is possibly weak when the access party starts to use, so that the model needs to be iteratively updated by manually marked data, the recognition capability of the model can be stronger and stronger, the model training system can regularly run a model training program every day and can also be manually operated to start training, meanwhile, one part (such as 10%) of the marked data is used as a test data set, the test data set is used for evaluation after model training is completed every time, and an evaluation result is recorded into the database for a calling party to make a decision.

As shown in fig. 1, an external user invokes a service through an HTTP request, and first, the request enters an intelligent customer service robot service, the intelligent customer service robot service responds to a question of the user according to a knowledge base and an intention recognition model configured in a management background, the provided answers include "direct answer", "guided answer", "no answer", and the like, and the response content is returned to the caller in the form of a JSON character string.

According to an aspect of the embodiment of the present application, a method embodiment of an operation method of an intelligent customer service is also provided. Fig. 2 is a flowchart of an alternative method for operating a smart customer service according to an embodiment of the present application, and as shown in fig. 2, the method may include the following steps:

step S202, the customer service robot receives the first question content sent by the intelligent terminal.

For example, a game player has a game client in his/her smart terminal, and asks "XXX items how to obtain" in the game client.

In step S204, the customer service robot identifies an intention identification result matching the first question content through an intention identification model, the intention identification model is obtained by training multiple sets of question contents, and each question content in the same set of question contents is labeled to match the same answer content.

The intention recognition model recognizes the intention of the user and then searches for the response content corresponding to the intention, such as "XX prop requires the player to enter a certain designated area of the map and acquire by eliminating a certain monster".

In step S206, the customer service robot returns the first reply content determined according to the intention recognition result to the intelligent terminal.

According to the technical scheme, in the process of asking questions of the user, the intention of the user (namely the content which the user wants to know) is identified through an intention identification model, namely the user can understand the user questions in a generalization mode, and then the answer content matched with the question content is obtained, instead of only identifying the existing fixed question method in the knowledge base, so that the technical problem that the effective answer rate of the intelligent customer service in the related technology is low can be solved, and the technical effect of improving the effective answer rate is further achieved. The technical solution of the present application is further described below with reference to specific embodiments.

Step 1, training the intention recognition model, which is optional, and can be a model user training the model by himself, or a trained model purchased from other platforms, and the specific training scheme refers to the following.

Step 11, training the initial recognition model by using data in a first training set, where the first training set includes a part of reference questions in the target knowledge base, corresponding reply contents, and corresponding tag information, and the tag is used to indicate whether the reply contents are matched with the reference questions, for example, the question "how to obtain the XX prop" and the reply contents "obtain from the northwest corner of the XX map", the tag contents are "1" or "0", "1" indicates matching ", and" 0 "indicates mismatching.

And step 12, after the number of training rounds reaches a certain value (such as 1000 rounds), identifying data in a first verification set by using the trained initial identification model to determine whether the reference questions in the verification set are matched with the corresponding reply contents, wherein the first verification set comprises another part of the reference questions in the target knowledge base and the corresponding reply contents.

And step 13, taking the trained initial recognition model as an intention recognition model when the recognition accuracy of the trained initial recognition model on the data in the first verification set reaches a first threshold (e.g. 99%).

And 14, under the condition that the identification accuracy of the trained initial identification model to the data in the first verification set does not reach the first threshold value, continuing to train the initial identification model by using the data in the first training set until the identification accuracy of the trained initial identification model to the data in the first verification set reaches the first threshold value.

In the related technical scheme, the maintenance cost of the knowledge base is high, when a user cannot search a result, the user needs to modify or newly add configuration knowledge bars or keywords to quickly respond, meanwhile, as a plurality of items exist, common knowledge bars are often arranged among the items, and the same knowledge bars need to be repeatedly manually maintained.

In the technical scheme of the application, a pre-trained intention recognition model can be configured for the customer service robot; after configuring the intention recognition model, configuring a knowledge base for the intention recognition model (i.e. the customer service robot corresponding to the model), that is, selecting a target knowledge base (which may be one or more, such as a special knowledge base including a general knowledge base and a certain domain) for the customer service robot from a plurality of knowledge bases, wherein each knowledge base in the plurality of knowledge bases can store a domain question and corresponding response content. By adopting a flexible knowledge base configuration scheme, when a knowledge bar is updated or newly added, only one knowledge base can be modified, and then the knowledge base is configured to the customer service robot to be used.

And 2, the customer service robot receives the first question content sent by the intelligent terminal.

Optionally, a plurality of interaction modes are prompted on the intelligent terminal, and the plurality of interaction modes include a task mode, a chatting mode and a manual mode. Under the condition of receiving a response message of a user selecting a task mode on the intelligent terminal, the intelligent customer service robot receives a first question content transmitted by the intelligent terminal through a hypertext transfer protocol (HTTP).

Optionally, after the intelligent terminal is prompted with a plurality of interaction modes, receiving second questioning contents sent by the intelligent terminal under the condition that a response message of the intelligent terminal selecting the chatting mode is received; encoding semantic vectors of the second question content through an encoder, namely encoding the received question content into the same semantic space (a large number of reference questions and semantic vectors corresponding to reply content exist in the semantic space) to obtain the semantic vectors for expressing the second question content; decoding the semantic vector of the second question content by using a decoder to obtain second answer content, for example, searching a reference question closest to the question by calculating the similarity of the semantic vector, and taking the answer content corresponding to the reference question as the second answer content; and returning the second reply content to the intelligent terminal.

And 3, identifying an intention identification result matched with the first question content by the customer service robot through the intention identification model.

And 4, returning the first reply content determined according to the intention recognition result to the intelligent terminal.

In step 4, two implementation manners are included, one of which is to obtain a plurality of candidate reply contents matched with the intention recognition result when the confidence of the intention recognition result is greater than a target threshold (for example, 80%), and select the first reply content with the highest matching degree with the first question content from the plurality of candidate reply contents to return to the intelligent terminal; secondly, when the confidence of the intention recognition result is not greater than a target threshold (such as 80%), searching first reply content matched with the first question content from the target knowledge base, and returning the first reply content to the intelligent terminal.

In the above embodiment, the target knowledge base stores a plurality of reference questions and the reply content corresponding to each reference question, and when searching for the first reply content matching the first question content from the target knowledge base, the following steps may be performed:

step 41, the customer service robot searches for a candidate reference question matched with the first question content from the target knowledge base, for example, calculates the similarity of semantic vectors between the first question content and the reference question in the knowledge base, and takes the candidate reference question with the similarity reaching a certain threshold (e.g., 70%).

And 42, the customer service robot feeds the candidate reference problems back to the intelligent terminal so as to enable the user to select the problems which the user actually wants to express.

And 43, the customer service robot determines the target reference problem selected by the intelligent terminal in the candidate reference problems, and takes the target reference problem as a problem which the user actually wants to express.

And step 44, taking the reply content corresponding to the target reference question in the target knowledge base as the first reply content.

In the related art, the customer service robot often lacks an effective statistical analysis means to analyze what the user asks, and often needs to manually analyze from massive user questions in order to update a knowledge base or answer a conversation. To overcome this problem, the present application provides a solution for knowledge base supplementation by using artificial intelligence techniques, as shown in steps 5 and 6.

After the above step 4 is completed, the following processes of knowledge base supplement by cluster analysis (i.e., step 5), knowledge base supplement by word frequency analysis (i.e., step 6), and model generalization (steps 7 to 8) may be selectively performed.

And 5, supplementing the knowledge base through clustering analysis.

Step 51, obtaining historical question contents, wherein the historical question contents comprise the question contents received by the customer service robot, such as the first question contents.

And step 52, performing cluster analysis on the historical questioning contents, for example, performing cluster analysis by using a k-means algorithm to obtain multiple types of questioning topics.

In step 53, the target knowledge base is supplemented with response contents matching with multiple types of questioning topics, for example, if the obtained questioning topics include "XX monster" and "XX prop", the knowledge base is supplemented with question and response contents related to "XX monster" and "XX prop".

If clustering is performed using the K-means algorithm, which is an unsupervised machine learning algorithm that classifies unlabeled data (i.e., data that does not define classes or groups), the goal of the algorithm is to find groups in the data that are labeled by the variable K. The algorithm iteratively operates to assign each data point to one of the K groups based on the provided features. Data points are clustered based on feature similarity.

And 6, supplementing the knowledge base through word frequency statistics.

And step 61, obtaining historical question contents, wherein the historical question contents comprise the question contents received by the customer service robot, such as the first question contents.

And step 62, carrying out word frequency statistics on the historical question content to obtain the word frequency of the words in the historical question content.

And step 63, determining the most frequent words in all the words as target words, supplementing the reply contents associated with the target words in the target knowledge base, wherein the word frequency of the target words is greater than the word frequency of the words except the target words in the historical question contents.

And (3) word frequency statistics, namely counting word frequencies after word segmentation according to hours through a timing task, then adding and counting, so that a word frequency statistical table can be obtained, selecting N words with the most occurrence times in a certain period of time, obtaining hot words in the period of time, drawing a hot word statistical word cloud picture, knowing what the user asks for, analyzing user data in real time, optimizing a knowledge base and optimizing configuration.

And 7, carrying out data annotation. For each historical reply content and corresponding question, the annotation may be made in the same manner as the first question content.

And step 71, after the first reply content determined according to the intention identification result is returned to the intelligent terminal, labeling the first question content, wherein the label of the first question content is used for indicating whether the first reply content as the reply content of the first question content by the customer service robot passes verification or not, for example, whether the first reply content is matched with the first question content is verified again by manual work, and the verification result is used as a marking basis.

And step 72, storing the labeled first question content and the labeled first answer content in a historical question-answer database.

And 8, carrying out model generalization training.

And step 81, using one part of reference questions and corresponding reply contents in the historical question-answer database as a second training set, and using the other part of reference questions and corresponding reply contents in the historical question-answer database as a second verification set.

And 82, training the intention recognition model by using the data in the second training set.

And step 83, after the training reaches a certain number of rounds (such as 1000), testing the trained intention recognition model by using the data in the second verification set, namely, the intention recognition model can accurately give accurate reply contents matched with the question, if the answer contents can be given, the recognition is correct, otherwise, the answer contents are incorrect.

And 84, stopping training when the recognition accuracy of the trained intention recognition model on the data in the second verification set reaches a second threshold value.

And step 85, under the condition that the recognition accuracy of the trained intention recognition model on the data in the second verification set does not reach the second threshold, continuing to train the intention recognition model by using the data in the second training set until the recognition accuracy of the trained intention recognition model on the data in the second verification set reaches the second threshold.

In the above scheme, the intention recognition models of the versions obtained by the first training and generalized training each time can be stored in the database, and after each training is completed, the user can select whether to select the latest version or return to any previous version; considering that the identification precision of the model is improved in the continuous generalization training process, but the adaptive breadth is possibly reduced, in the using process, for users with different precisions and adaptive scopes, the users can be configured with the intention identification model meeting the version with the precision and the breadth.

In the running process of the intelligent customer service robot, a annotating person or an automatic annotating system can carry out spot check and annotation on historical question-answer data of the intelligent customer service robot, an annotated result can be recorded in a database and used for training of an intention recognition model, a model training system can automatically operate a training model, the training data of the model is annotated data, the model can be automatically evaluated by using test data after the model training is finished (the annotated data can be divided into two parts, one part is used for model training, and the other part is used for model testing), and a trained model storage path and an evaluated result can be stored in the database. The caller can select whether to use the new training model or not in the management background through the evaluation result, and can also return to the previous model versions. The data analysis service can read historical question answering data, carry out word frequency statistics and cluster analysis, and the analysis result is stored in a database and can be checked by a user through a management background.

In the scheme, a method for realizing the intelligent customer service robot is provided. The method mainly comprises the following steps: automatically inputting question-answer knowledge bars manually or by a machine, providing a plurality of question similarity questions, and training an intention recognition model on the basis of the question similarity questions; a user transmits a question and related parameters of a user to call a service through an HTTP request; the intelligent customer service robot service judges the user intention through a search engine and an intention recognition model and provides a template answer; the data analysis service analyzes historical question and answer data through means of word frequency statistics, text clustering and the like, and helps a user to adjust a knowledge base and robot rules; and a labeling person or automatic labeling equipment labels the historical question and answer data, and the labeled data is used for training the intention recognition model.

As an alternative example, the technical solution of the present application is further described below with reference to specific embodiments.

The implementation process of the scheme can be implemented in an intelligent customer service robot system as shown in fig. 1. The system comprises the following parts: the intelligent customer service robot system comprises an intelligent customer service robot service, a management background, a data annotation platform, a model training system, a data analysis service and knowledge database and a historical question and answer database. The intelligent customer service robot service comprises a model intention identification module and a search engine module, answers are provided for users through the two modules, the intention identification module performs text classification through a machine learning model to achieve the purpose of identifying the intention of the users, and the search engine module obtains knowledge bars related to questions asked by the users in a mode of searching a knowledge base; the management background can dynamically update the robot knowledge bar and the configuration intention recognition model, and can call data analysis service to obtain the analysis result of historical question-answer data; the data annotation platform can select and label wrong dialogs identified in historical question and answer data by using a manual spot check acquisition machine, and the labeled data can be used for updating a machine learning intention identification model; the model training system can manually or automatically train the model by using manually marked data and evaluate the trained model; the data analysis service can analyze historical question and answer data in the modes of word frequency statistics, clustering algorithm and the like, and assists users or administrators to timely and efficiently adjust the relevant configuration of a knowledge base or a robot; the knowledge database and the historical question-answer database are used for storing knowledge base data, related configuration data, analysis data, historical conversation data, manual marking data, model evaluation and updating data and the like.

The services of the intelligent customer service robot mainly comprise a chatting model, intention identification, knowledge base search, guided response, question and answer template matching, manual transfer and the like, and a program flow diagram is shown in figure 3.

Step S301, the caller transmits user question information and selects a question-answering mode. Two question-answering modes can be selected here: "chatting mode" and "task mode".

In step S302, the user selects the chat mode, and the user question is introduced into a seq2seq model (Sequence-to-Sequence, i.e. Sequence-to-Sequence process).

The seq2seq model employed in the present application is shown in FIG. 4 and comprises an Encoder Encoder (i.e., the left part of c, h)₁To h₄For the coding layer of the encoder, h₀、x₁To x₄As input) and Decoder (i.e., c right side portion, h'₁To h'₄Being a decoding layer of a decoder, y₁To y₃For output), the Encoder may be an RNN model (Recurrent Neural Network model), the input sentence is encoded into a semantic vector context vector, Decoder is the inverse process of the Encoder, each state is determined by the previous state and the context vector, the question input by the user is encoded by the Encoder, the answer is generated by the Decoder, here, the seq2seq chatting model can be trained by using open field dialogue corpus, the user question passes through the chatting model, the model generates a chatting reply, and the reply is returned to the user.

In step S303, the user selects the task mode, and first the user asks a question to enter an intention recognition model.

The intention recognition model is mainly used for recognizing the real standard intention of the user to ask questions, and is a text classification model, which is not limited to text classification models such as TextCNN, RNN, BERT, etc., and the classification flow is shown in fig. 5 (where the word segmentation and word vector model method and flow are described below in relation to the data analysis service).

The training data source intended to be identified here is two parts, one part being a knowledge base entered by the user, the form of the knowledge base being: one part of the methods 1, 2, 3, … and N and the answers corresponding to the methods can be data collected automatically by a marking person through a marking acquisition machine, when the robot is just started to use, the model can be trained by using a knowledge base, because of similar methods, the model can have generalization capability, and when a user asks questions in different modes, the intention of the user can be recognized accurately. After the robot runs for a period of time, the annotating personnel can annotate historical data, and after the annotated data exist, the annotated data can be added into the training data set, so that the generalization capability of the model can be further improved.

Step S304, after the intention recognition is completed, whether the recognition confidence coefficient meets the threshold value condition is judged.

Step S305, when the threshold is met, the answer template is matched directly according to the intention, namely the answer template information is added in the input of the knowledge base.

In step S306, when the threshold is not satisfied, a knowledge base search is performed using the question of the user.

The search may be implemented using the Elasticsearch search engine (other similar engines may be used instead).

And step S307, judging whether the search has a result, and prompting that an answer cannot be given when the search has no result.

In step S308, when the result can be searched out, a guidance answer is performed.

And when the result cannot be searched out, the answer cannot be given, and the answer dialect which cannot be given is returned. If the search result exists, the Elasticissearch gives N results through the relevance of the text, and the intelligent customer service robot service gives a plurality of search standard question lists from high to low through the relevance to the user as a 'guide answer'.

In step S309, the user makes a question selection.

The "guiding answer" is to let the user select the question most similar to his intention through the searched list, so that when the intention can not identify the question, several questions with the highest relevance can be provided through searching, if the user selects a certain question, the answer is provided directly through the answer template, and if the user does not select the question, the process returns to the initial process.

The intelligent customer service robot service also comprises manual switching logic, and the intelligent customer service robot can be switched into manual service at a proper time as the assistance of manual customer service, so that the user experience cannot be influenced. The logic of intelligent customer service robot service to manual work is shown in fig. 6, and includes "N times of guidance to manual work" (N is a natural number greater than or equal to 2, and can be configured specifically as required), "N times of unresponsive question to manual work", "N times of questioning the same question to manual work", "N times of anger to manual work", and "keyword to manual work", where "N times" may be "multiple times" continuously and "multiple times" cumulatively. One or more manual switching rules are configured in the management background, so that the aim of flexibly switching to manual service is fulfilled.

According to the 'N angry to manual' rule, emotion analysis needs to be carried out on user questions, and a fasttext algorithm can be used for training a classification model on a public emotion data set and a labeled data set to achieve the purpose of emotion analysis.

In the scheme, the data analysis service mainly analyzes user question data from historical question and answer data, knows question direction and hot questions of a questioner through analysis results, determines knowledge bars needing to be added or adjusted through the analysis results, and modifies various configurations such as manual rules.

First, the processed text may be segmented, which is a process of recombining continuous word sequences into word sequences according to a certain specification, for example, the text "i comes to the university of qinghua in beijing", and this section of text may be cut into an array of several words [ i, come, beijing, university of qinghua ] by the segmentation processing, where a jieba segmentation tool may be used to implement the segmentation function.

After the word segmentation is completed, there are two analysis means to implement the analysis, such as cluster analysis and word frequency statistics shown in fig. 7. Clustering analysis requires the use of word vector (word embedding) algorithms to process word arrays into numerical matrices that can be recognized and operated on by computers, such as: the array [ i, come, beijing, university of qinghua ], can be processed into the following numerical matrix: [0.32,0.56,0.33 … ], [0.23,0.16,0.93 … ], [0.09,0.16,0.13 … ], [0.12,0.51,0.83 … ], where word vector operations can be performed using word embedding tools such as word2vec, glove, etc., through which mapping operations of natural language to numerical matrices are achieved.

After the word embedding operation is completed, clustering analysis is realized by using a k-means algorithm, and k similar topics can be obtained through the clustering analysis, so that the knowledge base can be perfected through the topics.

And (3) performing word frequency statistics, namely performing word frequency statistics on the word after word segmentation according to hours through a timing task, and then performing addition and counting, so that a word frequency statistical table can be obtained, as shown in fig. 8. Through word frequency statistics, the user can know what contents the user asks, the user data is analyzed in real time, a knowledge base is optimized, and configuration is optimized.

In the related technology, the customer service robot has low direct answer rate, poor user experience, high maintenance cost of the knowledge base, no effective analysis means, inflexible configuration and can not well meet the requirements of customer service personnel. By adopting the technical scheme of the application, the following technical effects can be realized: the user intention is analyzed by combining a search engine text search and semantic understanding intention recognition model, so that a better recognition effect is achieved; model training data sets are continuously added through a labeling system, so that the model can be quickly updated in an iterative manner, and the recognition capability is gradually enhanced; the model is automatically updated and evaluated, and a manager can quickly update the model through the evaluation result; an effective analysis means helps a user to analyze the user problem; through HTTP interface access, the user can access simply and conveniently, and can flexibly define various rules and configurations.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.

According to another aspect of the embodiment of the present application, there is also provided an operation device of the intelligent customer service, which is used for implementing the operation method of the intelligent customer service. Fig. 9 is a schematic diagram of an alternative intelligent customer service operation device according to an embodiment of the present application, and as shown in fig. 9, the device may include:

a receiving unit 901, configured to receive a first question content sent by an intelligent terminal;

the identification unit 903 is used for identifying an intention identification result matched with the first question content through an intention identification model, wherein the intention identification model is obtained by adopting a plurality of groups of question contents for training, and each question content in the same group of question contents is marked to be matched with the same answer content;

a returning unit 905, configured to return the first reply content determined according to the intention recognition result to the intelligent terminal.

It should be noted that the receiving unit 901 in this embodiment may be configured to execute step S202 in this embodiment, the identifying unit 903 in this embodiment may be configured to execute step S204 in this embodiment, and the returning unit 905 in this embodiment may be configured to execute step S206 in this embodiment.

It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may operate in a hardware environment as shown in fig. 1, and may be implemented by software or hardware.

Through the module, in the process of asking questions by a user, the intention of the user is identified through the intention identification model, which is equivalent to the situation that the user questions can be understood in a generalization manner, and then the answer content matched with the content of the questions is obtained, instead of only identifying the existing fixed questioning method in the knowledge base, the technical problem that the effective answer rate of the intelligent customer service is low in the related technology can be solved, and the technical effect of improving the effective answer rate is achieved.

Optionally, the apparatus may further include a labeling unit, configured to label the first question content after returning the first answer content determined according to the intention recognition result to the intelligent terminal, where the label of the first question content is used to indicate whether the first answer content is correct answer content of the first question content; and storing the labeled first question content and the labeled first answer content into a historical question-answer database.

It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be operated in a hardware environment as shown in fig. 1, and may be implemented by software, or may be implemented by hardware, where the hardware environment includes a network environment.

According to another aspect of the embodiment of the application, a server or a terminal for implementing the operation method of the intelligent customer service is also provided.

Fig. 10 is a block diagram of a terminal according to an embodiment of the present application, and as shown in fig. 10, the terminal may include: one or more processors 1001 (only one of which is shown in fig. 10), memory 1003, and a transmission apparatus 1005, the terminal may further include an input-output device 1007, as shown in fig. 10.

The memory 1003 may be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for operating the intelligent customer service in the embodiment of the present application, and the processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1003, that is, implements the above-mentioned method for operating the intelligent customer service. The memory 1003 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1003 may further include memory located remotely from the processor 1001, which may be connected to a terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmitting device 1005 is used for receiving or transmitting data via a network, and can also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmitting device 1005 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmitting device 1005 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

Among them, the memory 1003 is used to store an application program, in particular.

The processor 1001 may call an application stored in the memory 1003 via the transmitting device 1005 to perform the following steps:

receiving first questioning content sent by an intelligent terminal;

identifying an intention identification result matched with the first question content through an intention identification model, wherein the intention identification model is obtained by adopting a plurality of groups of question contents for training, and each question content in the same group of question contents is marked to be matched with the same answer content;

and returning the first reply content determined according to the intention recognition result to the intelligent terminal.

The processor 1001 is further configured to perform the following steps:

under the condition that the matching degree between the intention recognition result and the first question content is larger than a target threshold value, returning the intention recognition result to the intelligent terminal as first reply content;

and under the condition that the matching degree between the intention recognition result and the first question content is not larger than a target threshold value, searching first reply content matched with the first question content from a target knowledge base, and returning the first reply content to the intelligent terminal.

By adopting the embodiment of the application, in the process of asking questions by the user, the intention of the user is identified through the intention identification model, which is equivalent to the situation that the user questions can be understood in a generalization manner, and then the response content matched with the content of the questions is obtained, instead of only identifying the existing fixed questioning method in the knowledge base, the technical problem that the effective answer rate of the intelligent customer service is low in the related technology can be solved, and the technical effect of improving the effective answer rate is further achieved.

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.

It can be understood by those skilled in the art that the structure shown in fig. 10 is only an illustration, and the terminal may be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 10 is a diagram illustrating a structure of the electronic device. For example, the terminal may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Embodiments of the present application also provide a storage medium. Alternatively, in this embodiment, the storage medium may be a program code for executing an operation method of the smart customer service.

Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

receiving first questioning content sent by an intelligent terminal;

Optionally, the storage medium is further arranged to store program code for performing the steps of:

Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. An operation method of intelligent customer service is applied to a customer service robot, and the method comprises the following steps:

receiving first questioning content sent by an intelligent terminal;

2. The method according to claim 1, wherein returning the first reply content determined according to the intention recognition result to the smart terminal comprises:

under the condition that the confidence of the intention recognition result is greater than a target threshold value, returning the first reply content matched with the first question content to the intelligent terminal;

and under the condition that the confidence of the intention recognition result is not greater than the target threshold, searching the first reply content matched with the first question content from a target knowledge base, and returning the first reply content to the intelligent terminal.

3. The method according to claim 2, wherein the target knowledge base stores a plurality of reference questions and response content corresponding to each reference question, and wherein searching the target knowledge base for the first response content matching the first question content comprises:

searching candidate reference questions matched with the first question contents from the target knowledge base;

feeding back the candidate reference problem to the intelligent terminal;

determining a target reference problem selected by the intelligent terminal in the candidate reference problems;

and taking the reply content corresponding to the target reference question in the target knowledge base as the first reply content.

4. The method of claim 2, wherein prior to finding the first response content matching the first question content from a target knowledge base, the method further comprises:

selecting the target knowledge base for the customer service robot from a plurality of knowledge bases, wherein each knowledge base in the plurality of knowledge bases stores a field of questions and corresponding reply content.

5. The method according to claim 1, wherein after receiving the first questioning content sent by the intelligent terminal, the method further comprises:

acquiring historical questioning content, wherein the historical questioning content comprises the first questioning content;

performing clustering analysis on the historical question content to obtain multiple types of question topics;

and supplementing the target knowledge base with the reply contents matched with the various question topics.

6. The method according to claim 1, wherein after receiving the first questioning content sent by the intelligent terminal, the method further comprises:

performing word frequency statistics on the historical question content to obtain word frequency of words in the historical question content;

supplementing reply content associated with a target word in a target knowledge base, wherein the word frequency of the target word is greater than the word frequency of words except the target word in the historical questioning content.

7. An operating device for intelligent customer service, the device comprising:

the receiving unit is used for receiving first questioning content sent by the intelligent terminal;

the identification unit is used for identifying an intention identification result matched with the first question content through an intention identification model, wherein the intention identification model is obtained by adopting a plurality of groups of question contents for training, and each question content in the same group of question contents is marked to be matched with the same answer content;

and the return unit is used for returning the first reply content determined according to the intention recognition result to the intelligent terminal.

8. An operating system for intelligent customer service, the operating system comprising:

the customer service robot is used for identifying an intention identification result matched with the first question content through an intention identification model under the condition that the first question content of the intelligent terminal is received, and returning first reply content determined according to the intention identification result to the intelligent terminal;

and the model training system is used for training an intention recognition model by adopting a plurality of groups of questioning contents and providing the trained intention recognition model for the customer service robot to call, wherein each questioning content in the same group of questioning contents is marked to be matched with the same reply content.

9. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any of the preceding claims 1 to 6.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of the preceding claims 1 to 6 by means of the computer program.