Embodiment
Embodiments herein proposes that a kind of new robot customer service turns the method for artificial customer service, in robot customer service with using
Validity feature is manually marked in the session sample at family and suitably goes out artificial point, confidence assessment models are trained, and utilize instruction
The validity feature of confidence assessment models and current sessions after white silk, is assessed current robot service quality, to find
Transfer the suitable time point of artificial customer service, to determine artificial point according to the actual carry out situation of current sessions, can be lifted
The experience of user, and can avoids the unnecessary live load of artificial customer service, so as to solve problems of the prior art.
Embodiments herein can be applied in any equipment with calculating and storage capacity, such as can be hand
The physical equipments such as machine, tablet personal computer, PC (Personal Computer, PC), notebook, server, virtual machine are patrolled
Collect equipment;The physically or logically equipment of different responsibilities can also be shared by two or more, is mutually cooperateed with to realize this Shen
Various functions that please be in embodiment.
In embodiments herein, the flow that robot customer service turns the method for artificial customer service is as shown in Figure 1.
In embodiments herein, using machine learning techniques come assessment models of building up confidence, for robot customer service
Assessed with the artificial customer service that whether should transfer of some time point during user conversation.Specifically, by a number of machine
The conversation recording of device people customer service and user are as session sample, manually mark validity feature and each session on session sample
Suitably go out artificial point in journey, confidence assessment models are trained using session sample.
Wherein, validity feature is the abstract table of the various factors related to artificial demand of transferring to occurring in session sample
Reach.Validity feature can be generated, assessed by the way of artificial or artificial and data mining technology is combined.One kind is realized
, can be by technical staff according to the working experience, the historical data of robot customer service and user conversation, Yong Huwen of itself in mode
The key to exercises is determined situation etc. and drawn to summarize, refine.
, can be first by the artificial Service Quality that can describe robot customer service summarized, refined in another implementation
Amount and the feature for turning artificial wish of user, by feature undetermined, suitably go out artificial point and actually go out artificial point as feature undetermined
It is labeled on session sample, with reference to the service effectiveness of each session in session sample, is treated using predetermined data analysis algorithm
Determine feature to be assessed and integrated, obtain, covering factor comprehensive several validity features obvious to artificial factors influencing demand of transferring.
For example, can be with manual analysis robot client and the historical data of user conversation, therefrom excavating influences user to manual service
The factor of demand;These factors are refined, sorted out, abstract to turn to feature undetermined, feature undetermined is from session context, industry
The complete above-mentioned factor for describing the artificial demand of influence switching of each different aspects such as business, Consumer's Experience, service track;According to
The problem of user feedback, user, solves situation and the satisfaction situation of user etc. to draw the clothes of each session in session sample
Business effect, there is feature to mark using feature selecting, Feature Extraction Algorithm, suitably go out artificial point and actually go out what is manually put
Session sample and service effectiveness are analyzed, and draw validity feature.
The various data analysis algorithms that can be used for carrying out tagsort and assessing may be used to generate validity feature, this
Do not limited in the embodiment of application.In application scenes, used data analysis algorithm can not only be dialogue-based
Sample and service effectiveness draw validity feature, additionally it is possible to provide the weight of influence of each validity feature to artificial demand of transferring;
In these application scenarios, the weight of each validity feature comments confidence after the training for confidence assessment models, and/or training
In the use for estimating model.
In the conversation procedure of robot customer service and user, time point of the artificial customer service generally after user makes a speech replaces machine
Device people's customer service.In each session of session sample, if the time point after some user speech is to replace machine with artificial customer service
The appropriate time point of device people's customer service, then it can be marked as suitably going out artificial point.There can be one to be fitted to multiple in one session
Preferably go out artificial point.
Confidence assessment models are used for obtaining confidence assessed value based on robot customer service and the session of user, and confidence is assessed
Value is estimate of the confidence assessment models to artificial desirability of transferring, in other words, should be by artificial visitor to certain time point in session
The score value that clothes are got instead of the degree of robot customer service.In the embodiment of the present application to confidence assessment models used by it is specific
Algorithm does not limit, for example, it may be SVR (Support Vector Regression, support vector regression) algorithm, LR
(Logistic Regression, logistic regression) algorithm or GBDT (Gradient Boosting Decision Tree, change
For decision tree) algorithm etc..
, can be in the conversation procedure of real-time robot customer service and user after the completion of the training of confidence assessment models, profit
Judge the time point after user's speech in a session with confidence assessment models, if need to switch to artificial customer service.
Step 110, validity feature is obtained from robot customer service and at least one wheel session of user.
In embodiments herein, made a speech with user's speech or a robot customer service speech and a user
For a round.The first round of usual robot customer service and the session of user is user's speech, and the second wheel and subsequent passes are one
Secondary robot customer service speech and a user make a speech.Each round is terminated with user's speech, and the time point that is to say can be by people
Work customer service replaces the time point of robot customer service.
The validity feature obtained in this step will be used as the input of confidence assessment models, for obtaining current time point
Confidence assessed value.Can be using the session of predetermined number of iterations number before current time point as the basis for obtaining validity feature, if currently
The session round that time point has been carried out is less than predetermined number of iterations number, then whole session is as the basis for obtaining validity feature;Also may be used
To regard the whole session carried out as the basis for obtaining validity feature all the time;Do not limit.
In one implementation, directly can be taken turns to from the one of robot customer service and user in more wheel sessions, according to one
Fixed Rule Extraction goes out validity feature.For example, to text session, semantic analysis, Keywords matching, business tine are referred to
With etc. prior art the recognition rule of each validity feature is set, so as to obtain validity feature from above-mentioned session automatically;To language
Sound session, text session first can be converted into using speech recognition technology, validity feature is being obtained using the above method.
In another implementation, first can be taken turns to from the one of robot customer service and user extracted in more wheel sessions it is original
Feature, effective spy is obtained after being split, combined, classified, and/or deleted to primitive character according still further to feature preprocessing rule
Sign.Similar, to text session, it may be referred to the prior arts such as semantic analysis, Keywords matching, business tine matching and set respectively
The discovery rule of individual primitive character, primitive character is extracted so as to be taken turns to automatically from one in more wheel sessions;, can be with to voice conversation
Voice conversation is first converted to by text session using speech recognition technology, original is being extracted using the discovery rule of primitive character
Beginning feature.
Feature preprocessing rule describes the mapping relations from primitive character to validity feature, including following one kind is to more
Kind:Which or which primitive character can be deleted, which primitive character can be split as which validity feature, which or which
Primitive character can be attributed to represent the validity feature of some classification, which primitive character can be combined as which validity feature etc.
Deng.
For example, in a kind of application scenarios, the primitive character in two dimensions of business and user feedback is as shown in table 1:
Table 1
In primitive character in table 1, if a reply of robot customer service is answer of revealing all the details, the answer of the reply
Matching degree is not necessarily high (if robot customer service can inquire the higher answer of matching degree, with the answer rather than to reveal all the details
Answer is multiplexed family back and forth), it is assumed that answer matches degree and answer of whether revealing all the details all are validity features, then can pre-process and advise in feature
Such one can be included in then:If one reply can extract out whether reveal all the details answer primitive character and answer matches degree it is original
Feature, then the two primitive characters are merged into answer primitive character of whether revealing all the details;In order to avoid the two validity features are used simultaneously
Cause double influence of the same facts to confidence assessed value.
Because confidence assessment models are trained based on validity feature, therefore to the use of confidence assessment models after training
Using validity feature as input.Due to the session sample for training confidence assessment models to need certain amount manually to mark, and reality is objective
Family service in session case may because the increase of business, the change of operation flow, the change of active language and constantly change.
In this implementation, the feature of generality and abstract can be used as validity feature, and according to business development and change
Concrete condition set the primitive character, primitive character to find rule and feature preprocessing rule, be so not required to because actual industry
The change of business situation constantly regenerates session sample and training confidence assessment models, and confidence still can be kept to assess mould
The order of accuarcy of type.
Primitive character finds that rule and/or feature preprocessing rule can be solidificated in a manner of code and complete this step
In program, it can also write in configuration file.To the application scenarios of configuration file be present, can first be obtained before this step is performed
Configuration file, therefrom read primitive character and find rule and/or feature preprocessing rule, then be applied to primitive character extraction
And/or obtain validity feature from primitive character.
It should be noted that primitive character can be selected in feature undetermined, will can also be not present in feature undetermined
Other features do not limit as primitive character.
Step 120, validity feature is inputted into confidence assessment models, obtains the current confidence of robot customer service and user conversation
Assessed value.
Step 130, current confidence assessed value meet it is pre- make artificial condition when, user is transferred artificial customer service.
It is after the validity feature of current time point is corresponded in getting robot customer service and user conversation, validity feature is defeated
Enter in the confidence assessment models to after training, you can obtain the confidence assessed value of current time point.If current confidence assessed value expires
Foot makes artificial condition in advance, then it is assumed that current time point needs artificial customer service intervention, and user is forwarded into artificial customer service.
It can be that current confidence assessed value is more than or less than predetermined confidence threshold value to make artificial condition in advance, depending on practical application field
Current confidence assessed value more Gao Shi represents the artificial desirability of stronger switching, or the weaker artificial demand journey of switching in scape
Degree.Predetermined confidence threshold value can be considered by technical staff the fitting degree of confidence assessment models and session sample after training,
The factors such as the quantitative proportion of user conversation and artificial customer service in practical application scene determine.Can also be by setting certain mark
Standard, predetermined confidence threshold value is automatically determined according to set standard by program.For example, after session sample being input into training
Confidence assessment models in, obtain corresponding in session sample and suitably go out the sample confidence assessed value manually put;Set a series of
The concrete numerical value of different predetermined confidence threshold values, calculate the sample confidence assessed value when selecting the predetermined confidence threshold value of different numerical value
Coverage rate and accuracy rate, setting covers for the judgment criteria of coverage rate and accuracy rate according to the evaluation of judgment criteria is best
Numerical value corresponding to lid rate and accuracy rate is as predetermined confidence threshold value.Wherein, coverage rate is suitably to go out artificial point pair in session sample
In all sample confidence assessed values answered, meet to make artificial condition (concrete numerical value for being more than or less than predetermined confidence threshold value) in advance
Sample confidence assessed value shared by ratio;Accuracy rate is that all satisfactions are made in the sample confidence assessed value of artificial condition in advance,
Corresponding to the ratio suitably gone out shared by the sample confidence assessed value manually put.
In addition, in the application scenarios using configuration file, can also be by making artificial condition also writes configuration file in advance
In, and the pre- artificial condition of making read from configuration file is applied to step 130.
It can be seen that in embodiments herein, validity feature is manually marked in the session sample of robot customer service and user
Suitably go out artificial point, mould is assessed using the session sample training confidence assessment models after mark, and using the confidence after training
Type, the validity feature in current session is assessed, the artificial visitor that whether should currently transfer is judged according to the output of model
Clothes, so as to determine artificial point according to the actual carry out situation of current sessions, can lift the experience of user, and can is kept away
The unnecessary live load of manpower-free's customer service, improve the efficiency of service of customer service system.
In the application example of the application, using the historical record of some robot customer services and the session of user as meeting
Sample is talked about, by professional contact staff's analysis session sample, based on different dimension (including business dimension, Consumer's Experience dimension, clothes
Track dimension of being engaged in etc.) analysis session sample, the factor for influenceing the artificial demand of switching is summarized, refined, is configured to spy undetermined
Sign.
Feature undetermined is marked in session sample, suitably goes out artificial point and actually goes out artificial point, according to each in session sample
The problem of user feedback of individual session, user, solves situation and the satisfaction situation of user, and construction is treated using session sample
Determine feature and carry out data analysis, therefrom evaluate validity feature and draw the weight of validity feature.
Validity feature is marked in session sample and suitably goes out artificial point, using the session sample after mark to LR algorithm
Confidence assessment models are trained.Session in session sample is input to the LR confidence assessment models after training, obtains session
Correspond to the sample confidence assessed value for suitably going out and manually putting in sample, by technical staff according to LR confidence assessment models to session sample
This coverage rate and accuracy rate, determines predetermined information threshold value.
Technical staff writes primitive character in configuration file and finds rule and feature preprocessing rule, and is stored in predetermined
Position.This application example loads primitive character discovery rule and feature preprocessing rule after bringing into operation from configuration file.
After user starts the session with robot customer service, at the end of each user makes a speech, found according to primitive character
Rule, all primitive characters are extracted in having conversated from session start to current time point, reapply feature pretreatment rule
Then, be mapped as belonging to current time point by all primitive characters one arrives multiple validity features.
By the LR confidence assessment models after the validity feature input training for belonging to current time point, obtain current confidence and assess
Value.If LR confidence assessment models represent that the artificial demand of switching is stronger so that the assessed value drawn is higher, then in current confidence assessed value
During more than predetermined information threshold value, user is transferred to artificial customer service, otherwise engaged in the dialogue by robot customer service continuation with user.
Corresponding with the realization of above-mentioned flow, embodiments herein additionally provides the dress that a kind of robot customer service turns artificial customer service
Put, the device can be realized by software, can also be realized by way of hardware or software and hardware combining.It is implemented in software to be
Example, is the CPU (Central Process Unit, central processing unit) by place equipment as the device on logical meaning
Corresponding computer program instructions are read what operation in internal memory was formed.For hardware view, except the CPU shown in Fig. 2,
Outside internal memory and nonvolatile memory, the equipment where robot customer service turns the device of artificial customer service generally also includes being used for
Carry out the chip etc. of wireless signal transmitting-receiving other hardware, and/or other hardware such as board for realizing network communicating function.
Fig. 3 show a kind of device of the artificial customer service of robot customer service turn of the embodiment of the present application offer, including effectively special
Acquiring unit, current confidence assessment unit and artificial customer service adapter unit are levied, wherein:Validity feature acquiring unit is used for from machine
Validity feature is obtained in people's customer service and at least one wheel session of user;Current confidence assessment unit, which is used to input validity feature, to be believed
Heart assessment models, obtain the current confidence assessed value of robot customer service and user conversation;The confidence assessment models are using mark
There is validity feature and suitably go out the robot customer service manually put and the session sample of user is trained, it is described suitably to go out artificial point
To replace the appropriate time point of robot customer service with artificial customer service;Artificial customer service adapter unit is used to meet in current confidence assessed value
It is pre- when making artificial condition, user is transferred artificial customer service.
In one example, the validity feature acquiring unit is specifically used for:From robot customer service and at least one wheel of user
Primitive character is extracted in session, after being split, combined, classified, and/or deleted to primitive character according to feature preprocessing rule
Obtain validity feature.
In above-mentioned example, described device also includes configuration file acquiring unit, for obtaining configuration file;The configuration text
Part includes primitive character and finds rule and/or feature preprocessing rule;The validity feature acquiring unit is specifically used for:Slave
In device people customer service and at least one wheel session of user, Rule Extraction primitive character is found according to primitive character.
Optionally, also include in the configuration file:Make artificial condition in advance.
In above-mentioned example, the primitive character includes:Business correlation, answer matches degree, answer number of repetition, whether pocket
Whether bottom answer, answer are that enquirement, user clearly propose that substitution work and user have potential substitution work tendency, the emotion of user to incline
It is at least one in the problem of explaining oneself to, user.
In a kind of implementation, the validity feature is by tentation data parser according to being marked with feature undetermined, suitable
Go out the artificial session sample put and actually go out the robot manually put and user and the wherein service effectiveness of session, treated to some
Determine feature to be assessed and drawn after being integrated;The feature undetermined can describe the service quality of robot customer service and turning for user
Artificial wish.
In above-mentioned implementation, the validity feature has respective weight, is calculated by the tentation data parser
Draw;The confidence assessment models are trained according to the weight of validity feature.
Optionally, the pre- artificial condition of making includes:Current confidence assessed value is more than or less than predetermined confidence threshold value;Institute
Predetermined confidence threshold value is stated to be determined according to the coverage rate and accuracy rate of several sample confidence assessed values;The sample confidence assessed value
For session sample is inputted after confidence assessment models into obtained output;The coverage rate is suitably to go out artificial point pair in session sample
In all sample confidence assessed values answered, meet the pre- ratio made shared by the sample confidence assessed value of artificial condition;The standard
True rate is made in the sample confidence assessed value of artificial condition in advance for all satisfactions, is commented corresponding to the sample confidence manually put suitably is gone out
Ratio shared by valuation.
Optionally, the confidence assessment models use support vector regression SVR algorithms, logistic regression LR algorithm or iteration
Decision tree GBDT algorithms.
The preferred embodiment of the application is the foregoing is only, not limiting the application, all essences in the application
God any modification, equivalent substitution and improvements done etc., should be included within the scope of the application protection with principle.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and internal memory.
Internal memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM).Internal memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moved
State random access memory (DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic cassette tape, the storage of tape magnetic rigid disk or other magnetic storage apparatus
Or any other non-transmission medium, the information that can be accessed by a computing device available for storage.Define, calculate according to herein
Machine computer-readable recording medium does not include temporary computer readable media (transitory media), such as data-signal and carrier wave of modulation.
It should also be noted that, term " comprising ", "comprising" or its any other variant are intended to nonexcludability
Comprising so that process, method, commodity or equipment including a series of elements not only include those key elements, but also wrapping
Include the other element being not expressly set out, or also include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that wanted including described
Other identical element also be present in the process of element, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer program product.
Therefore, the application can be using the embodiment in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Form.Deposited moreover, the application can use to can use in one or more computers for wherein including computer usable program code
The shape for the computer program product that storage media is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.