CN107590159A - The method and apparatus that robot customer service turns artificial customer service - Google Patents

The method and apparatus that robot customer service turns artificial customer service Download PDF

Info

Publication number
CN107590159A
CN107590159A CN201610538674.6A CN201610538674A CN107590159A CN 107590159 A CN107590159 A CN 107590159A CN 201610538674 A CN201610538674 A CN 201610538674A CN 107590159 A CN107590159 A CN 107590159A
Authority
CN
China
Prior art keywords
customer service
user
artificial
confidence
session
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610538674.6A
Other languages
Chinese (zh)
Inventor
陈利霞
杜敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Advanced New Technologies Co Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201610538674.6A priority Critical patent/CN107590159A/en
Priority to TW106118915A priority patent/TWI677846B/en
Priority to PCT/CN2017/090322 priority patent/WO2018006727A1/en
Publication of CN107590159A publication Critical patent/CN107590159A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a kind of method that robot customer service turns artificial customer service, including:Validity feature is obtained from robot customer service and at least one wheel session of user;Validity feature is inputted into confidence assessment models, obtains the current confidence assessed value of robot customer service and user conversation;The confidence assessment models are using being marked with validity feature and suitably go out the robot customer service manually put and the session sample of user is trained, and described suitably to go out artificial point be the appropriate time point with artificial customer service instead of robot customer service;Current confidence assessed value meet it is pre- make artificial condition when, user is transferred artificial customer service.The technical scheme of the application can identify that user turns the desirability of artificial customer service according to the actual conditions of current sessions, while satisfaction of the raising user to service, reduce the unnecessary work of artificial customer service, improve the efficiency of service of customer service system.

Description

The method and apparatus that robot customer service turns artificial customer service
Technical field
The application is related to technical field of data processing, more particularly to a kind of robot customer service turns the method and dress of artificial customer service Put.
Background technology
With the development of internet, based on the virtual robot of artificial intelligence technology enterprise customer's service field application It is more and more extensive.Robot customer service need not rest, can more quick and standardization response user the problem of, with voice pair The form of words or text chat is linked up with user, and artificial customer service is freed from a large amount of repeated question and answer.
To some unconventional customer problems, robot customer service, which is often difficult to provide, makes customer satisfaction system answer.Visitor at present It is robot customer service with artificial customer service and depositing to take the most commonly used framework in center, default first to receive user by robot customer service, when The session round of robot customer service and user are (with user's speech or a robot customer service speech and user's hair Say as a round) exceed predetermined when going out artificial wheel subthreshold, artificial customer service of transferring.This mode is in most cases caught Catch and carry out more rounds with user less than the transit point suitably by robot customer service to artificial customer service, such as robot customer service Effective interaction, although having reached predetermined goes out artificial wheel subthreshold, do not run into the service obstacle for needing manpower intervention, This mode can cause the unnecessary extra work of artificial customer service;And user has the problem of urgent to need to submit artificial treatment as early as possible When, again can be because reaching the predetermined efficiency and Consumer's Experience for going out manually to take turns the invalid reciprocal effect customer service system before subthreshold.
The content of the invention
In view of this, the application provides a kind of method that robot customer service turns artificial customer service, including:
Validity feature is obtained from robot customer service and at least one wheel session of user;
Validity feature is inputted into confidence assessment models, obtains the current confidence assessed value of robot customer service and user conversation; The confidence assessment models use is marked with validity feature and suitably goes out the robot customer service manually put and the session sample of user It is trained, it is described suitably to go out artificial point to replace the appropriate time point of robot customer service with artificial customer service;
Current confidence assessed value meet it is pre- make artificial condition when, user is transferred artificial customer service.
Present invention also provides the device that a kind of robot customer service turns artificial customer service, including:
Validity feature acquiring unit, for obtaining validity feature from least one wheel session of robot customer service and user;
Current confidence assessment unit, for validity feature to be inputted into confidence assessment models, obtain robot customer service and user The current confidence assessed value of session;The robot that the confidence assessment models are manually put using being marked with validity feature and suitably going out The session sample of customer service and user are trained, described suitably to go out artificial point to replace the appropriate of robot customer service with artificial customer service Time point;
Artificial customer service adapter unit, for when current confidence assessed value meets to make artificial condition in advance, user to be transferred Artificial customer service.
From above technical scheme, in embodiments herein, using being marked with validity feature and suitably go out artificial point Robot customer service and user session sample training confidence assessment models, by the meeting of ongoing robot client and user After words are input to confidence assessment models, the current confidence assessed value exported according to model is currently artificial if appropriate for switching to judge Customer service, so as to identify that user turns the desirability of artificial customer service according to the actual conditions of current sessions, improving user While to the satisfaction of service, reduce the unnecessary work of artificial customer service, improve the efficiency of service of customer service system.
Brief description of the drawings
Fig. 1 is the flow chart of a kind of method that robot customer service turns artificial customer service in the embodiment of the present application;
Fig. 2 is a kind of hardware structure diagram for the equipment for running the embodiment of the present application;
Fig. 3 is a kind of building-block of logic of the device that robot customer service turns artificial customer service in the embodiment of the present application.
Embodiment
Embodiments herein proposes that a kind of new robot customer service turns the method for artificial customer service, in robot customer service with using Validity feature is manually marked in the session sample at family and suitably goes out artificial point, confidence assessment models are trained, and utilize instruction The validity feature of confidence assessment models and current sessions after white silk, is assessed current robot service quality, to find Transfer the suitable time point of artificial customer service, to determine artificial point according to the actual carry out situation of current sessions, can be lifted The experience of user, and can avoids the unnecessary live load of artificial customer service, so as to solve problems of the prior art.
Embodiments herein can be applied in any equipment with calculating and storage capacity, such as can be hand The physical equipments such as machine, tablet personal computer, PC (Personal Computer, PC), notebook, server, virtual machine are patrolled Collect equipment;The physically or logically equipment of different responsibilities can also be shared by two or more, is mutually cooperateed with to realize this Shen Various functions that please be in embodiment.
In embodiments herein, the flow that robot customer service turns the method for artificial customer service is as shown in Figure 1.
In embodiments herein, using machine learning techniques come assessment models of building up confidence, for robot customer service Assessed with the artificial customer service that whether should transfer of some time point during user conversation.Specifically, by a number of machine The conversation recording of device people customer service and user are as session sample, manually mark validity feature and each session on session sample Suitably go out artificial point in journey, confidence assessment models are trained using session sample.
Wherein, validity feature is the abstract table of the various factors related to artificial demand of transferring to occurring in session sample Reach.Validity feature can be generated, assessed by the way of artificial or artificial and data mining technology is combined.One kind is realized , can be by technical staff according to the working experience, the historical data of robot customer service and user conversation, Yong Huwen of itself in mode The key to exercises is determined situation etc. and drawn to summarize, refine.
, can be first by the artificial Service Quality that can describe robot customer service summarized, refined in another implementation Amount and the feature for turning artificial wish of user, by feature undetermined, suitably go out artificial point and actually go out artificial point as feature undetermined It is labeled on session sample, with reference to the service effectiveness of each session in session sample, is treated using predetermined data analysis algorithm Determine feature to be assessed and integrated, obtain, covering factor comprehensive several validity features obvious to artificial factors influencing demand of transferring. For example, can be with manual analysis robot client and the historical data of user conversation, therefrom excavating influences user to manual service The factor of demand;These factors are refined, sorted out, abstract to turn to feature undetermined, feature undetermined is from session context, industry The complete above-mentioned factor for describing the artificial demand of influence switching of each different aspects such as business, Consumer's Experience, service track;According to The problem of user feedback, user, solves situation and the satisfaction situation of user etc. to draw the clothes of each session in session sample Business effect, there is feature to mark using feature selecting, Feature Extraction Algorithm, suitably go out artificial point and actually go out what is manually put Session sample and service effectiveness are analyzed, and draw validity feature.
The various data analysis algorithms that can be used for carrying out tagsort and assessing may be used to generate validity feature, this Do not limited in the embodiment of application.In application scenes, used data analysis algorithm can not only be dialogue-based Sample and service effectiveness draw validity feature, additionally it is possible to provide the weight of influence of each validity feature to artificial demand of transferring; In these application scenarios, the weight of each validity feature comments confidence after the training for confidence assessment models, and/or training In the use for estimating model.
In the conversation procedure of robot customer service and user, time point of the artificial customer service generally after user makes a speech replaces machine Device people's customer service.In each session of session sample, if the time point after some user speech is to replace machine with artificial customer service The appropriate time point of device people's customer service, then it can be marked as suitably going out artificial point.There can be one to be fitted to multiple in one session Preferably go out artificial point.
Confidence assessment models are used for obtaining confidence assessed value based on robot customer service and the session of user, and confidence is assessed Value is estimate of the confidence assessment models to artificial desirability of transferring, in other words, should be by artificial visitor to certain time point in session The score value that clothes are got instead of the degree of robot customer service.In the embodiment of the present application to confidence assessment models used by it is specific Algorithm does not limit, for example, it may be SVR (Support Vector Regression, support vector regression) algorithm, LR (Logistic Regression, logistic regression) algorithm or GBDT (Gradient Boosting Decision Tree, change For decision tree) algorithm etc..
, can be in the conversation procedure of real-time robot customer service and user after the completion of the training of confidence assessment models, profit Judge the time point after user's speech in a session with confidence assessment models, if need to switch to artificial customer service.
Step 110, validity feature is obtained from robot customer service and at least one wheel session of user.
In embodiments herein, made a speech with user's speech or a robot customer service speech and a user For a round.The first round of usual robot customer service and the session of user is user's speech, and the second wheel and subsequent passes are one Secondary robot customer service speech and a user make a speech.Each round is terminated with user's speech, and the time point that is to say can be by people Work customer service replaces the time point of robot customer service.
The validity feature obtained in this step will be used as the input of confidence assessment models, for obtaining current time point Confidence assessed value.Can be using the session of predetermined number of iterations number before current time point as the basis for obtaining validity feature, if currently The session round that time point has been carried out is less than predetermined number of iterations number, then whole session is as the basis for obtaining validity feature;Also may be used To regard the whole session carried out as the basis for obtaining validity feature all the time;Do not limit.
In one implementation, directly can be taken turns to from the one of robot customer service and user in more wheel sessions, according to one Fixed Rule Extraction goes out validity feature.For example, to text session, semantic analysis, Keywords matching, business tine are referred to With etc. prior art the recognition rule of each validity feature is set, so as to obtain validity feature from above-mentioned session automatically;To language Sound session, text session first can be converted into using speech recognition technology, validity feature is being obtained using the above method.
In another implementation, first can be taken turns to from the one of robot customer service and user extracted in more wheel sessions it is original Feature, effective spy is obtained after being split, combined, classified, and/or deleted to primitive character according still further to feature preprocessing rule Sign.Similar, to text session, it may be referred to the prior arts such as semantic analysis, Keywords matching, business tine matching and set respectively The discovery rule of individual primitive character, primitive character is extracted so as to be taken turns to automatically from one in more wheel sessions;, can be with to voice conversation Voice conversation is first converted to by text session using speech recognition technology, original is being extracted using the discovery rule of primitive character Beginning feature.
Feature preprocessing rule describes the mapping relations from primitive character to validity feature, including following one kind is to more Kind:Which or which primitive character can be deleted, which primitive character can be split as which validity feature, which or which Primitive character can be attributed to represent the validity feature of some classification, which primitive character can be combined as which validity feature etc. Deng.
For example, in a kind of application scenarios, the primitive character in two dimensions of business and user feedback is as shown in table 1:
Table 1
In primitive character in table 1, if a reply of robot customer service is answer of revealing all the details, the answer of the reply Matching degree is not necessarily high (if robot customer service can inquire the higher answer of matching degree, with the answer rather than to reveal all the details Answer is multiplexed family back and forth), it is assumed that answer matches degree and answer of whether revealing all the details all are validity features, then can pre-process and advise in feature Such one can be included in then:If one reply can extract out whether reveal all the details answer primitive character and answer matches degree it is original Feature, then the two primitive characters are merged into answer primitive character of whether revealing all the details;In order to avoid the two validity features are used simultaneously Cause double influence of the same facts to confidence assessed value.
Because confidence assessment models are trained based on validity feature, therefore to the use of confidence assessment models after training Using validity feature as input.Due to the session sample for training confidence assessment models to need certain amount manually to mark, and reality is objective Family service in session case may because the increase of business, the change of operation flow, the change of active language and constantly change. In this implementation, the feature of generality and abstract can be used as validity feature, and according to business development and change Concrete condition set the primitive character, primitive character to find rule and feature preprocessing rule, be so not required to because actual industry The change of business situation constantly regenerates session sample and training confidence assessment models, and confidence still can be kept to assess mould The order of accuarcy of type.
Primitive character finds that rule and/or feature preprocessing rule can be solidificated in a manner of code and complete this step In program, it can also write in configuration file.To the application scenarios of configuration file be present, can first be obtained before this step is performed Configuration file, therefrom read primitive character and find rule and/or feature preprocessing rule, then be applied to primitive character extraction And/or obtain validity feature from primitive character.
It should be noted that primitive character can be selected in feature undetermined, will can also be not present in feature undetermined Other features do not limit as primitive character.
Step 120, validity feature is inputted into confidence assessment models, obtains the current confidence of robot customer service and user conversation Assessed value.
Step 130, current confidence assessed value meet it is pre- make artificial condition when, user is transferred artificial customer service.
It is after the validity feature of current time point is corresponded in getting robot customer service and user conversation, validity feature is defeated Enter in the confidence assessment models to after training, you can obtain the confidence assessed value of current time point.If current confidence assessed value expires Foot makes artificial condition in advance, then it is assumed that current time point needs artificial customer service intervention, and user is forwarded into artificial customer service.
It can be that current confidence assessed value is more than or less than predetermined confidence threshold value to make artificial condition in advance, depending on practical application field Current confidence assessed value more Gao Shi represents the artificial desirability of stronger switching, or the weaker artificial demand journey of switching in scape Degree.Predetermined confidence threshold value can be considered by technical staff the fitting degree of confidence assessment models and session sample after training, The factors such as the quantitative proportion of user conversation and artificial customer service in practical application scene determine.Can also be by setting certain mark Standard, predetermined confidence threshold value is automatically determined according to set standard by program.For example, after session sample being input into training Confidence assessment models in, obtain corresponding in session sample and suitably go out the sample confidence assessed value manually put;Set a series of The concrete numerical value of different predetermined confidence threshold values, calculate the sample confidence assessed value when selecting the predetermined confidence threshold value of different numerical value Coverage rate and accuracy rate, setting covers for the judgment criteria of coverage rate and accuracy rate according to the evaluation of judgment criteria is best Numerical value corresponding to lid rate and accuracy rate is as predetermined confidence threshold value.Wherein, coverage rate is suitably to go out artificial point pair in session sample In all sample confidence assessed values answered, meet to make artificial condition (concrete numerical value for being more than or less than predetermined confidence threshold value) in advance Sample confidence assessed value shared by ratio;Accuracy rate is that all satisfactions are made in the sample confidence assessed value of artificial condition in advance, Corresponding to the ratio suitably gone out shared by the sample confidence assessed value manually put.
In addition, in the application scenarios using configuration file, can also be by making artificial condition also writes configuration file in advance In, and the pre- artificial condition of making read from configuration file is applied to step 130.
It can be seen that in embodiments herein, validity feature is manually marked in the session sample of robot customer service and user Suitably go out artificial point, mould is assessed using the session sample training confidence assessment models after mark, and using the confidence after training Type, the validity feature in current session is assessed, the artificial visitor that whether should currently transfer is judged according to the output of model Clothes, so as to determine artificial point according to the actual carry out situation of current sessions, can lift the experience of user, and can is kept away The unnecessary live load of manpower-free's customer service, improve the efficiency of service of customer service system.
In the application example of the application, using the historical record of some robot customer services and the session of user as meeting Sample is talked about, by professional contact staff's analysis session sample, based on different dimension (including business dimension, Consumer's Experience dimension, clothes Track dimension of being engaged in etc.) analysis session sample, the factor for influenceing the artificial demand of switching is summarized, refined, is configured to spy undetermined Sign.
Feature undetermined is marked in session sample, suitably goes out artificial point and actually goes out artificial point, according to each in session sample The problem of user feedback of individual session, user, solves situation and the satisfaction situation of user, and construction is treated using session sample Determine feature and carry out data analysis, therefrom evaluate validity feature and draw the weight of validity feature.
Validity feature is marked in session sample and suitably goes out artificial point, using the session sample after mark to LR algorithm Confidence assessment models are trained.Session in session sample is input to the LR confidence assessment models after training, obtains session Correspond to the sample confidence assessed value for suitably going out and manually putting in sample, by technical staff according to LR confidence assessment models to session sample This coverage rate and accuracy rate, determines predetermined information threshold value.
Technical staff writes primitive character in configuration file and finds rule and feature preprocessing rule, and is stored in predetermined Position.This application example loads primitive character discovery rule and feature preprocessing rule after bringing into operation from configuration file.
After user starts the session with robot customer service, at the end of each user makes a speech, found according to primitive character Rule, all primitive characters are extracted in having conversated from session start to current time point, reapply feature pretreatment rule Then, be mapped as belonging to current time point by all primitive characters one arrives multiple validity features.
By the LR confidence assessment models after the validity feature input training for belonging to current time point, obtain current confidence and assess Value.If LR confidence assessment models represent that the artificial demand of switching is stronger so that the assessed value drawn is higher, then in current confidence assessed value During more than predetermined information threshold value, user is transferred to artificial customer service, otherwise engaged in the dialogue by robot customer service continuation with user.
Corresponding with the realization of above-mentioned flow, embodiments herein additionally provides the dress that a kind of robot customer service turns artificial customer service Put, the device can be realized by software, can also be realized by way of hardware or software and hardware combining.It is implemented in software to be Example, is the CPU (Central Process Unit, central processing unit) by place equipment as the device on logical meaning Corresponding computer program instructions are read what operation in internal memory was formed.For hardware view, except the CPU shown in Fig. 2, Outside internal memory and nonvolatile memory, the equipment where robot customer service turns the device of artificial customer service generally also includes being used for Carry out the chip etc. of wireless signal transmitting-receiving other hardware, and/or other hardware such as board for realizing network communicating function.
Fig. 3 show a kind of device of the artificial customer service of robot customer service turn of the embodiment of the present application offer, including effectively special Acquiring unit, current confidence assessment unit and artificial customer service adapter unit are levied, wherein:Validity feature acquiring unit is used for from machine Validity feature is obtained in people's customer service and at least one wheel session of user;Current confidence assessment unit, which is used to input validity feature, to be believed Heart assessment models, obtain the current confidence assessed value of robot customer service and user conversation;The confidence assessment models are using mark There is validity feature and suitably go out the robot customer service manually put and the session sample of user is trained, it is described suitably to go out artificial point To replace the appropriate time point of robot customer service with artificial customer service;Artificial customer service adapter unit is used to meet in current confidence assessed value It is pre- when making artificial condition, user is transferred artificial customer service.
In one example, the validity feature acquiring unit is specifically used for:From robot customer service and at least one wheel of user Primitive character is extracted in session, after being split, combined, classified, and/or deleted to primitive character according to feature preprocessing rule Obtain validity feature.
In above-mentioned example, described device also includes configuration file acquiring unit, for obtaining configuration file;The configuration text Part includes primitive character and finds rule and/or feature preprocessing rule;The validity feature acquiring unit is specifically used for:Slave In device people customer service and at least one wheel session of user, Rule Extraction primitive character is found according to primitive character.
Optionally, also include in the configuration file:Make artificial condition in advance.
In above-mentioned example, the primitive character includes:Business correlation, answer matches degree, answer number of repetition, whether pocket Whether bottom answer, answer are that enquirement, user clearly propose that substitution work and user have potential substitution work tendency, the emotion of user to incline It is at least one in the problem of explaining oneself to, user.
In a kind of implementation, the validity feature is by tentation data parser according to being marked with feature undetermined, suitable Go out the artificial session sample put and actually go out the robot manually put and user and the wherein service effectiveness of session, treated to some Determine feature to be assessed and drawn after being integrated;The feature undetermined can describe the service quality of robot customer service and turning for user Artificial wish.
In above-mentioned implementation, the validity feature has respective weight, is calculated by the tentation data parser Draw;The confidence assessment models are trained according to the weight of validity feature.
Optionally, the pre- artificial condition of making includes:Current confidence assessed value is more than or less than predetermined confidence threshold value;Institute Predetermined confidence threshold value is stated to be determined according to the coverage rate and accuracy rate of several sample confidence assessed values;The sample confidence assessed value For session sample is inputted after confidence assessment models into obtained output;The coverage rate is suitably to go out artificial point pair in session sample In all sample confidence assessed values answered, meet the pre- ratio made shared by the sample confidence assessed value of artificial condition;The standard True rate is made in the sample confidence assessed value of artificial condition in advance for all satisfactions, is commented corresponding to the sample confidence manually put suitably is gone out Ratio shared by valuation.
Optionally, the confidence assessment models use support vector regression SVR algorithms, logistic regression LR algorithm or iteration Decision tree GBDT algorithms.
The preferred embodiment of the application is the foregoing is only, not limiting the application, all essences in the application God any modification, equivalent substitution and improvements done etc., should be included within the scope of the application protection with principle.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net Network interface and internal memory.
Internal memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM).Internal memory is computer-readable medium Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data. The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moved State random access memory (DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electric erasable Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM), Digital versatile disc (DVD) or other optical storages, magnetic cassette tape, the storage of tape magnetic rigid disk or other magnetic storage apparatus Or any other non-transmission medium, the information that can be accessed by a computing device available for storage.Define, calculate according to herein Machine computer-readable recording medium does not include temporary computer readable media (transitory media), such as data-signal and carrier wave of modulation.
It should also be noted that, term " comprising ", "comprising" or its any other variant are intended to nonexcludability Comprising so that process, method, commodity or equipment including a series of elements not only include those key elements, but also wrapping Include the other element being not expressly set out, or also include for this process, method, commodity or equipment intrinsic want Element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that wanted including described Other identical element also be present in the process of element, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer program product. Therefore, the application can be using the embodiment in terms of complete hardware embodiment, complete software embodiment or combination software and hardware Form.Deposited moreover, the application can use to can use in one or more computers for wherein including computer usable program code The shape for the computer program product that storage media is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.) Formula.

Claims (18)

1. a kind of method that robot customer service turns artificial customer service, it is characterised in that including:
Validity feature is obtained from robot customer service and at least one wheel session of user;
Validity feature is inputted into confidence assessment models, obtains the current confidence assessed value of robot customer service and user conversation;It is described The use of confidence assessment models is marked with validity feature and suitably goes out the robot customer service manually put and the session sample progress of user Training, it is described suitably to go out artificial point to replace the appropriate time point of robot customer service with artificial customer service;
Current confidence assessed value meet it is pre- make artificial condition when, user is transferred artificial customer service.
2. according to the method for claim 1, it is characterised in that described from robot customer service and at least one wheel session of user Middle acquisition validity feature, including:Primitive character is extracted from robot customer service and at least one wheel session of user, it is pre- according to feature Processing rule obtains validity feature after being split, combined, classified, and/or being deleted to primitive character.
3. according to the method for claim 2, it is characterised in that methods described also includes:Obtain configuration file;The configuration File includes primitive character and finds rule and/or feature preprocessing rule;
It is described to extract primitive character from robot customer service and at least one wheel session of user, including:From robot customer service with using In at least one wheel session at family, Rule Extraction primitive character is found according to primitive character.
4. according to the method for claim 3, it is characterised in that also include in the configuration file:Make artificial condition in advance.
5. according to the method for claim 2, it is characterised in that the primitive character includes:Business correlation, answer matches Whether degree, answer number of repetition, answer of whether revealing all the details, answer are that enquirement, user clearly propose that substitution work, user have potential substitution Work tendency, the Sentiment orientation of user and user are at least one in the problem of explaining oneself.
6. according to the method for claim 1, it is characterised in that the validity feature is by tentation data parser according to mark Note has feature undetermined, suitably goes out artificial point and actually goes out the robot manually put and user session sample and wherein session Service effectiveness, drawn after some features undetermined are assessed and integrated;The feature undetermined can describe robot customer service Service quality and turn artificial wish of user.
7. according to the method for claim 6, it is characterised in that the validity feature has respective weight, by described pre- Determine data analysis algorithm to be calculated;
The confidence assessment models are trained according to the weight of validity feature.
8. according to the method for claim 1, it is characterised in that the pre- artificial condition of making includes:Current confidence is assessed Value is more than or less than predetermined confidence threshold value;
The predetermined confidence threshold value determines according to the coverage rate and accuracy rate of several sample confidence assessed values;The sample confidence Assessed value is that session sample is inputted after confidence assessment models to obtained output;The coverage rate is suitably to go out people in session sample In all sample confidence assessed values corresponding to work point, meet the pre- ratio made shared by the sample confidence assessed value of artificial condition; The accuracy rate is made in the sample confidence assessed value of artificial condition in advance for all satisfactions, corresponding to suitably going out the sample manually put Ratio shared by confidence assessed value.
9. according to the method for claim 1, it is characterised in that the confidence assessment models use support vector regression SVR Algorithm, logistic regression LR algorithm or iteration decision tree GBDT algorithms.
10. a kind of robot customer service turns the device of artificial customer service, it is characterised in that including:
Validity feature acquiring unit, for obtaining validity feature from least one wheel session of robot customer service and user;
Current confidence assessment unit, for validity feature to be inputted into confidence assessment models, obtain robot customer service and user conversation Current confidence assessed value;The robot customer service that the confidence assessment models are manually put using being marked with validity feature and suitably going out Be trained with the session sample of user, it is described suitably go out artificial point be with artificial customer service replace robot customer service it is appropriate when Point;
Artificial customer service adapter unit, for when current confidence assessed value meets to make artificial condition in advance, user being transferred artificial Customer service.
11. device according to claim 10, it is characterised in that the validity feature acquiring unit is specifically used for:Slave Primitive character is extracted in device people customer service and at least one wheel session of user, primitive character is torn open according to feature preprocessing rule Divide, obtain validity feature after combination, classification, and/or deletion.
12. device according to claim 11, it is characterised in that described device also includes:Configuration file acquiring unit, use In acquisition configuration file;The configuration file includes primitive character and finds rule and/or feature preprocessing rule;
The validity feature acquiring unit is specifically used for:From robot customer service and at least one wheel session of user, according to original Feature finds Rule Extraction primitive character.
13. device according to claim 12, it is characterised in that also include in the configuration file:Artificial bar is made in advance Part.
14. device according to claim 11, it is characterised in that the primitive character includes:Business correlation, answer Whether it is that enquirement, user clearly propose that substitution work, user have potential change with degree, answer number of repetition, answer of whether revealing all the details, answer Artificial tendency, the Sentiment orientation of user and user are at least one in the problem of explaining oneself.
15. device according to claim 10, it is characterised in that the validity feature by tentation data parser according to Feature undetermined is marked with, suitably goes out artificial point and actually goes out the session sample of the robot manually put and user and wherein session Service effectiveness, drawn after some features undetermined are assessed and integrated;The feature undetermined can describe robot customer service Service quality and user turn artificial wish.
16. device according to claim 15, it is characterised in that the validity feature has respective weight, by described Tentation data parser is calculated;
The confidence assessment models are trained according to the weight of validity feature.
17. device according to claim 10, it is characterised in that the pre- artificial condition of making includes:Current confidence is commented Valuation is more than or less than predetermined confidence threshold value;
The predetermined confidence threshold value determines according to the coverage rate and accuracy rate of several sample confidence assessed values;The sample confidence Assessed value is that session sample is inputted after confidence assessment models to obtained output;The coverage rate is suitably to go out people in session sample In all sample confidence assessed values corresponding to work point, meet the pre- ratio made shared by the sample confidence assessed value of artificial condition; The accuracy rate is made in the sample confidence assessed value of artificial condition in advance for all satisfactions, corresponding to suitably going out the sample manually put Ratio shared by confidence assessed value.
18. device according to claim 10, it is characterised in that the confidence assessment models use support vector regression SVR algorithms, logistic regression LR algorithm or iteration decision tree GBDT algorithms.
CN201610538674.6A 2016-07-08 2016-07-08 The method and apparatus that robot customer service turns artificial customer service Pending CN107590159A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201610538674.6A CN107590159A (en) 2016-07-08 2016-07-08 The method and apparatus that robot customer service turns artificial customer service
TW106118915A TWI677846B (en) 2016-07-08 2017-06-07 Method and device for transferring robot customer service to manual customer service
PCT/CN2017/090322 WO2018006727A1 (en) 2016-07-08 2017-06-27 Method and apparatus for transferring from robot customer service to human customer service

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610538674.6A CN107590159A (en) 2016-07-08 2016-07-08 The method and apparatus that robot customer service turns artificial customer service

Publications (1)

Publication Number Publication Date
CN107590159A true CN107590159A (en) 2018-01-16

Family

ID=60904586

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610538674.6A Pending CN107590159A (en) 2016-07-08 2016-07-08 The method and apparatus that robot customer service turns artificial customer service

Country Status (3)

Country Link
CN (1) CN107590159A (en)
TW (1) TWI677846B (en)
WO (1) WO2018006727A1 (en)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108363745A (en) * 2018-01-26 2018-08-03 阿里巴巴集团控股有限公司 The method and apparatus that robot customer service turns artificial customer service
CN108418981A (en) * 2018-03-16 2018-08-17 苏州思必驰信息科技有限公司 The method and device of dialogue service is provided for client
CN108712583A (en) * 2018-06-05 2018-10-26 上海木木机器人技术有限公司 A kind of manual service method and system based on robot
CN109308320A (en) * 2018-07-20 2019-02-05 北京智能点科技有限公司 Conversation process configuration method is taken turns more by a kind of robot
CN109446305A (en) * 2018-10-10 2019-03-08 长沙师范学院 The construction method and system of intelligent tour customer service system
CN109685550A (en) * 2018-12-04 2019-04-26 北京猎户星空科技有限公司 Smart machine control method, device, electronic equipment and storage medium
CN109787885A (en) * 2019-01-16 2019-05-21 中民乡邻投资控股有限公司 A kind of question and answer method of servicing
CN109887525A (en) * 2019-01-04 2019-06-14 平安科技(深圳)有限公司 Intelligent customer service method, apparatus and computer readable storage medium
CN109949830A (en) * 2019-03-12 2019-06-28 中国联合网络通信集团有限公司 User's intension recognizing method and equipment
WO2019201202A1 (en) * 2018-04-16 2019-10-24 华为技术有限公司 Call handling method and apparatus, server, storage medium, and system
CN110472023A (en) * 2019-07-10 2019-11-19 深圳追一科技有限公司 Customer service switching method, device, computer equipment and storage medium
CN110674385A (en) * 2018-06-14 2020-01-10 阿里巴巴集团控股有限公司 Method and device for matching customer service in customer service upgrading scene
WO2020034928A1 (en) * 2018-08-15 2020-02-20 深圳追一科技有限公司 Method and system for switching customer service session, and storage medium
CN110838014A (en) * 2019-09-20 2020-02-25 北京智齿博创科技有限公司 Artificial intelligent routing strategy in on-line customer service system
CN110990553A (en) * 2019-12-18 2020-04-10 上海智勘科技有限公司 Coupling method and system of intelligent sound box voice interaction system and insurance agent
CN111143537A (en) * 2019-12-30 2020-05-12 税友软件集团股份有限公司 Service method, device, equipment and medium based on intelligent customer service system
CN111405129A (en) * 2020-03-12 2020-07-10 中国建设银行股份有限公司 Intelligent outbound risk monitoring method and device
CN111510563A (en) * 2020-04-16 2020-08-07 中国银行股份有限公司 Intelligent outbound method and device, storage medium and electronic equipment
CN111526253A (en) * 2020-03-09 2020-08-11 深圳追一科技有限公司 Call control method, device, computer equipment and storage medium
CN111754061A (en) * 2019-11-22 2020-10-09 北京沃东天骏信息技术有限公司 Method and device for controlling man-machine distribution, server equipment and storage medium
CN111985248A (en) * 2020-06-30 2020-11-24 联想(北京)有限公司 Information interaction method and device
CN112580917A (en) * 2019-09-30 2021-03-30 深圳无域科技技术有限公司 Client characteristic evaluation method and device
CN113886547A (en) * 2021-09-29 2022-01-04 平安银行股份有限公司 Client real-time conversation switching method and device based on artificial intelligence and electronic equipment

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110232573A (en) * 2018-03-06 2019-09-13 广州供电局有限公司 Based on interactive intelligent response system
CN111625918B (en) * 2019-02-27 2023-05-09 阿里巴巴集团控股有限公司 Technological parameter recommendation method and device and electronic equipment
CN110163655B (en) * 2019-04-15 2024-03-05 中国平安人寿保险股份有限公司 Agent distribution method, device and equipment based on gradient lifting tree and storage medium
CN110457578B (en) * 2019-07-11 2023-07-18 创新先进技术有限公司 Customer service demand identification method and device
CN112433598B (en) * 2019-08-07 2024-06-04 科沃斯商用机器人有限公司 Man-machine interaction method, device and storage medium
CN110602335B (en) * 2019-08-14 2022-11-18 中国平安财产保险股份有限公司 Method and device for setting terminal priority, computer equipment and storage medium
TWI751504B (en) * 2020-02-27 2022-01-01 中華電信股份有限公司 Dialogue system and method for human-machine cooperation
CN111583023A (en) * 2020-05-07 2020-08-25 中国工商银行股份有限公司 Service processing method, device and computer system
CN112543185B (en) * 2020-11-23 2022-09-06 建信金融科技有限责任公司 Customer service method, device and system
CN113051386B (en) * 2021-04-30 2024-02-02 中国银行股份有限公司 Adjustment method and device for transferring manual service, electronic equipment and storage medium
CN113315876B (en) * 2021-05-27 2023-01-20 中国银行股份有限公司 Telephone bank service control method, device, server and storage medium
CN113239170B (en) * 2021-06-01 2023-11-28 平安科技(深圳)有限公司 Interaction generating method, device, equipment and medium based on mutual role perception
CN113434630B (en) * 2021-06-25 2023-07-25 平安科技(深圳)有限公司 Customer service evaluation method, customer service evaluation device, terminal equipment and medium
CN113887246A (en) * 2021-10-19 2022-01-04 京东科技信息技术有限公司 Method and device for detecting consistency of man-machine schemes in customer service field and storage medium
CN114117157B (en) * 2021-11-19 2024-04-09 招联消费金融股份有限公司 Session processing method, apparatus, computer device and storage medium
CN118013009B (en) * 2024-02-19 2024-06-18 广东融益数字科技有限公司 Intelligent customer service interaction method and system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013019200A1 (en) * 2011-07-31 2013-02-07 Hewlett-Packard Development Company, L.P. Systems and methods of knowledge transfer
CN105591882A (en) * 2015-12-10 2016-05-18 北京中科汇联科技股份有限公司 Method and system for mixed customer services of intelligent robots and human beings
CN105592237A (en) * 2014-10-24 2016-05-18 ***通信集团公司 Method and apparatus for session switching, and intelligent customer service robot
CN105701088A (en) * 2016-02-26 2016-06-22 北京京东尚科信息技术有限公司 Method and device for switching machine conversation to artificial conversation

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8379830B1 (en) * 2006-05-22 2013-02-19 Convergys Customer Management Delaware Llc System and method for automated customer service with contingent live interaction
TWI546768B (en) * 2011-12-13 2016-08-21 詹偉強 Customer service device with automatic recognition function for multiple websites
CN104991887B (en) * 2015-06-18 2018-01-19 北京京东尚科信息技术有限公司 The method and device of information is provided
CN105072173A (en) * 2015-08-03 2015-11-18 谌志群 Customer service method and system for automatically switching between automatic customer service and artificial customer service

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013019200A1 (en) * 2011-07-31 2013-02-07 Hewlett-Packard Development Company, L.P. Systems and methods of knowledge transfer
CN105592237A (en) * 2014-10-24 2016-05-18 ***通信集团公司 Method and apparatus for session switching, and intelligent customer service robot
CN105591882A (en) * 2015-12-10 2016-05-18 北京中科汇联科技股份有限公司 Method and system for mixed customer services of intelligent robots and human beings
CN105701088A (en) * 2016-02-26 2016-06-22 北京京东尚科信息技术有限公司 Method and device for switching machine conversation to artificial conversation

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108363745A (en) * 2018-01-26 2018-08-03 阿里巴巴集团控股有限公司 The method and apparatus that robot customer service turns artificial customer service
JP6991341B2 (en) 2018-01-26 2022-01-12 アドバンスド ニュー テクノロジーズ カンパニー リミテッド Methods and equipment for the transition from robotic customer service to human customer service
JP2021524071A (en) * 2018-01-26 2021-09-09 アドバンスド ニュー テクノロジーズ カンパニー リミテッド Methods and equipment for the transition from robotic customer service to human customer service
US10977664B2 (en) 2018-01-26 2021-04-13 Advanced New Technologies Co., Ltd. Method and apparatus for transferring from robot customer service to human customer service
TWI698830B (en) * 2018-01-26 2020-07-11 香港商阿里巴巴集團服務有限公司 Method and device for transferring robot customer service to manual customer service, computer equipment and computer readable storage medium
CN108363745B (en) * 2018-01-26 2020-06-30 阿里巴巴集团控股有限公司 Method and device for changing robot customer service into manual customer service
WO2019144773A1 (en) * 2018-01-26 2019-08-01 阿里巴巴集团控股有限公司 Method and apparatus for transferring from robot customer service to human customer service
CN108418981A (en) * 2018-03-16 2018-08-17 苏州思必驰信息科技有限公司 The method and device of dialogue service is provided for client
CN110519472A (en) * 2018-03-16 2019-11-29 苏州思必驰信息科技有限公司 The method and device of dialogue service is provided for client
CN108418981B (en) * 2018-03-16 2019-10-29 苏州思必驰信息科技有限公司 The method and device of dialogue service is provided for client
US11240374B2 (en) 2018-04-16 2022-02-01 Huawei Technologies Co., Ltd. Call processing method and apparatus, server, storage medium, and system
WO2019201202A1 (en) * 2018-04-16 2019-10-24 华为技术有限公司 Call handling method and apparatus, server, storage medium, and system
CN110392168A (en) * 2018-04-16 2019-10-29 华为技术有限公司 Call processing method, device, server, storage medium and system
CN108712583B (en) * 2018-06-05 2021-03-23 上海智蕙林医疗科技有限公司 Robot-based manual service method and system
CN108712583A (en) * 2018-06-05 2018-10-26 上海木木机器人技术有限公司 A kind of manual service method and system based on robot
CN110674385A (en) * 2018-06-14 2020-01-10 阿里巴巴集团控股有限公司 Method and device for matching customer service in customer service upgrading scene
CN109308320A (en) * 2018-07-20 2019-02-05 北京智能点科技有限公司 Conversation process configuration method is taken turns more by a kind of robot
WO2020034928A1 (en) * 2018-08-15 2020-02-20 深圳追一科技有限公司 Method and system for switching customer service session, and storage medium
CN109446305A (en) * 2018-10-10 2019-03-08 长沙师范学院 The construction method and system of intelligent tour customer service system
CN109685550A (en) * 2018-12-04 2019-04-26 北京猎户星空科技有限公司 Smart machine control method, device, electronic equipment and storage medium
CN109887525A (en) * 2019-01-04 2019-06-14 平安科技(深圳)有限公司 Intelligent customer service method, apparatus and computer readable storage medium
CN109787885A (en) * 2019-01-16 2019-05-21 中民乡邻投资控股有限公司 A kind of question and answer method of servicing
CN109949830A (en) * 2019-03-12 2019-06-28 中国联合网络通信集团有限公司 User's intension recognizing method and equipment
CN109949830B (en) * 2019-03-12 2021-03-30 中国联合网络通信集团有限公司 User intention identification method and device
CN110472023A (en) * 2019-07-10 2019-11-19 深圳追一科技有限公司 Customer service switching method, device, computer equipment and storage medium
CN110838014A (en) * 2019-09-20 2020-02-25 北京智齿博创科技有限公司 Artificial intelligent routing strategy in on-line customer service system
CN112580917B (en) * 2019-09-30 2024-04-05 深圳无域科技技术有限公司 Customer characteristic evaluation method and device
CN112580917A (en) * 2019-09-30 2021-03-30 深圳无域科技技术有限公司 Client characteristic evaluation method and device
CN111754061A (en) * 2019-11-22 2020-10-09 北京沃东天骏信息技术有限公司 Method and device for controlling man-machine distribution, server equipment and storage medium
CN110990553A (en) * 2019-12-18 2020-04-10 上海智勘科技有限公司 Coupling method and system of intelligent sound box voice interaction system and insurance agent
CN111143537A (en) * 2019-12-30 2020-05-12 税友软件集团股份有限公司 Service method, device, equipment and medium based on intelligent customer service system
CN111526253A (en) * 2020-03-09 2020-08-11 深圳追一科技有限公司 Call control method, device, computer equipment and storage medium
CN111405129A (en) * 2020-03-12 2020-07-10 中国建设银行股份有限公司 Intelligent outbound risk monitoring method and device
CN111510563A (en) * 2020-04-16 2020-08-07 中国银行股份有限公司 Intelligent outbound method and device, storage medium and electronic equipment
CN111985248A (en) * 2020-06-30 2020-11-24 联想(北京)有限公司 Information interaction method and device
CN113886547A (en) * 2021-09-29 2022-01-04 平安银行股份有限公司 Client real-time conversation switching method and device based on artificial intelligence and electronic equipment

Also Published As

Publication number Publication date
TW201804420A (en) 2018-02-01
TWI677846B (en) 2019-11-21
WO2018006727A1 (en) 2018-01-11

Similar Documents

Publication Publication Date Title
CN107590159A (en) The method and apparatus that robot customer service turns artificial customer service
TWI698830B (en) Method and device for transferring robot customer service to manual customer service, computer equipment and computer readable storage medium
US20190122146A1 (en) Dynamic and Intuitive Aggregation of a Training Dataset
CN108763445A (en) Construction method, device, computer equipment and the storage medium in patent knowledge library
CN109784414A (en) Customer anger detection method, device and electronic equipment in a kind of phone customer service
CN107071193A (en) The method and apparatus of interactive answering system accessing user
CN110580516B (en) Interaction method and device based on intelligent robot
CN112015896B (en) Emotion classification method and device based on artificial intelligence
CN108595609A (en) Generation method, system, medium and equipment are replied by robot based on personage IP
CN115374259A (en) Question and answer data mining method and device and electronic equipment
CN108829777A (en) A kind of the problem of chat robots, replies method and device
CN113204643B (en) Entity alignment method, device, equipment and medium
CN111783473B (en) Method and device for identifying best answer in medical question and answer and computer equipment
CN110069686A (en) User behavior analysis method, apparatus, computer installation and storage medium
CN116028613B (en) General knowledge question answering method, system, computer device and storage medium
US20230351121A1 (en) Method and system for generating conversation flows
CN112381777A (en) Image processing method and device, electronic equipment and storage medium
CN114048319B (en) Humor text classification method, device, equipment and medium based on attention mechanism
CN113378826B (en) Data processing method, device, equipment and storage medium
Tripathi et al. Facial expression recognition using data mining algorithm
JP7111017B2 (en) Paralinguistic information estimation model learning device, paralinguistic information estimation device, and program
JP2018092347A (en) Information processing apparatus, information processing method and program
CN109885687A (en) A kind of sentiment analysis method, apparatus, electronic equipment and the storage medium of text
Ali et al. Identifying and Profiling User Interest over time using Social Data
CN111241236A (en) Task-oriented question-answering method, system, electronic equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1249205

Country of ref document: HK

TA01 Transfer of patent application right

Effective date of registration: 20191211

Address after: P.O. Box 31119, grand exhibition hall, hibiscus street, 802 West Bay Road, Grand Cayman, Cayman Islands

Applicant after: Innovative advanced technology Co., Ltd

Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands

Applicant before: Alibaba Group Holding Co., Ltd.

TA01 Transfer of patent application right
RJ01 Rejection of invention patent application after publication

Application publication date: 20180116

RJ01 Rejection of invention patent application after publication