CN111565254B - Call data quality inspection method and device, computer equipment and storage medium - Google Patents
Call data quality inspection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111565254B CN111565254B CN202010672539.7A CN202010672539A CN111565254B CN 111565254 B CN111565254 B CN 111565254B CN 202010672539 A CN202010672539 A CN 202010672539A CN 111565254 B CN111565254 B CN 111565254B
- Authority
- CN
- China
- Prior art keywords
- call
- quality
- call data
- tag
- quality inspection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/22—Arrangements for supervision, monitoring or testing
- H04M3/2227—Quality of service monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Telephonic Communication Services (AREA)
Abstract
The application relates to a call data quality inspection method, a call data quality inspection device, a computer device and a storage medium. The quality inspection efficiency can be improved.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for quality inspection of call data, a computer device, and a storage medium.
Background
With the development of information technology, the voice robot has been widely applied in many fields, for example, in the aspect of customer service, the voice robot can provide services such as autonomous online question answering, consultation and instruction execution through natural and smooth human-computer interaction.
When man-machine interaction is carried out, the voice robot can automatically carry out outbound to the user. In order to improve the quality and efficiency of the outbound call of the voice robot, the call mode and content of the outbound call of the voice robot need to be optimized in a mode of quality inspection of call data. In the prior art, the call data generated by the automatic call-out of the voice robot is manually subjected to quality inspection, and the call with problems is marked and optimized.
However, the voice robot has high outbound efficiency, the number of call data generated every day is tens of thousands, the number of call data of each person for manual quality inspection every day is limited, and the complete quality inspection of all call data is difficult to complete, so that the call data quality inspection efficiency is low.
Disclosure of Invention
In view of the above, it is desirable to provide a call data quality inspection method, a call data quality inspection device, a computer device, and a storage medium, which can improve call data quality inspection efficiency.
In a first aspect, an embodiment of the present application provides a call data quality inspection method, where the method includes:
acquiring current call data to be inspected;
analyzing the characteristics of the call data to be quality checked according to a preset call characteristic rule to obtain a corresponding call label; the call tag comprises an abnormal tag and a normal tag;
if the call tag of the call data to be quality-checked is an abnormal tag, outputting the call data to be quality-checked to a quality-check area; the call data to be quality checked is stored in the quality check area.
In one embodiment, after outputting the call data to be quality-checked to the quality-check area, the method further comprises:
performing quality inspection on call data in a quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result includes the reason of the abnormal call data.
In one embodiment, the exception tags include at least one of a cross-voice tag, a too-fast-speech tag, a low-confidence tag, and a too-much-rejected tag.
In one embodiment, the analyzing the characteristics of the call data to be quality-checked according to the preset call characteristic rule to obtain the corresponding call tag includes:
analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule;
and if the voice cross time exceeds a preset time threshold, determining that the call tag of the call data to be subjected to quality inspection is the voice cross tag.
In one embodiment, the analyzing the characteristics of the call data to be quality-checked according to the preset call characteristic rule to obtain the corresponding call tag includes:
analyzing the speech rate of a user in the call data to be quality checked according to the call characteristic rule;
and if the speech rate exceeds a preset speech rate threshold value, determining the call tag of the call data to be subjected to quality inspection as an over-speech rate tag.
In one embodiment, the analyzing the characteristics of the call data to be quality-checked according to the preset call characteristic rule to obtain the corresponding call tag includes:
according to the call characteristic rule, obtaining the confidence coefficient of the call data to be quality checked, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot;
and if the confidence coefficient is lower than a preset threshold value, determining the call label of the call data to be subjected to the quality inspection as a low confidence coefficient label.
In one embodiment, the analyzing the characteristics of the call data to be quality-checked according to the preset call characteristic rule to obtain the corresponding call tag includes:
analyzing the times of rejecting user semantics by the voice robot in the call data to be quality checked according to the call characteristic rule;
and if the times of rejecting the user semantics exceed a preset rejection time threshold, determining the call label of the call data to be quality tested as an excessive rejection label.
In a second aspect, an embodiment of the present application provides a call data quality inspection device, where the call data quality inspection device includes:
the acquisition module is used for acquiring current call data to be quality checked;
the tag determination module is used for analyzing the characteristics of the call data to be quality tested according to a preset call characteristic rule to obtain a corresponding call tag; the call tag comprises an abnormal tag and a normal tag;
the quality inspection module is used for outputting the call data to be inspected to a quality inspection area if the call tag of the call data to be inspected is an abnormal tag; the call data to be quality checked is stored in the quality check area.
In one embodiment, the quality inspection module is configured to perform quality inspection on call data in a quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result includes the reason of the abnormal call data.
In one embodiment, the exception tags include at least one of a cross-voice tag, a too-fast-speech tag, a low-confidence tag, and a too-much-rejected tag.
In one embodiment, the tag determination module includes:
the voice cross analysis unit is used for analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule;
and the voice cross tag determining unit is used for determining that the call tag of the call data to be quality checked is the voice cross tag if the voice cross time exceeds a preset time threshold.
In one embodiment, the tag determination module includes:
the speech rate analysis unit is used for analyzing the speech rate of the user in the call data to be subjected to quality inspection according to the call characteristic rule;
and the speech rate tag determining unit is used for determining the call tag of the call data to be subjected to quality inspection as an over-speech rate tag if the speech rate exceeds a preset speech rate threshold value.
In one embodiment, the tag determination module includes:
the confidence coefficient acquisition unit is used for acquiring the confidence coefficient of the call data to be quality-checked according to the call characteristic rule, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot;
and the confidence degree label unit is used for determining the call label of the call data to be inspected as a low confidence degree label if the confidence degree is lower than a preset threshold value.
In one embodiment, the tag determination module includes:
the rejection frequency acquisition unit is used for analyzing the frequency of rejecting the user semantics by the voice robot in the call data to be quality tested according to the call characteristic rule;
and the rejection label unit is used for determining the call label of the call data to be quality checked as the excessive rejection label if the number of times of rejecting the user semantics exceeds a preset rejection number threshold.
In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of any one of the methods provided in the foregoing first aspect when executing the computer program.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of any one of the methods provided in the embodiments of the first aspect.
According to the call data quality inspection method, the call data quality inspection device, the computer equipment and the storage medium, the current call data to be quality inspected is obtained, the characteristics of the call data to be quality inspected are analyzed according to the preset call characteristic rule, a corresponding call label is obtained, and if the call label of the call data to be quality inspected is an abnormal label, the call data to be quality inspected is output to a quality inspection area for storing the call data to be quality inspected. When the conversation data that faces a large amount of needs quality control, carry out pronunciation feature analysis and call label to each conversation data earlier automatically, tentatively select the conversation data of unusual label, then export unusual conversation data to the quality control region and carry out the quality control, through tentatively screening like this, only need carry out the quality control to the conversation data that have unusual label, make the conversation data quantity of treating the quality control less, the quality control efficiency has been improved, and can carry out the quality control of pertinence according to unusual label at the quality control in-process, instruct the quality control direction with unusual label, the quality control efficiency has further been improved.
Drawings
FIG. 1a is a diagram of an exemplary embodiment of a call data quality inspection method;
FIG. 1b is a diagram of the internal structure of a telephone robot in one embodiment;
FIG. 2 is a flow chart illustrating a call data quality inspection method according to an embodiment;
FIG. 3 is a flow chart illustrating a call data quality inspection method according to an embodiment;
FIG. 4 is a flow chart illustrating a method for quality inspection of call data according to an embodiment;
FIG. 5 is a flow chart illustrating a method for quality inspection of call data according to an embodiment;
FIG. 6 is a flow chart illustrating a method for quality inspection of call data according to an embodiment;
FIG. 7 is a flow chart of a call data quality inspection method in one embodiment;
FIG. 8 is a block diagram showing the structure of a call data quality inspection apparatus according to an embodiment;
FIG. 9 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1a, the present application provides an application environment of a call data quality inspection method, where a voice robot 01 may perform voice interaction with a user, for example, in a scenario of telemarketing, question answering, consultation, instruction execution, and the like. The voice robot 01 includes, but is not limited to, a robot of a plurality of service types, such as an outbound robot, a chat robot, an intelligent customer service, and an intelligent assistant. The internal structure of the voice robot can be seen in fig. 1b, and the voice robot includes a processor, a memory, a network interface and a database which are connected through a system bus. Wherein the processor is configured to provide computational and control capabilities. The memory comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database is used for storing relevant data of call data quality inspection. The network interface is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a call data quality inspection method. It is understood that the internal structure of the voice robot shown in fig. 1b is only an example and is not intended to be limiting.
The embodiment of the application provides a call data quality inspection method and device, computer equipment and a storage medium, and the call data quality inspection efficiency can be improved. The following describes in detail the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems by embodiments and with reference to the drawings. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. In the call data quality inspection method provided by the present application, the main execution body in fig. 2 to 7 is a voice robot. The execution main bodies in fig. 2 to 7 may also be a call data quality inspection device, which may be implemented as part or all of a voice robot by software, hardware, or a combination of software and hardware.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments.
In an embodiment, as shown in fig. 2, a call data quality inspection method is provided, which is applied to the voice robot in fig. 1a for explanation by way of example, where the embodiment relates to a specific process in which the voice robot analyzes current call data to be quality inspected according to a preset call characteristic rule, obtains a call tag corresponding to the call data to be quality inspected, and outputs the call data to be quality inspected to a quality inspection area if the call tag of the call data to be quality inspected is an abnormal tag, and the embodiment includes the following steps:
s101, obtaining current call data to be inspected.
The call data refers to a call record generated when the robot calls out the call, for example, a data record generated in the process of the call with the user a after the voice robot calls out to the user a in the scenes of telemarketing, question answering, consultation and the like. In practical application, when the voice robot calls out, a great amount of call data can be generated, quality inspection can be performed on each piece of call data, and then the current call data to be subjected to quality inspection refers to the call data which needs to be subjected to quality inspection currently.
The voice robot can acquire the call data to be quality tested from a storage module which stores all the call data to be quality tested, or the storage module sends the call data to be quality tested at regular time; if the call data is stored in the external device, the voice robot acquires the call data to be subjected to the quality inspection from the external device, and the acquisition mode is not limited in the embodiment of the application.
S102, analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule to obtain a corresponding call label; the call tag includes an abnormal tag and a normal tag.
The preset call characteristic rule refers to a rule which is established in advance and used for analyzing call data characteristics, namely characteristics existing in a call process of the voice robot and a user, such as whether voice cross exists, information (size, tone, emotion, speed and the like) related to voices of two parties, recognition conditions of the voice robot to the voice of the user and the like. The call tag is a tag which is divided correspondingly to each piece of call data and comprises an abnormal tag and a normal tag, and if the call data correspond to the abnormal tag, the call tag refers to the condition that the characteristics of the call data are abnormal and do not meet the requirements after the characteristics of the call data are analyzed; if the call data corresponds to the normal label, the call data conforms to the requirements after the characteristics of the call data are analyzed, and no abnormal condition exists.
In practical application, after the voice robot acquires the current call data to be quality-checked, analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule, for example, if the call characteristic rule defines that the call characteristics include whether voice cross exists, analyzing whether voice cross exists between the voice robot and a user voice stream in the call data to be quality-checked; or, if the call characteristics defined in the call characteristic rule include that the user speech rate is too fast, analyzing whether the user speech rate is too fast in the call data to be subjected to quality inspection; or, if the call characteristics defined in the call characteristic rule include that the user semantics are not recognized, analyzing whether the voice robot in the call data to be subjected to the quality inspection has the condition that the user semantics are not recognized. After analyzing the characteristics of the call data to be quality-checked, if the characteristics of the call data to be quality-checked do not meet the requirements, determining that the call tag is an abnormal tag, such as a voice cross tag, a too-fast-speech tag, a too-recognition rejected tag, and the like, otherwise, determining that the call tag is a normal tag.
S103, if the call tag of the call data to be quality-checked is an abnormal tag, outputting the call data to be quality-checked to a quality-check area; the call data to be quality checked is stored in the quality check area.
And if the call tag of the call data to be quality-checked is an abnormal tag, the voice robot outputs the call data to be quality-checked to a quality-checking area. The quality inspection area stores call data needing quality inspection, namely call data which are output to the quality inspection area and are all abnormal labels, and the abnormal labels need to be comprehensively detected, for example, deep analysis and targeted quality inspection can be performed on the basis of the abnormal labels to detect problems in the call data; specifically, the specific reasons of the abnormal labels are deeply analyzed, for example, if the abnormal labels are too many rejected labels, the reasons of the excessive rejected labels are that the semantic database data of the voice robot is insufficient, or the abnormal labels are caused by the accent of the user; if the abnormal label is a voice cross label, deeply analyzing whether the voice robot calls or the user calls or the network delay causes the voice cross label; after specific reasons are deeply analyzed, quality inspection is carried out according to the analyzed abnormal reasons in a targeted manner during quality inspection, for example, the abnormal reason is voice crossover caused by network delay, the quality of network data transmission used by the voice robot and the user in a call is inspected in the targeted manner, if the network data transmission has problems, the problems need to be solved, and therefore, according to the specific problems and the reasons for generating the problems, the interaction between the voice robot and the user during the outbound call is improved, and the outbound call quality of the voice robot is improved.
According to the call data quality inspection method provided by the embodiment of the application, the current call data to be quality inspected is obtained, the characteristics of the call data to be quality inspected are analyzed according to the preset call characteristic rule, a corresponding call label is obtained, and if the call label of the call data to be quality inspected is an abnormal label, the call data to be quality inspected is output to a quality inspection area for storing the call data to be quality inspected. When the conversation data that faces a large amount of needs quality control, carry out pronunciation feature analysis and call label to each conversation data earlier automatically, tentatively select the conversation data of unusual label, then export unusual conversation data to the quality control region and carry out the quality control, through tentatively screening like this, only need carry out the quality control to the conversation data that have unusual label, make the conversation data quantity of treating the quality control less, the quality control efficiency has been improved, and can carry out the quality control of pertinence according to unusual label at the quality control in-process, instruct the quality control direction with unusual label, the quality control efficiency has further been improved.
On the basis of the requirement of improving the quality inspection accuracy while improving the efficiency of the quality inspection process, an embodiment is provided, and after the call data to be inspected are output to the quality inspection area, the method further comprises the following steps: performing quality inspection on call data in a quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result includes the reason of the abnormal call data.
For example, a neural network model may be trained by using a deep learning technique, and after the trained quality inspection model is obtained, the call data (including the abnormal labels) in the quality inspection area may be sequentially detected by the quality inspection model to detect the specific abnormal reasons of the call data, and the specific abnormal reasons of the call data include but are not limited to: voice crossover, too fast speech, low confidence, too much misrecognition, etc. The embodiment performs quality inspection on the call data in the quality inspection area through a special quality inspection model, so that the efficiency of the quality inspection process can be improved, and the accuracy of the quality inspection reason is also improved.
In the following, specific abnormal tags are taken as examples, and a detailed description is provided for a specific process of analyzing the call data to be quality-checked by the voice robot to obtain corresponding call tags in several embodiments. Optionally, the abnormal label comprises at least one of a voice cross label, a too fast voice label, a low confidence label, and a too many rejected labels. The voice cross-finger voice robot and the voice stream of the user have an intersection, and the specific expression is that the voice robot and the user speak simultaneously in the telephone conversation process. The too fast speaking speed means that the speaking speed of a user is too fast, the speed of speech of a general user can be increased when the emotion is excited, and the side face reflects the problem of the voice robot. The low confidence coefficient indicates that the voice robot scores a lower score for the semantic recognition result of the voice of the user, that is, the score is lower, which indicates that the voice robot has a misunderstanding of the user semantic. The fact that the voice robot cannot recognize the semantics of the user is referred to as the fact that the voice robot cannot recognize the semantics of the user, and the fact that the voice robot cannot recognize the voice of the user at all is referred to as the fact that the voice robot cannot recognize the voice of the user.
In one embodiment, illustrating voice crossover, as shown in fig. 3, one embodiment of S102 described above includes the steps of:
s201, analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule.
The preset call feature rule defines a plurality of feature detections, and in this embodiment, the detected feature is a corresponding voice crossover feature in the call rule. That is, the voice robot analyzes the voice crossing duration of the voice flow of the voice robot and the user in the call data to be quality tested, for example, each time node of respective speaking of the voice robot and the user is determined, then comparison is performed, the overlapping part in the voices of both sides is exceeded, and the duration of the overlapping part is obtained.
S202, if the voice cross time exceeds a preset time threshold, determining that the call tag of the call data to be quality checked is the voice cross tag.
In the conversation rule, the voice crossing duration can be preset with the duration of a voice crossing part, and after the duration exceeds a set duration threshold, the voice crossing duration is regarded as abnormal. Therefore, if the voice cross time of the voice flow of the voice robot and the user acquired by the voice robot exceeds the preset time threshold set in the rule, for example, 2s, the voice robot determines that the call data to be subjected to the quality inspection is abnormal, and marks a voice cross tag on the call data.
In the embodiment, the voice crossing time of the voice flow of the voice robot and the user in the call data is detected, when the voice crossing time of the two parties exceeds the preset time threshold, that is, the situation is considered that the voice robot and the user speak simultaneously in the telephone exceeds the threshold, and the abnormal situation is detected by using the voice crossing time as a detection basis, so that the accuracy of abnormal detection is improved.
In an embodiment, the speech rate of the user is explained, and as shown in fig. 4, an embodiment of the S102 includes the following steps:
s301, analyzing the speech rate of the user in the call data to be quality tested according to the call characteristic rule.
The speech robot analyzes the speech rate of the user in the call data to be quality checked according to the speech rate features in the call feature rule, for example, the speech rate of the user can be obtained by obtaining the number of phonemes in the speech of the user and the corresponding time, wherein the speech rates in different time periods can be obtained, so that the speech rate can be analyzed by combining the specific semantic content of the user in each time period when the abnormal reason of the speech rate of the user is analyzed subsequently.
And S302, if the speech rate exceeds a preset speech rate threshold value, determining the call tag of the call data to be subjected to quality inspection as an over-speech rate tag.
Similarly, a speech rate threshold is set in the call characteristic rule, and if the speech rate threshold is exceeded, the call characteristic rule is regarded as abnormal. If the voice robot analyzes that the speech rate of the user exceeds a preset speech rate threshold value, for example, 10 speech rates per second, and the speech rate is abnormal, it is determined that the call tag (abnormal tag) of the call data to be subjected to the quality inspection is an over-speech rate tag.
In the embodiment, by detecting the speech rate of the user in the call data to be quality checked, when the speech rate of the user exceeds a preset speech rate threshold, that is, the speech rate of the user is too fast, it is determined that the emotion of the user is excited, and the situation is abnormal. The speech speed of the user is used as a detection basis, and the accuracy of anomaly detection is improved.
In an embodiment, the confidence of the call data is explained, and as shown in fig. 5, an embodiment of the S102 includes the following steps:
s401, according to the call characteristic rule, obtaining the confidence coefficient of the call data to be quality checked, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot.
The voice robot analyzes the confidence of the call data to be quality-checked according to the confidence feature in the call feature rule, for example, the voice robot scores the semantic recognition result of the user, and determines the confidence according to the scoring result, for example, if the voice robot scores 0.4, the confidence is 0.4, and certainly, a range may be set, for example, the voice robot scores between 0 and 1.
S402, if the confidence coefficient is lower than a preset threshold value, determining that the call label of the call data to be subjected to quality inspection is a low confidence coefficient label.
And setting a confidence threshold in the call characteristic rule, and determining that the call characteristic rule is abnormal if the confidence threshold is exceeded. If the confidence coefficient of the call data to be quality-checked exceeds a preset confidence coefficient threshold value, for example, the confidence coefficient threshold value is 0.5, but the confidence coefficient of the call data is 0.6, which is an abnormal condition, the call label of the call data to be quality-checked is determined to be a low confidence coefficient label.
In the embodiment, by detecting the confidence level of the call data to be quality-checked, when the confidence level of the call data exceeds a preset confidence level threshold, it is considered that a situation that the speech robot has a semantic meaning of a user understood by mistake occurs, and the situation is an abnormal situation. The confidence of the call data is used as a detection basis, so that the accuracy of the abnormal detection is improved.
In one embodiment, the number of times the user semantics are rejected is explained, and as shown in fig. 6, one embodiment of the above S102 includes the following steps:
s501, analyzing the times of refusing the user semantics of the voice robot in the call data to be quality checked according to the call characteristic rule.
And the voice robot analyzes the times of rejecting the user semantics by the voice robot in the call data to be quality tested according to the rejection characteristics in the call characteristic rule, wherein the rejection is the condition that the user semantics are not recognized at all, for example, the times of acquiring the condition that the user semantics are not recognized by the voice robot.
S501, if the number of times of rejecting the user semantics exceeds a preset rejection number threshold, determining the call tag of the call data to be quality tested as an excessive rejection tag.
The call characteristic rule sets a rejection frequency threshold, and if the rejection frequency threshold is exceeded, the call characteristic rule is regarded as abnormal. If the number of times of the voice robot analyzing the voice robot rejecting the user semantics in the call data to be quality tested exceeds the set threshold, determining that the voice robot rejects too much, which is an abnormal condition, for example, the threshold is 3 times, and then determining that the call tag of the call data to be quality tested is an excessive rejection tag.
In the embodiment, the number of times that the voice robot rejects the user semantics in the call data to be quality checked is used, and when the number of times that the voice robot rejects the user semantics exceeds a threshold value, it is determined that the situation is abnormal. The number of times that the voice robot rejects the user semantics is taken as a detection basis, so that the accuracy of anomaly detection is improved.
As shown in fig. 7, an embodiment of the present application further provides a call data quality inspection method, where the embodiment includes:
s1, acquiring the current call data to be inspected;
s2, analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule: the voice cross duration of the voice robot and the voice stream of the user, the speed of the user, the confidence coefficient and the times of rejecting the user semantics by the voice robot;
s3, determining the call labels of the call data to be quality tested to be voice cross labels, voice speed too fast labels, confidence coefficient low labels and recognition refusal too many labels respectively;
and S4, performing quality inspection on the call data of the labels through a preset quality inspection model to obtain a quality inspection result.
According to the embodiment, after the voice robot completes automatic outbound call through the preset call characteristic rule, the system automatically analyzes the generated call data, identifies the call characteristic through the preset rule, outputs the call label, and marks the call with the abnormal characteristic with the corresponding abnormal label, so that the call data are automatically analyzed and marked through the system, and only the call with the abnormal label is required to be subjected to quality inspection, so that the quality inspection efficiency is improved.
The implementation principle and technical effect of each step in the call data quality inspection method provided in the above embodiment are similar to those in the previous call data quality inspection method embodiments, and are not described herein again. The implementation manner of each step in the embodiment of fig. 7 is only an example, and is not limited to this, and the order of each step may be adjusted in practical application as long as the purpose of each step can be achieved.
It should be understood that although the various steps in the flow charts of fig. 2-7 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-7 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 8, there is provided a call data quality inspection apparatus, including: the system comprises an acquisition module 10, a label determination module 11 and a quality inspection module 12, wherein:
the acquisition module 10 is used for acquiring current call data to be quality checked;
the tag determining module 11 is configured to analyze the feature of the call data to be quality-checked according to a preset call feature rule to obtain a corresponding call tag; the call tag comprises an abnormal tag and a normal tag;
the quality inspection module 12 is used for outputting the call data to be inspected to the quality inspection area if the call tag of the call data to be inspected is an abnormal tag; the call data to be quality checked is stored in the quality check area.
In the embodiment, the current call data to be quality-inspected is obtained, the characteristics of the call data to be quality-inspected are analyzed according to the preset call characteristic rule, so that the corresponding call tag is obtained, and if the call tag of the call data to be quality-inspected is an abnormal tag, the call data to be quality-inspected is output to the quality inspection area where the call data to be quality-inspected is stored. When the conversation data that faces a large amount of needs quality control, carry out pronunciation feature analysis and call label to each conversation data earlier automatically, tentatively select the conversation data of unusual label, then export unusual conversation data to the quality control region and carry out the quality control, through tentatively screening like this, only need carry out the quality control to the conversation data that have unusual label, make the conversation data quantity of treating the quality control less, the quality control efficiency has been improved, and can carry out the quality control of pertinence according to unusual label at the quality control in-process, instruct the quality control direction with unusual label, the quality control efficiency has further been improved.
In an embodiment, the quality inspection module 12 is configured to perform quality inspection on call data in a quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result includes the reason of the abnormal call data.
In this embodiment, the call data (including the abnormal label) in the quality inspection area are sequentially detected through the quality inspection model, the specific abnormal reason of each call data is detected, and the call data in the quality inspection area is subjected to quality inspection through the special quality inspection model, so that not only the efficiency of the quality inspection process can be improved, but also the accuracy of the quality inspection reason is improved.
In one embodiment, the exception tags include at least one of a cross-voice tag, a too-fast-speech tag, a low-confidence tag, and a too-much-rejected tag.
The voice cross-finger voice robot and the voice stream of the user have an intersection, and the specific expression is that the voice robot and the user speak simultaneously in the telephone conversation process. The too fast speaking speed means that the speaking speed of a user is too fast, the speed of speech of a general user can be increased when the emotion is excited, and the side face reflects the problem of the voice robot. The low confidence coefficient indicates that the voice robot scores a lower score for the semantic recognition result of the voice of the user, that is, the score is lower, which indicates that the voice robot has a misunderstanding of the user semantic. The fact that the voice robot cannot recognize the semantics of the user is referred to as the fact that the voice robot cannot recognize the semantics of the user, and the fact that the voice robot cannot recognize the voice of the user at all is referred to as the fact that the voice robot cannot recognize the voice of the user.
In one embodiment, the tag determination module 11 includes:
the voice cross analysis unit is used for analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule;
and the voice cross tag determining unit is used for determining that the call tag of the call data to be quality checked is the voice cross tag if the voice cross time exceeds a preset time threshold.
In the embodiment, by detecting the voice crossing time length of the voice flow of the voice robot and the user in the call data, when the voice crossing time length of the two parties exceeds the preset time length threshold value, it is considered that the voice robot and the user speak simultaneously in the telephone exceeds the threshold value, and the abnormal condition is detected by using the voice crossing time length as a detection basis, so that the accuracy of abnormal detection is improved.
In one embodiment, the tag determination module 11 includes:
the speech rate analysis unit is used for analyzing the speech rate of the user in the call data to be subjected to quality inspection according to the call characteristic rule;
and the speech rate tag determining unit is used for determining the call tag of the call data to be subjected to quality inspection as an over-speech rate tag if the speech rate exceeds a preset speech rate threshold value.
In this embodiment, by detecting the speech rate of the user in the call data to be quality-checked, when the speech rate of the user exceeds the preset speech rate threshold, that is, according to the speech rate of the user being too fast, it is determined that the emotion of the user is excited, which is an abnormal situation. The speech speed of the user is used as a detection basis, and the accuracy of anomaly detection is improved.
In one embodiment, the tag determination module 11 includes:
the confidence coefficient acquisition unit is used for acquiring the confidence coefficient of the call data to be quality-checked according to the call characteristic rule, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot;
and the confidence degree label unit is used for determining the call label of the call data to be inspected as a low confidence degree label if the confidence degree is lower than a preset threshold value.
In this embodiment, by detecting the confidence level of the call data to be quality-checked, when the confidence level of the call data exceeds the preset confidence level threshold, it is considered that a situation in which the speech robot erroneously understands the user semantics occurs, which is an abnormal situation. The confidence of the call data is used as a detection basis, so that the accuracy of the abnormal detection is improved.
In one embodiment, the tag determination module 11 includes:
the rejection frequency acquisition unit is used for analyzing the frequency of rejecting the user semantics by the voice robot in the call data to be quality tested according to the call characteristic rule;
and the rejection label unit is used for determining the call label of the call data to be quality checked as the excessive rejection label if the number of times of rejecting the user semantics exceeds a preset rejection number threshold.
In the embodiment, the number of times that the voice robot rejects the user semantics in the call data to be quality checked is used, and when the number of times that the voice robot rejects the user semantics exceeds a threshold value, the abnormal condition is determined. The number of times that the voice robot rejects the user semantics is taken as a detection basis, so that the accuracy of anomaly detection is improved.
The implementation principle and technical effect of all the call data quality inspection devices provided in the above embodiments are similar to those of the call data quality inspection method embodiments, and are not described herein again.
For the specific limitation of the call data quality inspection device, reference may be made to the above limitation of the call data quality inspection method, and details are not described herein again. All or part of each module in the call data quality inspection device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a call data quality inspection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring current call data to be inspected;
analyzing the characteristics of the call data to be quality checked according to a preset call characteristic rule to obtain a corresponding call label; the call tag comprises an abnormal tag and a normal tag;
if the call tag of the call data to be quality-checked is an abnormal tag, outputting the call data to be quality-checked to a quality-check area; the call data to be quality checked is stored in the quality check area.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
performing quality inspection on call data in a quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result includes the reason of the abnormal call data.
In one embodiment, the exception tags include at least one of a cross-voice tag, a too-fast-speech tag, a low-confidence tag, and a too-much-rejected tag.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule;
and if the voice cross time exceeds a preset time threshold, determining that the call tag of the call data to be subjected to quality inspection is the voice cross tag.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
analyzing the speech rate of a user in the call data to be quality checked according to the call characteristic rule;
and if the speech rate exceeds a preset speech rate threshold value, determining the call tag of the call data to be subjected to quality inspection as an over-speech rate tag.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
according to the call characteristic rule, obtaining the confidence coefficient of the call data to be quality checked, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot;
and if the confidence coefficient is lower than a preset threshold value, determining the call label of the call data to be subjected to the quality inspection as a low confidence coefficient label.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
analyzing the times of rejecting user semantics by the voice robot in the call data to be quality checked according to the call characteristic rule;
and if the times of rejecting the user semantics exceed a preset rejection time threshold, determining the call label of the call data to be quality tested as an excessive rejection label.
The implementation principle and technical effect of the computer device provided by the above embodiment are similar to those of the above method embodiment, and are not described herein again.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring current call data to be inspected;
analyzing the characteristics of the call data to be quality checked according to a preset call characteristic rule to obtain a corresponding call label; the call tag comprises an abnormal tag and a normal tag;
if the call tag of the call data to be quality-checked is an abnormal tag, outputting the call data to be quality-checked to a quality-check area; the call data to be quality checked is stored in the quality check area.
In one embodiment, the computer program when executed by the processor further performs the steps of: performing quality inspection on call data in a quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result includes the reason of the abnormal call data.
In one embodiment, the exception tags include at least one of a cross-voice tag, a too-fast-speech tag, a low-confidence tag, and a too-much-rejected tag.
In one embodiment, the computer program when executed by the processor further performs the steps of:
analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule;
and if the voice cross time exceeds a preset time threshold, determining that the call tag of the call data to be subjected to quality inspection is the voice cross tag.
In one embodiment, the computer program when executed by the processor further performs the steps of:
analyzing the speech rate of a user in the call data to be quality checked according to the call characteristic rule;
and if the speech rate exceeds a preset speech rate threshold value, determining the call tag of the call data to be subjected to quality inspection as an over-speech rate tag.
In one embodiment, the computer program when executed by the processor further performs the steps of:
according to the call characteristic rule, obtaining the confidence coefficient of the call data to be quality checked, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot;
and if the confidence coefficient is lower than a preset threshold value, determining the call label of the call data to be subjected to the quality inspection as a low confidence coefficient label.
In one embodiment, the computer program when executed by the processor further performs the steps of:
analyzing the times of rejecting user semantics by the voice robot in the call data to be quality checked according to the call characteristic rule;
and if the times of rejecting the user semantics exceed a preset rejection time threshold, determining the call label of the call data to be quality tested as an excessive rejection label.
The implementation principle and technical effect of the computer-readable storage medium provided by the above embodiments are similar to those of the above method embodiments, and are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A call data quality inspection method is characterized by comprising the following steps:
acquiring current call data to be inspected;
analyzing the characteristics of the call data to be quality checked according to a preset call characteristic rule to obtain a corresponding call label; the call tag comprises an abnormal tag and a normal tag;
if the call tag of the call data to be quality-checked is an abnormal tag, outputting the call data to be quality-checked to a quality-check area; and the call data needing quality inspection is stored in the quality inspection area.
2. The method of claim 1, wherein after the outputting the call data to be quality tested to a quality testing area, the method further comprises:
performing quality inspection on the call data in the quality inspection area through a preset quality inspection model to obtain a quality inspection result; the quality inspection result comprises the reason of abnormal call data.
3. The method of claim 1 or 2, wherein the exception label comprises at least one of a cross-speech label, a too fast speech label, a low confidence label, and a too many rejected labels.
4. The method according to claim 3, wherein the analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule to obtain a corresponding call tag comprises:
analyzing the voice cross duration of the voice robot and the voice stream of the user in the call data to be quality tested according to the call characteristic rule;
and if the voice cross time exceeds a preset time threshold, determining the call tag of the call data to be subjected to quality inspection as the voice cross tag.
5. The method according to claim 3, wherein the analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule to obtain a corresponding call tag comprises:
analyzing the speech rate of the user in the call data to be quality tested according to the call characteristic rule;
and if the speech rate exceeds a preset speech rate threshold value, determining the call tag of the call data to be subjected to the quality inspection as the over-speech rate tag.
6. The method according to claim 3, wherein the analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule to obtain a corresponding call tag comprises:
according to the call characteristic rule, obtaining the confidence coefficient of the call data to be quality checked, wherein the confidence coefficient represents the scoring result of the semantic recognition result of the user by the voice robot;
and if the confidence coefficient is lower than a preset threshold value, determining the call label of the call data to be subjected to the quality inspection as the low confidence coefficient label.
7. The method according to claim 3, wherein the analyzing the characteristics of the call data to be quality-checked according to a preset call characteristic rule to obtain a corresponding call tag comprises:
analyzing the times of rejecting user semantics by the voice robot in the call data to be quality checked according to the call characteristic rule;
and if the number of times of rejecting the user semantics exceeds a preset rejection number threshold, determining the call label of the call data to be quality tested as the excessive rejection label.
8. A call data quality inspection apparatus, comprising:
the acquisition module is used for acquiring current call data to be quality checked;
the tag determination module is used for analyzing the characteristics of the call data to be quality checked according to a preset call characteristic rule to obtain a corresponding call tag; the call tag comprises an abnormal tag and a normal tag;
the quality inspection module is used for outputting the call data to be inspected to a quality inspection area if the call tag of the call data to be inspected is an abnormal tag; and the call data needing quality inspection is stored in the quality inspection area.
9. The apparatus of claim 8, wherein the exception label comprises at least one of a cross-speech label, an over-speech label, a low confidence label, and an over-recognition label.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010672539.7A CN111565254B (en) | 2020-07-14 | 2020-07-14 | Call data quality inspection method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010672539.7A CN111565254B (en) | 2020-07-14 | 2020-07-14 | Call data quality inspection method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111565254A CN111565254A (en) | 2020-08-21 |
| CN111565254B true CN111565254B (en) | 2020-10-09 |
Family
ID=72075417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010672539.7A Active CN111565254B (en) | 2020-07-14 | 2020-07-14 | Call data quality inspection method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111565254B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113645357B (en) * | 2021-08-06 | 2024-02-20 | 深圳追一科技有限公司 | Call quality inspection method, device, computer equipment and computer readable storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5208848A (en) * | 1991-08-26 | 1993-05-04 | At&T Bell Laboratories | Telecommunications call processing |
| US6480597B1 (en) * | 1998-06-12 | 2002-11-12 | Mci Communications Corporation | Switch controller for a telecommunications network |
| CN101389107B (en) * | 2007-09-10 | 2012-04-18 | 中兴通讯股份有限公司 | Large traffic simulation test system and method |
| CN108521525A (en) * | 2018-04-03 | 2018-09-11 | 南京甄视智能科技有限公司 | Intelligent robot customer service marketing method and system based on user tag system |
| CN109151218B (en) * | 2018-08-21 | 2021-11-19 | 平安科技（深圳）有限公司 | Call voice quality inspection method and device, computer equipment and storage medium |
| CN110853649A (en) * | 2019-11-05 | 2020-02-28 | 集奥聚合(北京)人工智能科技有限公司 | Label extraction method, system, device and medium based on intelligent voice technology |
| CN111010484A (en) * | 2019-12-16 | 2020-04-14 | 科大国创软件股份有限公司 | Automatic quality inspection method for call recording |
| CN111128241A (en) * | 2019-12-30 | 2020-05-08 | 上海浩琨信息科技有限公司 | Intelligent quality inspection method and system for voice call |
-
2020
- 2020-07-14 CN CN202010672539.7A patent/CN111565254B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111565254A (en) | 2020-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112804400B (en) | Customer service call voice quality inspection method and device, electronic equipment and storage medium | |
| CN111105782B (en) | Session interaction processing method and device, computer equipment and storage medium | |
| CN110472224B (en) | Quality of service detection method, apparatus, computer device and storage medium | |
| KR102431754B1 (en) | Apparatus for supporting consultation based on artificial intelligence | |
| US9159054B2 (en) | System and method for providing guidance to persuade a caller | |
| CN107818798A (en) | Customer service quality evaluating method, device, equipment and storage medium | |
| US9904927B2 (en) | Funnel analysis | |
| CN112037799B (en) | Voice interrupt processing method and device, computer equipment and storage medium | |
| US20100332287A1 (en) | System and method for real-time prediction of customer satisfaction | |
| CN110177182B (en) | Sensitive data processing method and device, computer equipment and storage medium | |
| CN111597818B (en) | Call quality inspection method, device, computer equipment and computer readable storage medium | |
| CN111145733B (en) | Speech recognition method, speech recognition device, computer equipment and computer readable storage medium | |
| CN105744090A (en) | Voice information processing method and device | |
| US20190295098A1 (en) | Performing Real-Time Analytics for Customer Care Interactions | |
| EP4181124A1 (en) | Communication system and related methods | |
| CN110556098B (en) | Voice recognition result testing method and device, computer equipment and medium | |
| CN111565254B (en) | Call data quality inspection method and device, computer equipment and storage medium | |
| US10068567B1 (en) | System, method, and computer program for automatic management of intent classification | |
| CN113571096B (en) | Speech emotion classification model training method and device, computer equipment and medium | |
| CN108597499B (en) | Voice processing method and voice processing device | |
| US11611554B2 (en) | System and method for assessing authenticity of a communication | |
| CN112434953A (en) | Customer service personnel assessment method and device based on computer data processing | |
| CN114067842B (en) | Customer satisfaction degree identification method and device, storage medium and electronic equipment | |
| US11064072B1 (en) | Caller state determination | |
| JP2017211586A (en) | Psychological analyzer, psychological analysis method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |