CN111340148A - Training method of business classification model, business classification method and terminal - Google Patents
Training method of business classification model, business classification method and terminal Download PDFInfo
- Publication number
- CN111340148A CN111340148A CN202010439817.4A CN202010439817A CN111340148A CN 111340148 A CN111340148 A CN 111340148A CN 202010439817 A CN202010439817 A CN 202010439817A CN 111340148 A CN111340148 A CN 111340148A
- Authority
- CN
- China
- Prior art keywords
- service
- classification
- business
- classification model
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a training method of a business classification model, a business classification method and a terminal, which can enable the accuracy of the business classification model aiming at each classification result to reach a corresponding target and realize the effect of optimizing aiming at multiple targets. The training method of the business classification model comprises the following steps: acquiring a plurality of groups of service sample data for training a service classification model; determining a target value of the classification accuracy measurement of the business classification model for each business class; and performing iterative training on the service classification model by using multiple groups of service sample data until an iterative convergence condition is reached, wherein after each iterative training, the accurate classification measurement of the service classification model for each service class is calculated, and the iterative convergence condition comprises that the accurate classification measurement of the service classification model for each service class reaches a corresponding target value.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method for training a service classification model, a method for classifying a service, and a terminal.
Background
The business classification model can be applied to various application scenes, for example, manual customer service is dispatched, with the increasing of the types of businesses carried in application software, the services provided for customers tend to be diversified and complicated, and when the scale reaches a certain degree, the problems brought forward by users to customer service hotlines tend to be diversified. The variety and complexity of the problem put a great deal of pressure on the customer service of the company. The cost of the customer service staff for training to answer all the problems is too high, the existing method is to divide the manual customer service into a plurality of groups according to the service categories to carry out respective training, and each group of customer service staff is mainly trained to answer the problems of the corresponding service categories. Before the reply of the artificial customer service, the AI customer service robot and the user can be configured to carry out dialogue, the problem of the user is guessed according to the description of the user, the information of affirmation/negation/supplement and the like of the user is obtained, and finally the dialogue is distributed to a proper skill group to enable the artificial customer service to communicate with the user. This process is called "dispatching. The order dispatching is actually a classification process, and a business classification model is used for determining which business class of artificial customer service is allowed to answer the question of the user according to the dialogue information of the user and some historical data. In the traditional business classification model, if the classification result includes a plurality of categories, the overall classification accuracy of the business classification model is generally improved, and the training target of the business classification model is single.
Disclosure of Invention
In view of this, embodiments of the present application provide a method for training a service classification model, a method for classifying a service, and a terminal, so that the service classification model can achieve a preset accuracy for each classification result.
In a first aspect, an embodiment of the present application provides a method for training a service classification model, where the method includes: acquiring a plurality of groups of service sample data for training a service classification model, wherein each service sample data comprises a service consultation sentence of a user used as input data of the service classification model and is pre-configured with a service class label, the service class is an output target of the service classification model, and different groups of service sample data correspond to different service classes; determining a target value of the classification accuracy measurement of the business classification model for each business class; and performing iterative training on the service classification model by using multiple groups of service sample data until an iterative convergence condition is reached, wherein after each iterative training, the accurate classification measurement of the service classification model for each service class is calculated, and the iterative convergence condition comprises that the accurate classification measurement of the service classification model for each service class reaches a corresponding target value.
Each group of service sample data comprises a service consultation statement of a user used as input data of a service classification model and a service class label used as an output target of the service classification model; the business consultation sentence is a sentence input by the user when the user consults business, for example, the user can consult different businesses such as finance type APP (application program) and AI (Artificial Intelligence, short for Artificial Intelligence) customer service through a chat interface, such as finance, insurance, safety and the like, the business consultation sentence comprises the sentence input by the user, optionally, the business consultation sentence also comprises a sentence fed back by the AI customer service, the business consultation sentence can be collected by the financial type APP server, and the business category label is a business category labeled in advance for the business category sample data and used as an output target of the training business classification model. The classification accuracy of the service classification models for each service category may be different, in this embodiment of the present application, the classification accuracy measure is used as a measure of the classification accuracy of the service classification models for each service category, and a target of the classification accuracy measure, that is, a target value of the classification accuracy measure, may be preset for each service category.
In an optional implementation manner, the service classification model includes a plurality of basic classification models, and training the service classification model using each set of service sample data includes: obtaining service sample data used by the iterative training from a plurality of groups of service sample data; performing corresponding classification processing on the service sample data by using each basic classification model to obtain a plurality of initial feature vectors, wherein each initial feature vector is used for representing a classification result of the corresponding basic classification model for the service sample data, and an optional implementation manner is that each element of each initial feature vector is used for representing the probability of one service classification; according to the fusion weight parameters of each basic classification model, fusing a plurality of initial feature vectors to obtain fusion feature vectors, wherein an optional fusion algorithm is to directly perform weighted average on the plurality of initial feature vectors according to the fusion weight parameters, optionally, a neural network model with the parameters as the fusion weight parameters can be used, and processing operations including convolution, pooling and the like are performed on the plurality of initial feature vectors to obtain the fusion feature vectors, which is not limited in the embodiment of the application; and determining the corresponding service class according to the fusion feature vector to obtain a classification result of the service classification model on the service sample data.
In an optional implementation manner, after determining the corresponding service class according to the fused feature vector, the method further includes: comparing whether the classification result of the service classification model is the same as the label of the service class corresponding to the service sample data; and updating the classification accuracy measurement of the service classification model aiming at each service class according to the comparison result.
In an optional embodiment, after updating the classification accuracy measure of the traffic classification model for each traffic class according to the comparison result, the method further includes: comparing the labels of the service classes corresponding to the service sample data with the classification result of each basic classification model respectively; updating the classification accuracy measurement of each basic classification model aiming at each service class according to the comparison result; and adjusting the fusion weight parameter of each basic classification model according to the difference value between the classification accuracy measure of each service class of the service classification model and the target value of the classification accuracy measure of the corresponding service class and by combining the classification accuracy measure of each basic classification model for each service class, wherein the adjusted fusion weight parameter is used in the next iterative training process of the service classification model.
In an optional implementation manner, after performing corresponding classification processing on the traffic sample data respectively by using each basic classification model to obtain a plurality of initial feature vectors, the method further includes: and adjusting the internal model parameters of the corresponding basic classification models according to the classification accuracy measurement of each basic classification model aiming at each service class. In the process of training the business classification model, the model internal parameters of each basic classification model in the business classification model are trained simultaneously, so that the classification accuracy of the business classification model can be further improved.
In an alternative embodiment, after each iterative training, the method further comprises: and adjusting the proportion of each group of service sample data in a plurality of groups of service sample data according to the difference between the classification accuracy measure of each service class and the corresponding target value of the service classification model. During training, the number of samples of the service class with higher requirement on classification accuracy measurement can be increased, the number of samples of the service class with lower requirement on classification accuracy measurement can be reduced, and the training process can be converged more quickly.
In an alternative embodiment, the plurality of base classification models includes a combination of at least two of the following models: a text matching model, an XGboost model, a Bert model and a neural network model.
In an alternative embodiment, the initial feature vector is a multidimensional vector, each dimension of the initial feature vector is used to represent the probability of one service class, and the service class with the highest probability may be used as the classification result. By outputting the probability of each traffic class, the probability of each traffic class can be measured, and more information can be provided.
In an alternative embodiment, the fused feature vector is a multidimensional vector, and each dimension of the fused feature vector is used to represent the probability of one service class. The information represented by each dimension of the fused feature vector is similar to the initial feature vector, and is not described herein again.
In an optional embodiment, each business sample data further comprises personal attribute data of the user as input data of the business classification model, and/or operation records of the user before inputting the business consultation sentence, wherein the input data of each basic classification model is one or more data in the business sample data. By enriching the content of the input service sample data, the classification result is more accurate.
In an alternative embodiment, the classification accuracy measure includes at least one of the following parameters: accuracy, precision, recall, harmonic mean.
In a second aspect, an embodiment of the present application further provides a service classification method, where the method includes: receiving a business consultation sentence of a user; generating business data according to business consultation sentences of the user; and determining a service class corresponding to the service data by using a service classification model, wherein the service classification model is obtained by training according to a training method of the service classification model provided in the first aspect and any optional embodiment thereof, and the service classification model is used for outputting a corresponding classification result according to input service data in a plurality of preset service classes.
In an optional implementation manner, after determining the service class corresponding to the service data by using the service classification model, the method further includes: receiving an artificial classification result fed back aiming at the business data, wherein the artificial classification result is a classification result which is selected from a plurality of business categories and corresponds to the business data; updating the classification accuracy measurement of the service classification model aiming at each service class according to the manual classification result; determining a target value of the classification accuracy measurement of the business classification model for each business class; and adjusting the fusion weight parameter of the business classification model according to the error between the updated classification accuracy measure of each business class and the corresponding target value of the business classification model, wherein the business classification model comprises a plurality of basic classification models, and the classification result output by the business classification model is the result obtained by fusing the classification result of each basic classification model aiming at the business data by using the fusion weight parameter.
In an optional embodiment, receiving the manual classification result fed back for the service data includes: distributing corresponding business personnel of business categories to users according to the classification result of the business classification model to the business data; and receiving a manual classification result fed back by the service personnel aiming at the service data.
In an optional implementation manner, after allocating the service personnel of the corresponding service category to the user, the method further includes: and establishing a dialogue connection between the terminal of the user and the terminal of the service personnel.
In an optional implementation, generating business data according to the business consultation statement of the user includes: acquiring personal attribute data of a user and/or an operation record of the user before inputting a business consultation sentence; and generating business data according to the acquired personal attribute data and/or operation records and the business consultation sentences.
In an optional implementation manner, the service classification model is configured to output a multi-dimensional fused feature vector, where each dimension of the fused feature vector is respectively used to represent a probability of a service class, and the service classification model is used to determine a service class corresponding to the service data, including: acquiring a fusion feature vector output by a service classification model; determining the probability of each service category according to the fusion feature vector; adjusting the probability of the specified service type in the fusion feature vector according to a pre-configured adjustment strategy aiming at the specified service type; and determining the service class with the highest probability in the adjusted fusion feature vector to obtain the service class corresponding to the service data.
In an optional implementation manner, adjusting the probability of a specific service class in the fused feature vector according to a pre-configured adjustment policy for the specific service class includes: acquiring a text in the service data; judging whether preset keywords associated with the specified service category exist in the text or not; and if so, improving the probability of the specified service class in the fusion feature vector.
In an optional implementation manner, before adjusting the probability of the specified service class in the fused feature vector according to a pre-configured adjustment policy for the specified service class, the method further includes: receiving an adjustment to a target value of a classification accuracy measure for a specified traffic class; and determining an adjustment strategy of the specified service class according to the adjustment of the target value, wherein the adjustment strategy is used for increasing or decreasing the probability of the specified service class in the fusion feature vector.
In a third aspect, an embodiment of the present application provides a terminal, including: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the terminal, cause the terminal to perform the method of training a traffic classification model according to the first aspect.
In a fourth aspect, an embodiment of the present application provides a terminal, including: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the terminal, cause the terminal to perform the traffic classification method according to the second aspect.
The technical solutions provided in any of the above aspects and any optional implementation manner thereof may set a target value for the classification accuracy measure of each classification result in advance, when training the service classification model, calculate the classification accuracy measure of the service classification model for each service category after each iterative training, and set a target value at which the classification accuracy measure of the service classification model for each service category reaches a corresponding target value as an iterative convergence condition, so that the classification accuracy measure of the service classification model for each service category reaches an expected accuracy degree, thereby achieving an effect of optimizing for a plurality of targets.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating a method for training a business classification model according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram illustrating a traffic classification method according to an embodiment of the present disclosure;
fig. 3 is a first interaction diagram of a service classification method provided in an embodiment of the present application;
fig. 4 is a schematic interaction diagram ii of a service classification method according to an embodiment of the present application;
fig. 5 is a third interaction diagram of a service classification method provided in the embodiment of the present application;
fig. 6 is a fourth interaction diagram of a service classification method provided in the embodiment of the present application;
FIG. 7 is a flowchart illustrating a method for training a business classification model according to an embodiment of the present application;
fig. 8 is a flowchart illustrating a service classification method according to an embodiment of the present application.
Detailed Description
For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.
It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
It should be understood that although the terms first, second, third, etc. may be used to describe a certain term in the embodiments of the present application, they should not be limited to these terms. The first, second and third are only used to distinguish these terms from each other.
The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
The method for training the traffic classification model provided in the embodiment of the present application may be performed by a computer device, which may include one or more processors, a memory, and one or more computer programs stored in the memory, where the one or more computer programs include instructions that, when executed by the device, cause the device to perform the method for training the traffic classification model provided in the embodiment of the present application. Embodiments of a training method for a business classification model provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
As shown in fig. 1, multiple sets of traffic sample data may be prepared in advance. Each group of business sample data comprises business consultation sentences, user attribute data and user operation records which are used as input data of a training business classification model, and artificial labeling labels of training targets which are used as output data of the training business classification model. The business consultation sentence is a sentence in natural language, the business consultation sentence can be collected by a server of the application program, and a history sentence input when the user uses the application program to consult business can be used as the business consultation sentence in the business sample data. The user attribute data is personal attribute data of the user, for example, the age of the user registered to use the application, the user age, the user's address, the user's occupation, an interest tag that the user has tagged to himself, and the like. The user operation record is a preceding operation record before the user inputs a history sentence, for example, a website browsed by the user, an operation performed by the user on an operable control in the application (for example, clicking an icon control for adding an object into a shopping cart), and the like. The manual labeling label is a service class labeled for the service sample data in advance and is used as an output target for training the service classification model, each group of service sample data comprises a manual labeling label, and after input data in the service sample data is input into the service classification model, the output classification result is expected to be the service class indicated by the manual labeling label. In the total service sample data, the service sample data is divided into a plurality of sets according to the service class of the manually labeled tag, and the proportion of each set in the total service sample data is different, as shown in fig. 1, the proportion of the service sample data of the "financing" service class in the total service sample data is a1, the proportion of the service sample data of the "insurance" service class is a2, the proportion of the sample data of the "safe" service class is a3, and a1+ a2+ a3= 1.
In the set of the service sample data, a group of service sample data is selected and input into the service classification model, so that the classification result of the service classification model for the group of service sample data can be obtained: a traffic class. The service classification model comprises a fusion model and a plurality of basic classification models: a neural network model, an XGboost model, a Bert model and a text matching model. The service sample data is firstly input into each basic classification model, optionally, the input required by each basic classification model may be different, for example, in an optional implementation, for the text matching model, only the user query statement may be input, for the neural network model, all the input data in the service sample data may be input, for the XGBoost model, only the user query statement and the user attribute data may be input, and the above input data used only for exemplary illustration to input different basic classification models may not be completely the same, and does not constitute a limitation to the embodiments of the present application. Each basic classification model may process input data, for example, convert text into word vectors using word2vec for user consultation sentences, or correspond each user attribute data to vectors according to a preset mapping relationship to enable machine recognition. After each basic classification model performs corresponding operation processing on input data, an initial feature vector for representing an output result is obtained, and optionally, the initial feature vector may be a multi-dimensional vector, each dimension is used for representing the probability of a service class corresponding to the dimension, for example, the initial feature vector u1 output by the neural network model is [0.1, 0.6, 0.3], the probabilities representing service classes "financing", "insurance", and "safety" are 0.1,0.6.0.3, respectively, the service class with the highest probability is "insurance", and thus, for the neural network model, the result of classifying the input data is "insurance".
After the initial feature vector output by each basic classification model is obtained, whether the classification is correct or not can be judged according to the comparison between the service class corresponding to the initial feature vector and the manual labeling label, and the classification accuracy measurement of each basic classification model is calculated. The classification accuracy measure in the embodiment of the present application may include one or more of an accuracy, a recall and a reconciliation average, where the accuracy is used to describe the number of paired samples divided by the number of all samples, the accuracy represents a proportion of an example classified into a certain traffic class, which is actually the traffic class, and the recall represents a proportion of the example classified into the traffic class, which is actually the traffic class, and the reconciliation average is formulated as: precision recall 2/(precision + recall), and harmonic mean is used to represent the harmonic mean of precision and recall.
Optionally, the basic classification model may be a classification model trained in advance, and parameters in the basic classification model are kept unchanged in the training process of the service classification model; or the model internal parameters of the basic classification model can be changed along with each training of the business classification model, after the initial feature vector output by each basic classification model is obtained, the model internal parameters of the basic classification model are adjusted according to the manual label, and when the next round of business sample data is input, the basic classification model for adjusting the model internal parameters is used for classification.
In order to enable the business classification model to obtain a more accurate classification result, after the initial feature vector output by each basic classification model is obtained, the initial feature vector output by each basic classification model is input into a fusion model, and the fusion model is used for fusing a plurality of initial feature vectors according to the fusion weight parameters of each basic classification model to obtain a fusion feature vector. As shown in fig. 1, the formula of the fusion model is: f (h1(b1, u1), h2(b2, u2), h3(b3, u3), h4(b4, u4)), wherein b 1-b 4 are fusion weight parameters of each basic classification model, u 1-u 4 are initial feature vectors output by each basic classification model, and the formula of a specific fusion model is as follows: (b1 × u1+ b2 × u2+ b3 × u3+ b4 × u 4)/4, optionally, the fusion model may further adopt a neural network model, and b 1-b 4 are used as internal parameters in the neural network model. Similar to the initial feature vector, the fusion feature vector output by the fusion model can determine the corresponding service class, i.e. obtain the classification result of the service classification model. After the classification result of the service classification model is obtained, whether the classification result of the service classification model is correct or not can be checked according to the manual labeling label labeled in advance by the service sample data, and the classification accuracy measurement of the service classification model for each service class is updated.
After the classification accuracy measurement of the service classification model is updated, the proportion of each service class in the service sample data and the fusion weight parameter of each basic classification model in the fusion model are adjusted according to the difference between the classification accuracy measurement of the statistical service classification model for each service class and the target value of the preset classification accuracy measurement and the classification accuracy measurement of each basic classification model for each service class.
An alternative embodiment of the adjustment strategy comprises:
① if the classification accuracy measure of the business classification model is lower than the target value of the classification accuracy measure for a certain business class, then increasing the fusion weight parameter of the basic classification model with higher classification accuracy measure for the business class, and decreasing the fusion weight parameter of the basic classification model with lower classification accuracy measure for the business class, or vice versa;
② if the classification accuracy measure of the service classification model is lower than the target value of the classification accuracy measure for a certain service class, then increasing the proportion of the service sample data of the service class in the total service sample data, and decreasing the proportion of the service sample data of other service classes in the total service sample data, or vice versa.
Further, one service sample data is selected from the adjusted service sample data, the adjusted service classification model is input, and then the parameters are adjusted again according to the output result of the adjusted service classification model.
And obtaining a trained service classification model after the iteration convergence condition is reached. The iteration convergence condition may be that the iteration times reach a specified numerical value, that is, the service classification model is trained by using the service sample data of the specified numerical value; or, the iterative convergence condition may also be convergence of the fusion weight parameter, and the training of the service classification model is stopped.
As shown in fig. 2, the embodiment of the present application further provides a service classification method, which is an optional implementation method of the service classification method provided in the embodiment of the present application, and the service classification model trained by the training method of the service classification model provided in the embodiment of the present application can be used for classification.
As shown in fig. 2, in the application scenario, when a consultation sentence input by a user is received, attribute data of the user and an operation record before the user inputs the consultation sentence are acquired as service data of the input service classification model. Specifically, as shown in fig. 3, the user may click on the selected application software on the interface of the mobile phone terminal, and the application software may be named "payment software" for example. The interface of the "payment software" application may be as shown in fig. 4, and the user may click and select a lower option icon 201 within the "payment software" application, illustratively, the option icon 201 is named "online consultation", thereby entering a dialog page for online consultation. The page of online consultation can be as shown in fig. 5, the user can edit the information to be sent in the input edit box 202, the user confirms that the sent content is displayed on the right side of the page, and the AI customer service can reply to the content input by the user. When the content AI customer service input by the user cannot answer, for example, the AI customer service cannot search for an answer to a consultation question put forward by the user, or the user inputs some keywords (does not solve my question), and the like, the user can go to manual customer service to answer.
At this time, the trained business classification model can be used for classification to judge which business class the manual customer service is transferred to.
The input data of the business classification model includes user consultation statements, such as all statements shown on the mobile phone interface in fig. 5 (including reply statements of AI customer service), attribute data of the user (such as age, sex, occupation, software service life, and the like) obtained according to the registration ID of the consultation user, and operation records of the user before consultation (such as an application for sending password modification, and the like).
After the business data is obtained, the business data is input to the business classification model, as shown in fig. 2, by each basic classification model: the method comprises the steps that a neural network model, an XGboost model, a Bert model and a text matching model are used for classifying input service data to obtain initial feature vectors u 1-u 4, and then a fusion model is used: f (h1(b1, u1), h2(b2, u2), h3(b3, u3) and h4(b4, u4)), calculating a plurality of initial feature vectors u 1-u 4 to obtain a fusion feature vector p, further determining the service class of which the service class is 'safe', selecting one personal customer service from the manual customer services of the 'safe' service class, and establishing the dialogue connection between the customer service and the user.
It should be noted that, in the service classification method provided in the embodiment of the present application, the fusion weight parameter of the service classification model may be adjusted according to the classification accuracy measure. Specifically, whether the classification result is correct or not may be fed back by the service staff, as shown in fig. 6, the interface displayed by the mobile phone of the manual customer service may be displayed, the manual customer service may edit the input content in the input edit box 204, and a dialog box 203 may be provided in the interface of the manual customer service to prompt the manual customer service to feed back the service classification of the content consulted by the user.
Furthermore, according to the classification result of the artificial customer service, the classification accuracy measure of each basic classification model in the business classification model and the classification accuracy measure of the business classification model can be updated, and after the update, the fusion weight parameters of the business classification model can be adjusted according to the following adjustment strategies: if the classification accuracy measure of the service classification model is lower than the target value of the classification accuracy measure aiming at a certain service class, the fusion weight parameter of the basic classification model with higher classification accuracy measure for the service class is improved, the fusion weight parameter of the basic classification model with lower classification accuracy measure for the service class is reduced, and vice versa. The target value of the classification accuracy measure is variable and can be set, and the service bearing capacity of each service class can be determined according to the number of artificial customer services of different service classes, for example, if the number of artificial customer services of a financial service class is large, the stronger the capacity of bearing the consultation conversation content is, the target accuracy rate allocated to the service class can be reduced, and if the number of artificial customer services of a safe service class is small, the weaker the capacity of bearing the consultation conversation content is, the target accuracy rate allocated to the service class can be improved.
In an alternative embodiment, the output of the service classification model may be adjusted using a service rule, for example, if the "financing" service class is a new online service class, and the service sample data used in training the service classification model is less, which may result in an inaccurate identification for the service class, a preset service rule may be used to adjust the fusion feature vector, an exemplary service rule is to set a keyword "fund", if the keyword appears in a user consultation statement, a preset value 0.2 is added to an element in the fusion feature vector representing the "financing" service class, and a service class with the highest probability is determined according to the adjusted fusion feature vector.
The following describes in detail steps of a training method of a business classification model provided in an embodiment of the present application with reference to the accompanying drawings, as shown in fig. 7, the training method of a business classification model provided in an embodiment of the present application includes the following steps:
step 1011, acquiring multiple groups of service sample data for training the service classification model:
each group of service sample data comprises a service consultation statement of a user used as input data of a service classification model and a service class label used as an output target of the service classification model; the service consultation sentence is a sentence input by the user when the user consults the service, for example, the user can consult different services such as finance, insurance and safety through a chat interface by using a finance APP (application) and an AI (Artificial Intelligence, short for Artificial Intelligence) customer service installed on a mobile phone, the service consultation sentence includes a sentence input by the user, optionally, the service consultation sentence can also include a sentence fed back by the AI customer service, and the service consultation sentence can be collected by the finance APP server.
In an optional embodiment, each set of business sample data further includes personal attribute data of the user as input data of the business classification model, and/or operation records of the user before inputting the business consultation sentence, wherein the input data of each basic classification model is one or more data in the business sample data. By enriching the content of the input service sample data, the classification result is more accurate.
The business class label is a business class labeled for the business sample data in advance and is used as an output target for training the business classification model. The output data of the traffic classification model may be the probability of each traffic class, and is output in the form of a multidimensional vector, and the numerical value of each dimension of the vector is used to represent the probability of one traffic class. By outputting the probability of each traffic class, the probability of each traffic class can be measured, and more information can be provided.
The classification accuracy of the service classification models for each service class is different, in the embodiment of the application, the classification accuracy measure is used as the measure of the classification accuracy of the service classification models for each service class, and a target of the classification accuracy measure, namely a target value of the classification accuracy measure, can be preset for each service class. In an alternative embodiment, the classification accuracy measure includes at least one of the following parameters: accuracy, precision, recall, harmonic mean.
And 102, training the service classification model by using multiple groups of service sample data until an iterative convergence condition is reached, wherein the iterative convergence condition comprises that the classification accuracy of the service classification model for each service class reaches a target value of the classification accuracy of the corresponding service class.
Optionally, the process of training the service classification model by using each group of service sample data in step 102 may include the following steps:
And 1023, determining the corresponding service class according to the fusion feature vector to obtain a classification result of the service classification model on the service sample data.
Optionally, after the plurality of basic classification models are used to perform corresponding classification processing on the service sample data respectively to obtain a plurality of initial feature vectors, the intra-model parameters of the corresponding basic classification models may also be adjusted according to the classification result of each basic classification model and the error of the service class label corresponding to the service sample data. In the process of training the business classification model, the fusion weight parameters in the business classification model can be trained, and the model internal parameters of each basic classification model in the business classification model can be trained simultaneously, so that the classification accuracy of the business classification model can be further improved.
And 1025, updating the classification accuracy measurement of the business classification model aiming at each business class according to the classification result of the business classification model and the error of the business class label corresponding to the business sample data.
In the embodiment of the application, the output of the service classification model is to fuse a plurality of basic classification models to obtain a final classification result, and the fusion weight parameter of each basic classification model is adjusted according to the difference between the classification accuracy measure of each service class by the service classification model and the target value of the classification accuracy measure of the corresponding service class and the classification accuracy measure of each basic classification model for each service class, so that the difference between the classification accuracy measure of the service classification model for each service class and the target value of the classification accuracy measure of the corresponding service class is smaller and smaller, the accuracy of the service classification model for each classification result can reach the corresponding target, and the effect of optimizing aiming at multiple targets is achieved.
In an optional implementation manner, after the classification accuracy measure of the service classification model for each service class is updated according to the classification result of the service classification model and the error of the service class label corresponding to the service sample data, the proportion of the service sample data of each service class may be adjusted according to the difference between the classification accuracy measure of the service classification model for each service class and the target value of the classification accuracy measure of the corresponding service class. During training, the number of samples of the service class with higher requirement on classification accuracy measurement can be increased, the number of samples of the service class with lower requirement on classification accuracy measurement can be reduced, and the training process can be converged more quickly.
The specific adjustment strategy may include:
① if the classification accuracy measure of the business classification model is lower than the target value of the classification accuracy measure for a certain business class, then increasing the fusion weight parameter of the basic classification model with higher classification accuracy measure for the business class, and decreasing the fusion weight parameter of the basic classification model with lower classification accuracy measure for the business class, or vice versa;
② if the classification accuracy measure of the service classification model is lower than the target value of the classification accuracy measure for a certain service class, then increasing the proportion of the service sample data of the service class in the total service sample data, and decreasing the proportion of the service sample data of other service classes in the total service sample data, or vice versa.
In the embodiment of the application, the output of the service classification model is to fuse a plurality of basic classification models to obtain a final classification result, and the fusion weight parameter of each basic classification model is adjusted according to the difference between the classification accuracy measure of each service class by the service classification model and the target value of the classification accuracy measure of the corresponding service class and the classification accuracy measure of each basic classification model for each service class, so that the difference between the classification accuracy measure of the service classification model for each service class and the target value of the classification accuracy measure of the corresponding service class is smaller and smaller, and the classification accuracy of the service classification model is improved.
The steps of the service classification method provided by the embodiment of the present application are described in detail below with reference to the accompanying drawings, as shown in fig. 8, the service classification method provided by the embodiment of the present application includes the following steps:
Optionally, after determining the service class corresponding to the service data by using the service classification model, the method further includes the following steps:
and 307, adjusting fusion weight parameters of the business classification models according to the errors between the updated classification accuracy measure of each business class and the corresponding target value of the business classification models, wherein the business classification models comprise a plurality of basic classification models, and the classification results output by the business classification models are the results obtained by fusing the classification results of each basic classification model aiming at the business data by using the fusion weight parameters.
Optionally, after each basic classification model performs corresponding classification processing on the service data, an output obtained by each basic classification model is an initial feature vector. And each initial feature vector is used for representing the classification result of the corresponding basic classification model aiming at the business data. An optional implementation manner is that the probability of each service class is determined according to a fusion feature vector, wherein the fusion feature vector is a multidimensional vector, and each dimension of the fusion feature vector is used for representing the probability of one service class; adjusting the probability of the specified service type in the fusion feature vector according to a pre-configured adjustment strategy aiming at the specified service type; and obtaining a classification result of the service classification model for the service data according to the adjusted fusion feature vector, wherein the classification result of the service classification model for the service data is the service class with the highest probability in the fusion feature vector.
Optionally, the step 304 of receiving the manual classification result fed back for the service data may include the following steps: distributing corresponding business personnel of business categories to users according to the classification result of the business classification model to the business data; receiving a manual classification result fed back by a service person for service data, for example, an application scenario in which the service person feeds back the manual classification result is shown in fig. 6.
In an optional implementation manner, after allocating the service personnel of the corresponding service category to the user, the method further includes: a conversational connection between the terminal of the user and the terminal of the service person is established as shown in fig. 5.
In an optional implementation, generating business data according to the business consultation statement of the user includes: acquiring personal attribute data of a user and/or an operation record of the user before inputting a business consultation sentence; and generating business data according to the acquired personal attribute data and/or operation records and the business consultation sentences.
In an optional implementation manner, the service classification model is configured to output a multi-dimensional fused feature vector, where each dimension of the fused feature vector is respectively used to represent a probability of a service class, and the service classification model is used to determine a service class corresponding to the service data, including: acquiring a fusion feature vector output by a service classification model; determining the probability of each service category according to the fusion feature vector; adjusting the probability of the specified service type in the fusion feature vector according to a pre-configured adjustment strategy aiming at the specified service type; and determining the service class with the highest probability in the adjusted fusion feature vector to obtain the service class corresponding to the service data.
In an optional implementation manner, adjusting the probability of a specific service class in the fused feature vector according to a pre-configured adjustment policy for the specific service class includes: acquiring a text in the service data; judging whether preset keywords associated with the specified service category exist in the text or not; and if so, improving the probability of the specified service class in the fusion feature vector.
In an optional implementation manner, before adjusting the probability of the specified service class in the fused feature vector according to a pre-configured adjustment policy for the specified service class, the method further includes: receiving an adjustment to a target value of a classification accuracy measure for a specified traffic class; and determining an adjustment strategy of the specified service class according to the adjustment of the target value, wherein the adjustment strategy is used for increasing or decreasing the probability of the specified service class in the fusion feature vector.
The embodiment of the application uses the service classification model obtained by the training of the embodiment of the service classification model training method, so that the accuracy of the service classification model for each classification result can reach a corresponding target, and the effect of optimizing aiming at multiple targets is realized. In some optional implementation manners of the service classification method provided in the embodiment of the present application, the service classification model may also be updated in real time according to feedback of the manual classification result, so that the real-time performance and flexibility of the service classification model are improved.
An embodiment of the present application further provides a terminal, including: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the terminal, cause the terminal to perform a method of training a traffic classification model as described in embodiments of the present application.
The embodiment of the present application further provides another terminal, including: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the terminal, cause the terminal to perform the traffic classification method of an embodiment of the present application.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.
Claims (21)
1. A training method of a business classification model, wherein the method comprises the following steps:
acquiring multiple groups of service sample data for training a service classification model, wherein each service sample data comprises a service consultation statement of a user used as input data of the service classification model and is preconfigured with a service class label, the service class is an output target of the service classification model, and different groups of service sample data correspond to different service classes;
determining a target value of a classification accuracy measure of the traffic classification model for each of the traffic classes;
and performing iterative training on the service classification model by using the multiple groups of service sample data until an iterative convergence condition is reached, wherein after each iterative training, the classification accuracy measure of the service classification model for each service class is calculated, and the iterative convergence condition comprises that the classification accuracy measure of the service classification model for each service class reaches a corresponding target value.
2. The method of claim 1, wherein the traffic classification model comprises a plurality of base classification models, training the traffic classification model using each set of the traffic sample data, comprising:
obtaining the service sample data used by the iterative training from the multiple groups of service sample data;
performing corresponding classification processing on the service sample data by using each basic classification model to obtain a plurality of initial feature vectors, wherein each initial feature vector is used for representing a classification result of the corresponding basic classification model for the service sample data;
fusing the plurality of initial feature vectors according to the fusion weight parameters of each basic classification model to obtain fusion feature vectors;
and determining a corresponding service class according to the fusion feature vector to obtain a classification result of the service classification model on the service sample data.
3. The method of claim 2, wherein after determining the corresponding traffic class from the fused feature vector, the method further comprises:
comparing whether the classification result of the service classification model is the same as the label of the service class corresponding to the service sample data;
and updating the classification accuracy measurement of the business classification model aiming at each business category according to the comparison result.
4. The method of claim 3, wherein after updating the classification accuracy metric of the traffic classification model for each of the traffic classes according to the comparison, the method further comprises:
comparing the labels of the service classes corresponding to the service sample data with the classification result of each basic classification model respectively;
updating the classification accuracy measurement of each basic classification model aiming at each service class according to the comparison result;
and adjusting the fusion weight parameter of each basic classification model according to the difference between the classification accuracy measure of each service class of the service classification model and the target value of the classification accuracy measure of the corresponding service class and by combining the classification accuracy measure of each basic classification model for each service class, wherein the adjusted fusion weight parameter is used in the next iterative training process of the service classification model.
5. The method according to claim 4, wherein after performing a corresponding classification process on the traffic sample data using each of the base classification models to obtain a plurality of initial feature vectors, the method further comprises:
and adjusting the internal model parameters of the corresponding basic classification models according to the classification accuracy measurement of each basic classification model aiming at each service class.
6. The method of any of claims 2-5, wherein after each iterative training, the method further comprises:
and adjusting the proportion of each group of service sample data in the multiple groups of service sample data according to the difference between the classification accuracy measure of each service class and the corresponding target value of the service classification model.
7. The method of any of claims 2-6, wherein the plurality of base classification models comprises a combination of at least two of: a text matching model, an XGboost model, a Bert model and a neural network model.
8. The method according to any of claims 2-7, wherein the initial feature vector is a multidimensional vector, each dimension of the initial feature vector being used to represent a probability of one traffic class.
9. The method according to any of claims 2-8, wherein the fused feature vector is a multidimensional vector, each dimension of the fused feature vector being used to represent a probability of one traffic class.
10. The method according to any one of claims 1-9, wherein each said business sample data further comprises personal attribute data of said user as input data of said business classification model, and/or operation records of said user before inputting said business consultation statement, wherein the input data of each said basic classification model is one or more data in said business sample data.
11. The method of any of claims 1-10, wherein the classification accuracy metric includes at least one of: accuracy, precision, recall, harmonic mean.
12. A traffic classification method, wherein the method comprises:
receiving a business consultation sentence of a user;
generating business data according to the business consultation statement of the user;
determining a business class corresponding to the business data by using a business classification model, wherein the business classification model is obtained by training according to the training method of the business classification model of any one of claims 1 to 11, and the business classification model is used for outputting a corresponding classification result according to the input business data in a plurality of preset business classes.
13. The method of claim 12, wherein after determining the traffic class corresponding to the traffic data using a traffic classification model, the method further comprises:
receiving an artificial classification result fed back aiming at the business data, wherein the artificial classification result is a classification result which is selected from the plurality of business categories and corresponds to the business data;
updating the classification accuracy measurement of the business classification model aiming at each business category according to the manual classification result;
determining a target value of a classification accuracy measure of the traffic classification model for each of the traffic classes;
and adjusting a fusion weight parameter of the business classification model according to the error between the updated classification accuracy measure of each business class and the corresponding target value of the business classification model, wherein the business classification model comprises a plurality of basic classification models, and the classification result output by the business classification model is the result obtained by fusing the classification result of each basic classification model aiming at the business data by using the fusion weight parameter.
14. The method of claim 13, wherein the receiving manual classification results for the traffic data feedback comprises:
distributing corresponding business personnel of business categories to the users according to the classification result of the business classification model on the business data;
and receiving the manual classification result fed back by the service personnel aiming at the service data.
15. The method of claim 14, wherein after assigning the user to a business person of the corresponding business category, the method further comprises:
and establishing a dialogue connection between the terminal of the user and the terminal of the service personnel.
16. The method of any one of claims 12-15, wherein the generating business data from the business advisory statement of the user comprises:
acquiring personal attribute data of the user and/or operation records of the user before inputting the business consultation sentences;
and generating the business data according to the acquired personal attribute data and/or the operation record and the business consultation statement.
17. The method according to any one of claims 12 to 16, wherein the business classification model is configured to output a multi-dimensional fused feature vector, each dimension of the fused feature vector is respectively used for representing a probability of a business class, and the determining the business class corresponding to the business data by using the business classification model includes:
acquiring the fusion feature vector output by the service classification model;
determining the probability of each service category according to the fusion feature vector;
adjusting the probability of the specified service class in the fusion feature vector according to a pre-configured adjustment strategy aiming at the specified service class;
and determining the service class with the highest probability in the adjusted fusion feature vector to obtain the service class corresponding to the service data.
18. The method of claim 17, wherein the adjusting the probability of the specified traffic class in the fused feature vector according to a pre-configured adjustment policy for the specified traffic class comprises:
acquiring a text in the service data;
judging whether preset keywords associated with the specified service category exist in the text or not;
and if so, improving the probability of the specified service class in the fusion feature vector.
19. The method of claim 18, wherein prior to adjusting the probability of a specified traffic class in the fused feature vector according to a pre-configured adjustment policy for the specified traffic class, the method further comprises:
receiving an adjustment to a target value of a classification accuracy measure for the specified traffic class;
and determining an adjustment strategy of the specified service class according to the adjustment of the target value, wherein the adjustment strategy is used for improving or reducing the probability of the specified service class in the fusion feature vector.
20. A terminal, characterized in that the terminal comprises:
one or more processors;
one or more memories;
and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the terminal, cause the terminal to perform the method of training a traffic classification model according to any of claims 1-11.
21. A terminal, characterized in that the terminal comprises:
one or more processors;
one or more memories;
and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the terminal, cause the terminal to perform the traffic classification method according to any of claims 12-19.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010439817.4A CN111340148B (en) | 2020-05-22 | 2020-05-22 | Training method of business classification model, business classification method and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010439817.4A CN111340148B (en) | 2020-05-22 | 2020-05-22 | Training method of business classification model, business classification method and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111340148A true CN111340148A (en) | 2020-06-26 |
| CN111340148B CN111340148B (en) | 2020-09-22 |
Family
ID=71184890
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010439817.4A Active CN111340148B (en) | 2020-05-22 | 2020-05-22 | Training method of business classification model, business classification method and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111340148B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307133A (en) * | 2020-10-29 | 2021-02-02 | 平安普惠企业管理有限公司 | Security protection method and device, computer equipment and storage medium |
| CN112699945A (en) * | 2020-12-31 | 2021-04-23 | 青岛海尔科技有限公司 | Data labeling method and device, storage medium and electronic device |
| CN112884009A (en) * | 2021-01-25 | 2021-06-01 | 北京紫光展锐通信技术有限公司 | Classification model training method and system |
| CN113672732A (en) * | 2021-08-19 | 2021-11-19 | 胜斗士(上海)科技技术发展有限公司 | Method and device for classifying business data |
| WO2022095378A1 (en) * | 2020-11-03 | 2022-05-12 | 平安科技（深圳）有限公司 | Artificial-intelligence-based training method and apparatus, and computer device and storage medium |
| CN114882299A (en) * | 2022-07-11 | 2022-08-09 | 深圳市信润富联数字科技有限公司 | Fruit sorting method and device, picking equipment and storage medium |
| CN114979723A (en) * | 2022-02-14 | 2022-08-30 | 杭州脸脸会网络技术有限公司 | Virtual intelligent customer service method, device, electronic device and storage medium |
| CN115102871A (en) * | 2022-05-20 | 2022-09-23 | 浙江大学 | Energy internet control terminal service processing method based on service feature vector |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108334891A (en) * | 2017-12-15 | 2018-07-27 | 北京奇艺世纪科技有限公司 | A kind of Task intent classifier method and device |
| CN109598281A (en) * | 2018-10-11 | 2019-04-09 | 阿里巴巴集团控股有限公司 | A kind of business risk preventing control method, device and equipment |
| CN110110059A (en) * | 2019-05-20 | 2019-08-09 | 挂号网（杭州）科技有限公司 | A kind of medical conversational system intention assessment classification method based on deep learning |
| CN110472665A (en) * | 2019-07-17 | 2019-11-19 | 新华三大数据技术有限公司 | Model training method, file classification method and relevant apparatus |
| CN110647671A (en) * | 2019-09-29 | 2020-01-03 | 北京邮电大学 | Data stream classification algorithm based on AAE-DWMIL-LearnNSE |
| CN110781919A (en) * | 2019-09-23 | 2020-02-11 | 腾讯云计算（北京）有限责任公司 | Classification model training method, classification device and classification equipment |
| CN110909144A (en) * | 2019-11-28 | 2020-03-24 | 中信银行股份有限公司 | Question-answer dialogue method and device, electronic equipment and computer readable storage medium |
-
2020
- 2020-05-22 CN CN202010439817.4A patent/CN111340148B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108334891A (en) * | 2017-12-15 | 2018-07-27 | 北京奇艺世纪科技有限公司 | A kind of Task intent classifier method and device |
| CN109598281A (en) * | 2018-10-11 | 2019-04-09 | 阿里巴巴集团控股有限公司 | A kind of business risk preventing control method, device and equipment |
| CN110110059A (en) * | 2019-05-20 | 2019-08-09 | 挂号网（杭州）科技有限公司 | A kind of medical conversational system intention assessment classification method based on deep learning |
| CN110472665A (en) * | 2019-07-17 | 2019-11-19 | 新华三大数据技术有限公司 | Model training method, file classification method and relevant apparatus |
| CN110781919A (en) * | 2019-09-23 | 2020-02-11 | 腾讯云计算（北京）有限责任公司 | Classification model training method, classification device and classification equipment |
| CN110647671A (en) * | 2019-09-29 | 2020-01-03 | 北京邮电大学 | Data stream classification algorithm based on AAE-DWMIL-LearnNSE |
| CN110909144A (en) * | 2019-11-28 | 2020-03-24 | 中信银行股份有限公司 | Question-answer dialogue method and device, electronic equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| ATRI MANDAL等: "Automated Dispatch of Helpdesk Email Tickets: Pushing the Limits with AI", 《THE THIRTY-FIRST AAAI CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307133A (en) * | 2020-10-29 | 2021-02-02 | 平安普惠企业管理有限公司 | Security protection method and device, computer equipment and storage medium |
| WO2022095378A1 (en) * | 2020-11-03 | 2022-05-12 | 平安科技（深圳）有限公司 | Artificial-intelligence-based training method and apparatus, and computer device and storage medium |
| CN112699945A (en) * | 2020-12-31 | 2021-04-23 | 青岛海尔科技有限公司 | Data labeling method and device, storage medium and electronic device |
| CN112699945B (en) * | 2020-12-31 | 2023-10-27 | 青岛海尔科技有限公司 | Data labeling method and device, storage medium and electronic device |
| CN112884009A (en) * | 2021-01-25 | 2021-06-01 | 北京紫光展锐通信技术有限公司 | Classification model training method and system |
| CN113672732A (en) * | 2021-08-19 | 2021-11-19 | 胜斗士(上海)科技技术发展有限公司 | Method and device for classifying business data |
| CN113672732B (en) * | 2021-08-19 | 2024-04-26 | 胜斗士(上海)科技技术发展有限公司 | Method and device for classifying service data |
| CN114979723A (en) * | 2022-02-14 | 2022-08-30 | 杭州脸脸会网络技术有限公司 | Virtual intelligent customer service method, device, electronic device and storage medium |
| CN114979723B (en) * | 2022-02-14 | 2023-08-29 | 杭州脸脸会网络技术有限公司 | Virtual intelligent customer service method, device, electronic device and storage medium |
| CN115102871A (en) * | 2022-05-20 | 2022-09-23 | 浙江大学 | Energy internet control terminal service processing method based on service feature vector |
| CN115102871B (en) * | 2022-05-20 | 2023-10-03 | 浙江大学 | Service feature vector-based energy internet control terminal service processing method |
| CN114882299A (en) * | 2022-07-11 | 2022-08-09 | 深圳市信润富联数字科技有限公司 | Fruit sorting method and device, picking equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111340148B (en) | 2020-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111340148B (en) | Training method of business classification model, business classification method and terminal | |
| US10467261B1 (en) | Methods, systems, and computer program product for implementing real-time classification and recommendations | |
| CN110070391B (en) | Data processing method and device, computer readable medium and electronic equipment | |
| US20200057612A1 (en) | Methods, systems, and computer program product for automatic generation of software application code | |
| US7644057B2 (en) | System and method for electronic communication management | |
| CN109800307B (en) | Product evaluation analysis method and device, computer equipment and storage medium | |
| CN113742492B (en) | Insurance scheme generation method and device, electronic equipment and storage medium | |
| CN111797320B (en) | Data processing method, device, equipment and storage medium | |
| CN112163424A (en) | Data labeling method, device, equipment and medium | |
| CN111191000A (en) | Dialog management method, device and system of intelligent voice robot | |
| CN111897528B (en) | Low-code platform for enterprise online education | |
| CN116601652A (en) | Expert matching system for supporting entrepreneur | |
| CN112950347B (en) | Resource data processing optimization method and device, storage medium and terminal | |
| CN114547475B (en) | Resource recommendation method, device and system | |
| CN114693215A (en) | Purchase request processing method and device, computer equipment and storage medium | |
| CN113064980A (en) | Intelligent question and answer method and device, computer equipment and storage medium | |
| CN113112282A (en) | Method, device, equipment and medium for processing consult problem based on client portrait | |
| CN111966805B (en) | Method, device, medium and electronic equipment for assisting in realizing session | |
| CN111179055A (en) | Credit limit adjusting method and device and electronic equipment | |
| CN109885695A (en) | Assets suggest generation method, device, computer equipment and storage medium | |
| CN116485185A (en) | Enterprise risk analysis system and method based on comparison data | |
| CN116187675A (en) | Task allocation method, device, equipment and storage medium | |
| CN114238370A (en) | Method and system for applying NER entity recognition algorithm in report query | |
| CN112860878A (en) | Service data recommendation method, storage medium and equipment | |
| CN113657496A (en) | Information matching method, device, equipment and medium based on similarity matching model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |