CN115098140A

CN115098140A - Model updating method and related device

Info

Publication number: CN115098140A
Application number: CN202210873112.2A
Authority: CN
Inventors: 李敬文
Original assignee: Bank of China Ltd
Current assignee: Bank of China Ltd
Priority date: 2022-07-21
Filing date: 2022-07-21
Publication date: 2022-09-23

Abstract

The application discloses a model updating method and a related device, which can be applied to the field of artificial intelligence and the field of finance. The method can obtain the model to be updated and the corresponding training index; acquiring a new training sample corresponding to the model from a database; performing the training on the model based on the new training sample and the training index; after the training is finished, storing a model file corresponding to the model; and updating the model by loading the model file on line. Therefore, the method can automatically and timely retrain the model based on the new training sample and the new training index, thereby realizing automatic updating of the model and having higher efficiency.

Description

Model updating method and related device

Technical Field

The invention relates to the field of artificial intelligence, in particular to a model updating method and a related device.

Background

With the continuous development of artificial intelligence technology and the further integration of internet finance, machine learning is also gradually applied to quality assessment in the delivery process of banking products. The product quality evaluation model is constructed based on data (work split data at a demand analysis stage, code data at a development stage, defect data at a test stage, etc.) at each stage of product delivery to predict the quality at the time of product delivery.

However, with the continuous popularization of agile delivery in banking industry, the iteration cycle of products is shorter and shorter, the types of models are more and more, the evaluation standard is changed frequently, and the product quality evaluation model also needs to be iterated in time to adapt to new situations. Therefore, the current model updating in time becomes a technical problem that needs to be solved urgently by those skilled in the art.

Disclosure of Invention

In view of the above, the present invention provides a model updating method and related apparatus that overcome or at least partially solve the above problems.

In a first aspect, a model updating method includes:

obtaining a model to be updated and a corresponding training index;

acquiring a new training sample corresponding to the model from a database;

performing the training on the model based on the new training sample and the training index;

after the training is finished, storing a model file corresponding to the model;

and updating the model by loading the model file on line.

With reference to the first aspect, in some optional embodiments, the obtaining the model to be updated and the corresponding training indicators includes:

obtaining a model updating request, wherein the model updating request carries a model identification;

and acquiring the pre-established model and the training index from a model library according to the model identification.

With reference to the first aspect, in some optional embodiments, the performing training on the model based on the new training sample and the training indicator includes:

performing feature processing on the new training sample to obtain a new feature;

and performing the training on the model by using the new characteristics.

With reference to the first aspect, in some optional embodiments, after the training of the model based on the new training sample and the training indicator, the method further includes:

calling a sklern toolkit, and calculating an AUC value, a true positive rate tpr and a false positive rate fpr corresponding to the model;

evaluating the performance of the model based on the AUC values, the true positive rate tpr and the false positive rate fpr.

and after the training is finished, storing the training log of the training and the training index of the training corresponding to the model.

With reference to the first aspect, in some optional embodiments, the updating the model by online loading the model file includes:

loading the decision tree recorded in the model file on line, and reading the branching condition of the decision tree recorded in the model file on line so as to load the model file;

and replacing the original model file with the model file, thereby updating the model.

In a second aspect, a model updating apparatus includes: the device comprises a model index acquisition unit, a training sample acquisition unit, a model training unit, a model file storage unit and a model updating unit;

the model index acquisition unit is used for acquiring a model to be updated and a corresponding training index;

the training sample acquisition unit is used for acquiring a new training sample corresponding to the model from a database;

the model training unit is used for carrying out the training on the model based on the new training sample and the training index;

the model file storage unit is used for storing the model file corresponding to the model after the training is finished;

and the model updating unit is used for loading the model file on line so as to update the model.

With reference to the second aspect, in some optional embodiments, the model index obtaining unit includes: an update request acquisition subunit and a model index acquisition subunit;

the update request obtaining subunit is configured to obtain a model update request, where the model update request carries a model identifier;

and the model index acquisition subunit is used for acquiring the pre-established model and the training index from a model library according to the model identification.

In a third aspect, a computer-readable storage medium has stored thereon a program which, when executed by a processor, implements the model updating method of any of the above.

In a fourth aspect, an electronic device comprises at least one processor, and at least one memory, bus, connected to the processor; the processor and the memory complete mutual communication through the bus; the processor is configured to call program instructions in the memory to perform any of the model update methods described above.

By means of the technical scheme, the model updating method and the related device provided by the invention can be used for updating the model to be updated and the corresponding training indexes; acquiring a new training sample corresponding to the model from a database; performing the training on the model based on the new training sample and the training index; after the training is finished, storing a model file corresponding to the model; and updating the model by loading the model file on line. Therefore, the method can automatically and timely retrain the model based on the new training sample and the new training index, thereby realizing automatic updating of the model and having higher efficiency.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 shows a flow chart of a first model update method;

FIG. 2 shows a flow chart of a second model updating method;

FIG. 3 shows a flow chart of a third model update method;

FIG. 4 shows a flow chart of a fourth model update method;

FIG. 5 shows a flow chart of a fifth model update method;

FIG. 6 shows a flow chart of a sixth model update method;

FIG. 7 is a schematic diagram showing a structure of a model updating apparatus;

fig. 8 shows a schematic structural diagram of an electronic device.

Detailed Description

It should be noted that the model updating method and the related device provided by the invention can be applied to the fields of artificial intelligence and finance. The above description is only an example, and does not limit the application fields of the model updating method and the related apparatus provided by the present invention.

The model updating method and the related device provided by the invention can be used in the financial field or other fields, for example, can be used in a model updating application scenario in the financial field. Other fields are any fields other than the financial field, for example, the field of artificial intelligence. The above description is only an example, and does not limit the application fields of the model updating method and the related apparatus provided by the present invention.

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

As shown in fig. 1, the present invention provides a model updating method, including: s100, S200, S300, S400, and S500;

s100, obtaining a model to be updated and a corresponding training index;

optionally, the invention may pre-construct models of different application scenarios, and may store the models in the model library after initial training of the models, so as to call and obtain corresponding models when needed.

Optionally, the model may be a decision tree model obtained based on decision tree training, which is not limited in the present invention.

Optionally, for said training index of the invention: different models, the corresponding training indexes can be different; the same model, the training metrics used in the update may be different.

Optionally, the method for obtaining the model to be updated and the corresponding training index is not specifically limited. For example, as shown in fig. 2, in combination with the embodiment shown in fig. 1, in some alternative embodiments, the S100 includes: s110 and S120;

s110, obtaining a model updating request, wherein the model updating request carries a model identification;

optionally, the execution subject of the present invention may obtain a model update request sent by a manager, and then parse the model update request to obtain information such as a corresponding model identifier (e.g., a model ID) from the model update request. In addition, the model update request may also carry information such as the model type, the data deadline to be used for training, and the threshold determination index of the label in the training data.

And S120, acquiring the pre-established model and the training index from a model library according to the model identification.

Optionally, the model identifiers correspond to the models one to one, and the corresponding relationship can be pre-established and stored in the invention, so that the corresponding models can be obtained according to the model identifiers when needed. It should be noted that: the training indicators are matched to the models, i.e. different models may match different training indicators. The training indexes of different models can be preset or input by related personnel when needed. For example, for a specific model, it may be iteratively updated for multiple times, and each iterative update requires retraining the model based on a new training index, so that before training, the executive subject of the present invention may obtain the new training index for the model input by the relevant person, which is not limited in this disclosure.

Alternatively, the model of the present invention may be a classification model, and usually the defect rate at a certain stage is higher than a certain value for discrimination. For example, if the standard of this year is 0.008, a value higher than 0.008 will be marked as 1, and a value equal to or lower than 0.008 will be marked as 0, and the training index can be understood as a value of 0.008.

Alternatively, the model of the present invention may be stored in the form of a file on the application deployment server, and the relevant information of the model (such as the corresponding id and description of the model) may be stored on the database.

S200, acquiring a new training sample corresponding to the model from a database;

optionally, as described above, the present invention may implement the purpose of model update iteration by retraining the model. Therefore, the invention can obtain a new training sample so as to be convenient for retraining the model based on the new training sample.

Optionally, the model is retrained subsequently based on a new training sample, and the performance of the model obtained by final training is definitely different from that of the model before training, so that the purpose of updating and iterating the model is realized.

S300, training the model at this time based on the new training sample and the training index;

optionally, the process of model training is not specifically limited by the present invention, and reference may be made to the process of model training in the related art, which is not limited by the present invention.

For example, as shown in fig. 3, in combination with the embodiment shown in fig. 1, in some alternative embodiments, the S300 includes: s210 and S220;

s210, performing feature processing on the new training sample to obtain a new feature;

optionally, the features of the model are fixed during the first training, so that each subsequent update is a rework of the model. The invention can adopt some common processing modes, for example, aiming at the characteristic of code defects, the original data is the number of code defects of each product, when the code defect is processed, the invention is used for counting the characteristic and frequency characteristic, and finally, new characteristics such as the number of the code defects of the product, the main defect type of the code defects, the secondary defect type of the code defects and the like are generated.

And S220, performing the training on the model by using the new characteristics.

Optionally, the present invention may employ an open source training tool, i.e., the training tool has packaged the associated algorithm. For example, the lightgbm tool can be used, the data is obtained in the front, and when retraining, only the parameters of the first training are needed to be used, the training data used by the training is input, and the training function is called, so that a new model can be obtained through training.

Alternatively, the lightgbm tool belongs to the known concept in the field, and the invention does not make much description, and refer to the related description in the field specifically.

Optionally, after feature screening and feature processing are performed, newly added features may be added to corresponding feature dictionaries, so that before prediction is performed by using models, characterization is performed on data to be tested based on the feature dictionaries, where one model corresponds to one feature dictionary.

Optionally, after the model is trained, the performance of the model can be evaluated, and a new version of the model is released after the evaluation is passed. For example, as shown in fig. 4 in combination with the embodiment shown in fig. 1, in some optional embodiments, after S300, the method further includes: s310 and S320;

s310, calling a sklern toolkit, and calculating an AUC value, a true positive rate tpr and a false positive rate fpr corresponding to the model;

s320, evaluating the performance of the model according to the AUC value, the true positive rate tpr and the false positive rate fpr.

Optionally, the invention may use a sklern toolkit to calculate an AUC value corresponding to the model, in addition, the true positive rate tpr and the false positive rate fpr may be calculated, and a corresponding ks value may be calculated based on the true positive rate tpr and the false positive rate fp. The ks value is understood to be the maximum of the absolute values of the true positive rate tpr and the false positive rate fpr. Besides, the method can be used for evaluating the performance of the model based on the accuracy, recall rate, precision rate and the like of the model obtained through calculation.

It should be noted that: the sklern toolkit, the AUC value, the true positive rate tpr, the false positive rate fpr good ks value, etc. all belong to the known concepts in the field, and the present invention does not describe this more, and please refer to the related description in the field specifically.

As shown in fig. 5, in combination with the embodiment shown in fig. 1, in some optional embodiments, after S300, the method further includes: s410;

and S410, after the training is finished, storing the training log of the training and the training index of the training corresponding to the model.

Optionally, as described above, the present invention may store some corresponding information after training the model of the new version, so as to facilitate consulting when needed. For example, the training log according to the present invention may record the conditions of training samples used in the present training, including: sample number and sample source, etc.

Optionally, the training index of the training may be stored in the database, so as to refer to the condition of the training index based on each update when needed, which is not limited in the present invention.

S400, storing a model file corresponding to the model after the training is finished;

optionally, the model of the present invention may be a decision tree model, and therefore, the corresponding model file may record the decision tree and the bifurcation condition. Decision trees are well known concepts in the art and are not described in great detail herein. The branch condition refers to a branch rule of each branch in the decision tree, which is not limited in the present invention.

And S500, updating the model by loading the model file on line.

Optionally, the present invention does not specifically limit the manner in which the model file is loaded. For example, as shown in fig. 6, in combination with the embodiment shown in fig. 1, in some alternative embodiments, the S500 includes: s510 and S520;

s510, loading the decision tree recorded in the model file online, and reading the branching condition of the decision tree recorded in the model file online so as to load the model file;

and S520, replacing the original model file with the model file, and updating the model.

Optionally, relevant personnel may select a new version of the model file online according to actual needs, and then click and load the model file, so as to load and use the new version of the model file to replace the original model file, thereby updating the corresponding model.

Optionally, it should be noted that: the execution main body of the invention can also obtain corresponding data according to the prediction requirement of a user, load the feature dictionary of the corresponding model to complete feature processing during feature processing, and predict the result by the current model.

As shown in fig. 7, the present invention provides a model updating apparatus, including: a model index acquisition unit 100, a training sample acquisition unit 200, a model training unit 300, a model file storage unit 400, and a model updating unit 500;

the model index obtaining unit 100 is configured to obtain a model to be updated and a corresponding training index;

the training sample obtaining unit 200 is configured to obtain a new training sample corresponding to the model from a database;

the model training unit 300 is configured to perform the current training on the model based on the new training sample and the training index;

the model file storage unit 400 is configured to store a model file corresponding to the model after the training is finished;

the model updating unit 500 is configured to update the model by loading the model file online.

With reference to the embodiment shown in fig. 7, in some optional embodiments, the model index obtaining unit 100 includes: an update request acquisition subunit and a model index acquisition subunit;

the update request acquisition subunit is configured to acquire a model update request, where the model update request carries a model identifier;

and the model index obtaining subunit is configured to obtain the pre-established model and the training index from a model library according to the model identifier.

In some optional embodiments, in combination with the embodiment shown in fig. 7, the model training unit 300 includes: a feature processing subunit and a model training subunit;

the characteristic processing subunit is used for carrying out characteristic processing on the new training sample so as to obtain a new characteristic;

and the model training subunit is used for performing the training on the model by using the new features.

In some alternative embodiments, in combination with the embodiment shown in fig. 7, the apparatus further comprises: a parameter value calculation unit and an evaluation unit;

the parameter value calculating unit is configured to, after the model is trained for this time based on the new training sample and the training index, call a sklern toolkit, and calculate an AUC value, a true positive rate tpr, and a false positive rate fpr corresponding to the model;

the evaluation unit is used for evaluating the performance of the model according to the AUC value, the true positive rate tpr and the false positive rate fpr.

In some alternative embodiments, in combination with the embodiment shown in fig. 7, the apparatus further comprises: a log index storage unit;

and the log index storage unit is used for storing the training log of the training and the training index of the training corresponding to the model after the model is trained for the time based on the new training sample and the training index and after the training for the time is finished.

In some optional embodiments, in combination with the embodiment shown in fig. 7, the model updating unit 500 includes: a logging subunit and a replacement subunit;

the recording subunit is configured to load the decision tree recorded in the model file online, and read the branching condition of the decision tree recorded in the model file online, so as to load the model file;

and the replacing subunit is used for replacing the original model file with the model file so as to update the model.

The present invention provides a computer readable storage medium having a program stored thereon, which when executed by a processor implements any of the model updating methods described above and related apparatus methods.

As shown in fig. 8, the present invention provides an electronic device 70, wherein the electronic device 70 comprises at least one processor 701, at least one memory 702 connected to the processor 701, and a bus 703; the processor 701 and the memory 702 complete communication with each other through the bus 703; the processor 701 is configured to call program instructions in the memory 702 to perform any of the model update methods and related apparatus methods described above.

In the present invention, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A model update method, comprising:

obtaining a model to be updated and a corresponding training index;

acquiring a new training sample corresponding to the model from a database;

and updating the model by loading the model file on line.

2. The method of claim 1, wherein the obtaining the model to be updated and the corresponding training indicators comprises:

obtaining a model updating request, wherein the model updating request carries a model identifier;

3. The method of claim 1, wherein the training the model based on the new training samples and the training indicators comprises:

and performing the training on the model by using the new characteristics.

4. The method of claim 1, wherein after the training the model based on the new training samples and the training indicators, the method further comprises:

5. The method of claim 1, wherein after the training the model based on the new training samples and the training indicators, the method further comprises:

6. The method of claim 1, wherein updating the model by loading the model file online comprises:

7. A model updating apparatus, comprising: the device comprises a model index acquisition unit, a training sample acquisition unit, a model training unit, a model file storage unit and a model updating unit;

8. The apparatus according to claim 7, wherein the model index obtaining unit includes: an update request acquisition subunit and a model index acquisition subunit;

9. A computer-readable storage medium on which a program is stored, the program, when being executed by a processor, implementing the model updating method according to any one of claims 1 to 6.

10. An electronic device comprising at least one processor, and at least one memory, bus connected to the processor; the processor and the memory complete mutual communication through the bus; the processor is configured to invoke program instructions in the memory to perform the model update method of any of claims 1 to 6.