CN111176705A - Feature library upgrading method and device - Google Patents

Feature library upgrading method and device Download PDF

Info

Publication number
CN111176705A
CN111176705A CN201911262764.7A CN201911262764A CN111176705A CN 111176705 A CN111176705 A CN 111176705A CN 201911262764 A CN201911262764 A CN 201911262764A CN 111176705 A CN111176705 A CN 111176705A
Authority
CN
China
Prior art keywords
feature
characteristic
record
field
library
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911262764.7A
Other languages
Chinese (zh)
Other versions
CN111176705B (en
Inventor
邹晓园
王润泽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201911262764.7A priority Critical patent/CN111176705B/en
Publication of CN111176705A publication Critical patent/CN111176705A/en
Application granted granted Critical
Publication of CN111176705B publication Critical patent/CN111176705B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/60Software deployment
    • G06F8/65Updates
    • G06F8/658Incremental updates; Differential updates

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

An embodiment of the present application provides a method for upgrading a feature library, including: receiving an upgrade instruction of a feature library, wherein the feature library comprises a feature record of a feature, the feature record comprises a first field and a second field configured for the feature, and the first field is assigned as a pre-update feature value of the feature; obtaining a stock upgrading file of the feature library according to the upgrading instruction; acquiring a first characteristic value of the characteristic from the stock upgrading file; in the feature record of the feature, assigning the second field as the first feature value to obtain an updated feature record of the feature; and writing the updated feature record into the feature library to replace the feature record of the feature in the feature library, thereby realizing updating the feature library without service interruption.

Description

Feature library upgrading method and device
Technical Field
The application relates to the technical field of computers and communication, in particular to a method and a device for upgrading a feature library.
Background
The algorithm model needs to be optimized and iterated along with the change of an application scene, namely, the algorithm model is upgraded from an old algorithm model to a new algorithm model. The new algorithm model obtained by upgrading cannot be compatible with the old algorithm model, so that the feature library of the algorithm model needs to be upgraded correspondingly.
In the prior art, the upgrade of the feature library needs to be stopped, that is, the service cannot be provided for the user during the upgrade of the feature library.
Disclosure of Invention
The embodiment of the application provides a method and a device for upgrading a feature library, so that the feature library is upgraded without being disturbed.
Other features and advantages of the present application will be apparent from the following detailed description, or may be learned by practice of the application.
According to an aspect of an embodiment of the present application, there is provided a method for upgrading a feature library, including:
receiving an upgrade instruction of a feature library, wherein the feature library comprises a feature record of a feature, the feature record comprises a first field and a second field configured for the feature, and the first field is assigned as a pre-update feature value of the feature;
obtaining a stock upgrading file of the feature library according to the upgrading instruction;
acquiring a first characteristic value of the characteristic from the stock upgrading file;
in the feature record of the feature, assigning the second field as the first feature value to obtain an updated feature record of the feature;
and writing the updated feature record into the feature library to replace the feature record of the feature in the feature library.
According to an aspect of an embodiment of the present application, there is provided an upgrade apparatus for a feature library, including:
the system comprises a receiving module, a judging module and a judging module, wherein the receiving module is used for receiving an upgrading instruction of a feature library, the feature library comprises a feature record of a feature, the feature record comprises a first field and a second field configured for the feature, and the first field is assigned as a feature value before the feature is updated;
the stock upgrading file acquisition module is used for acquiring the stock upgrading file of the feature library according to the upgrading instruction;
the first characteristic value acquisition module is used for acquiring a first characteristic value of the characteristic from the stock upgrading file;
the updating module is used for assigning the second field to be the first characteristic value in the characteristic record of the characteristic to obtain an updated characteristic record of the characteristic;
and the writing module is used for writing the updated feature record into the feature library so as to replace the feature record of the feature in the feature library.
In some embodiments of the present application, a pre-update feature value and a post-update feature value of a feature are stored by configuring a dual field, i.e., a first field and a second field, for each feature in a feature record. Because the reading and writing operations of the feature records in the feature library are performed in the storage layer without influencing the retrieval service provided by the retrieval layer, the external service can be provided according to the data in the cache of the retrieval layer in the upgrading process of the feature library, and the updating of the feature library without being delayed is realized.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
FIG. 1 is a flow diagram illustrating a method for upgrading a feature library according to one embodiment;
FIG. 2 is a flowchart illustrating steps subsequent to step 110, according to one embodiment;
FIG. 3 is a flow diagram of step 250 of the corresponding embodiment of FIG. 1 in one embodiment;
FIG. 4 is a flow diagram of step 250 of the corresponding embodiment of FIG. 1 in another embodiment;
FIG. 5 is a flow diagram of steps in one embodiment after step 190 of the corresponding embodiment of FIG. 1;
FIG. 6 is a flow diagram of steps in one embodiment after step 530 of the corresponding embodiment of FIG. 5;
FIG. 7 is a flow diagram of steps in one embodiment after step 650 of the corresponding embodiment of FIG. 6;
FIG. 8 is a state diagram illustrating the upgrade of a feature library, according to one embodiment;
FIG. 9 is a process diagram illustrating an upgrade of a feature library, according to one embodiment;
FIG. 10 is a block diagram illustrating an apparatus for upgrading a feature library, according to one embodiment;
FIG. 11 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Fig. 1 is a flowchart illustrating an upgrading method of a feature library according to an embodiment, where as shown in fig. 1, the method at least includes steps 110 to 190, which are as follows:
and 110, receiving an upgrading instruction of a feature library, wherein the feature library comprises a feature record of the feature, the feature record comprises a first field and a second field configured for the feature, and the first field is assigned as a feature value before the feature is updated.
The feature library is used for storing relevant information of the features, such as feature values of the features. The feature library is updated by, for example, adding a new feature to the feature library, updating a feature value of an original feature, deleting a feature, and the like, and is not limited specifically herein. It will be appreciated that in the feature library, features are represented by feature identifiers.
Specifically, in the feature library, the related information of the feature is represented as a feature record, that is, a feature record corresponds to the related information of a feature.
In the scheme of the disclosure, the feature record of the feature at least comprises a feature identification of the feature, a first field and a second field, wherein the first field and the second field are both used for characterizing the feature value of the feature. In one embodiment, the feature record may be: fea _ id feature0 feature1, where fea _ id represents a feature id, entry _ id represents a data entity representation, feature0 is a first field, and feature1 is a second field.
In the upgrade process of the feature library, feature value update of the feature is involved, that is, in each upgrade process, there are feature values corresponding to the feature before update (i.e., feature values before update) and feature values corresponding to the feature after update.
In the solution of this embodiment, in order to avoid copying the feature library, two fields are configured for the features in the feature record to store the feature value corresponding to the pre-update feature value and the feature value corresponding to the post-update feature value, respectively.
In the present embodiment, the first field is used to store the pre-update feature value of the feature, that is, the first field is assigned as the pre-update feature value in the feature record. While the second field in the feature record is not assigned when no feature update is performed.
And step 130, acquiring a stock upgrading file of the feature library according to the upgrading instruction.
The upgrading of the feature library relates to stock upgrading, which is to update the feature library in a full amount. In particular, for features present in the feature library, these features may not necessarily have active update behavior, and thus need to be upgraded on an inventory basis.
That is, the upgrade instruction of the present disclosure is initiated for performing a stock upgrade of the feature library, and therefore, the stock upgrade of the feature library is initiated based on the upgrade instruction.
The stock upgrading file refers to upgrading packages required by stock upgrading. Therefore, the stock upgrading file indicates that the features existing in the feature library correspond to the upgraded feature values, so that the stock upgrading of the feature library is performed according to the stock upgrading file.
In one embodiment, the inventory upgrade file may be a preconfigured file.
Step 150, obtaining a first feature value of the feature from the inventory upgrade file.
The first characteristic value is an updated characteristic value of the characteristic indicated by the stock upgrade file.
In step 170, in the feature record of the feature, the second field is assigned as the first feature value, so as to obtain the updated feature record of the feature.
The feature record of the feature is derived from the feature library, that is, in order to update the feature, the feature record of the feature is correspondingly read from the feature library, and a second field in the read feature record is assigned as the first feature value, and the second field corresponds to a second feature value (updated feature value) in which the feature is stored.
The updated feature record refers to a feature record in which the second field is assigned to the first feature value in the feature record of the feature.
And 190, writing the updated feature records into the feature library to replace the feature records of the features in the feature library.
Since the updated feature record is obtained by editing the feature record read from the feature library, the updated feature record is written into the feature library, that is, the feature record in the feature library is replaced by the updated feature record.
To this end, in the aspect of the present disclosure, a pre-update feature value and a post-update feature value of a feature are stored by configuring a double field, i.e., a first field and a second field, for each feature in a feature record, respectively. Because the reading and writing operations of the feature records in the feature library are performed in the storage layer, and the retrieval service provided by the retrieval layer to the outside is not influenced, the retrieval layer can continue to provide the service to the outside in the upgrading process of the feature library, and the updating of the feature library without being waited is realized.
In one embodiment, as shown in fig. 2, after step 110, the method further comprises:
step 210, an incremental upgrade file of the feature library is obtained, where the incremental upgrade file indicates the features to be written.
For feature library upgrades, incremental upgrades are also involved. The incremental upgrading refers to adding new features in the feature library or only updating the features which need to be updated in the feature library.
The to-be-written feature refers to a feature that the updated feature record needs to be written into the feature library in the incremental upgrade process, and specifically, the to-be-written feature may be a newly added feature or a feature that updates a feature value.
In other words, at least one of a feature addition and a feature value update is involved in the incremental upgrade process. The newly added features in the incremental upgrading process are called as new features, and the features in the feature library which need to be updated in the upgrading process are called as updated features.
The incremental upgrade file of the feature library is a file package for performing incremental upgrade on the feature library, and the incremental upgrade file may be a file independent of the stock upgrade file, or an upgrade file generated correspondingly due to the need of updating the feature library caused by the triggering of the business layer in the stock upgrade process, or an upgrade file generated correspondingly due to the need of upgrading a certain feature in the stock upgrade process due to the operational logic configured in the algorithm model. Of course, the above is merely an illustrative example, and in other embodiments, the incremental upgrade file may also be derived from other scenarios requiring incremental upgrade of the feature library.
The increment upgrade file at least indicates the characteristic value of the to-be-written characteristic after increment upgrade, and the characteristic value of the to-be-written characteristic after increment upgrade is the new characteristic value of the to-be-written characteristic. Specifically, for the new feature, the feature value of the new feature indicated in the incremental upgrade file is the new feature value of the new feature, and for the updated feature, the feature value of the updated feature indicated in the incremental upgrade file after the updated feature is updated is the new feature value of the updated feature.
And step 230, acquiring a new characteristic value corresponding to the to-be-written characteristic according to the increment upgrading file. And step 250, generating a writing characteristic record for the characteristics to be written according to the new characteristic value, wherein in the writing characteristic record, the first field of the characteristics to be written is assigned as the old characteristic value of the characteristics to be written, and the second field is assigned as the new characteristic value.
As described above, the feature to be written may be a new feature or an updated feature. For the updated feature, the old feature value of the updated feature is the current feature value of the updated feature, that is, the feature value before incremental upgrade.
For the new feature, the old feature value of the new feature may be a preset feature value, in other words, for the new feature, a feature value is preset as the old feature value. In another embodiment, when the feature library corresponds to the algorithm model, the old feature value of the newly added feature may be the feature value of the feature under the old algorithm model, and the new feature value of the newly added feature may be the feature value of the feature under the new algorithm model.
Therefore, on the basis of acquiring the new characteristic value and the old characteristic value of the characteristic to be written, a written characteristic record is correspondingly generated for the characteristic to be written.
Step 270, writing the written feature record into the feature library. And writing the written feature record of the feature to be written into the feature library to realize the upgrading of the feature to be written.
In one embodiment, the feature library corresponds to the algorithm model, and the feature library is updated after the algorithm model is updated from an old algorithm model to a new algorithm model; the features to be written include the added features, in this embodiment, step 230 includes:
and acquiring a third characteristic value of the newly added characteristic under the new algorithm model according to the increment upgrading file, and taking the third characteristic value as a new characteristic value.
For the algorithm model, due to the change of a specific application scene, optimization and iteration are correspondingly required, that is, the algorithm model is upgraded, and the upgraded algorithm model cannot be compatible with the old algorithm model, so stock data of the algorithm model, that is, a feature library of the algorithm model, needs to be correspondingly upgraded.
The algorithm model before upgrading is referred to as the old algorithm model. And the upgraded algorithm model is called a new algorithm model.
The increment updating file comprises a third characteristic value of the newly added characteristic under the new algorithm model, so that the third characteristic value of the newly added characteristic is correspondingly obtained from the increment updating file and is used as a new characteristic value. In this embodiment, as shown in fig. 3, step 250 includes:
step 310, acquiring a second characteristic value of the newly added feature under the old algorithm model, and taking the second characteristic value as an old characteristic value; and
in step 330, an initial feature record configured for the newly added feature is obtained.
And 350, in the initial characteristic record, assigning the first field as an old characteristic value and assigning the second field as a new characteristic value to obtain a write-in characteristic record of the new characteristics.
The second feature value of the newly added feature under the old algorithm model may be a feature value specified in the incremental upgrade file, or may be a pre-configured feature value, which is not specifically limited herein.
For the added feature, the feature library does not include the feature record of the added feature, and in order to obtain the written feature record of the added feature, an initial feature record is configured for the added feature in advance. It will be appreciated that the initial feature record has configured a first field and a second field for the newly added feature, respectively. Further, the initial feature record also includes a feature representation of the added feature.
And then, in the initial feature record of the newly added feature, the first field is assigned as the old feature value of the newly added feature, and the second field is assigned as the new feature value of the newly added feature, so as to obtain the write-in feature record of the newly added feature.
In one embodiment, the to-be-written feature includes an update feature, as shown in fig. 4, step 250 includes:
step 410, obtaining the feature record of the updated feature from the feature library, and using the value of the first field in the feature record of the updated feature as the old feature value.
And 430, assigning the second field as a new feature value in the feature record of the updated feature to obtain the write-in feature record of the updated feature.
For the updated feature, the feature library stores a feature record of the updated feature. Therefore, the feature record of the updated feature is read from the feature library, and the value of the first field in the read feature record is the old feature value of the updated feature. And then assigning a second field in the feature record of the update feature as a new feature value of the update feature, and correspondingly obtaining a write-in feature record of the update feature.
In an embodiment, the incremental upgrade file indicates the feature to be deleted, and after step 210, the method further includes: and deleting the feature records of the features to be deleted in the feature library.
In one embodiment, as shown in fig. 5, after step 190, the method further comprises:
step 510, traversing the feature records in the feature library according to the inventory upgrade file.
Step 530, if the characteristics in the characteristic library are traversed and determined to complete the stock upgrading, loading the new model data of the algorithm model corresponding to the characteristic library into a cache.
The stock upgrade file indicates the feature value of the feature included in the feature library before the feature library is upgraded after the stock upgrade, in other words, the feature included in the feature library before the feature library is upgraded can be known from the stock upgrade file. Therefore, traversing in the feature library according to the stock upgrading file can acquire whether the features related in the stock upgrading file are upgraded all the time, and if the features are upgraded all the time, the stock upgrading is finished; otherwise, if the features involved in the inventory upgrade file are not all upgraded, it indicates that the inventory upgrade is not completed.
For the algorithm model, inventory upgrade is performed as support for a new algorithm model, and if inventory upgrade is completed on the feature library, the feature library after inventory upgrade is completed can at least support the new algorithm model. Therefore, after the stock upgrading is completed, the new model data of the algorithm model is loaded into the cache, and therefore, the retrieval can be performed according to the new algorithm model based on the new model data.
The new model data is used to describe the new algorithm model, and similarly, the old model data is used to describe the old algorithm model.
It is worth mentioning that for retrieval, the storage layer is not directly accessed, i.e. the feature library is not accessed, but the data in the cache is used for retrieval. In the writing process of the feature library, the written data is correspondingly and synchronously updated into the cache, in other words, as the upgrading is carried out, the cache comprises the old feature value and the new feature value of the feature.
In the upgrading process, old model data of the algorithm model are also included in the cache. That is, after the stock upgrade is completed, the cache includes the old eigenvalue of the characteristic, the new eigenvalue, and the old model data, and after the new model data is loaded into the cache, on one hand, the retrieval can be performed based on the new model data and the new eigenvalue of the characteristic; on the other hand, the search may be performed based on old model data and old feature values of the features. That is, the search service may switch the algorithm model for searching.
After the upgrade is started and the upgrade process of the stock is not finished, the cache includes old model data and old characteristic values of the characteristics, so that the retrieval service can be provided based on the old algorithm model. Therefore, in the whole upgrading process, the cache always has the data required by correspondingly providing the retrieval service, so that the uninterrupted upgrading of the feature library is realized.
In an embodiment, the cache includes old model data of the algorithm model, and when the feature record is written into the feature library, the values of the first field and the second field are saved in the cache, as shown in fig. 6, and after step 530, the method further includes:
step 610, receiving a retrieval request, wherein the retrieval request comprises an indication identifier, and the indication identifier is used for indicating retrieval in the first field or retrieval in the second field.
In step 630, if the indication identifier indicates to search in the first field, the search is performed in the first field according to the old model data, and a first search result is obtained.
And step 650, if the indication mark indicates to search in the second field, searching in the second field according to the new model data to obtain a second search result.
As described above, after the inventory upgrade is completed and the new model data is loaded into the cache, the search service under the new algorithm model and the old algorithm model can be provided based on the data in the cache.
The characteristic record in the cache comprises a first field and a second field, wherein the value of the first field corresponds to the old characteristic value of the characteristic, and the value of the second field corresponds to the new characteristic value of the special diagnosis. For the feature library, the old feature value of the feature can support the old algorithm model to provide retrieval service; the new feature values of the features may support the new algorithm model to provide retrieval services.
On the basis, if the indication mark in the retrieval request indicates retrieval in the first field, namely retrieval is carried out in the old characteristic value, the retrieval is carried out in the first field according to the old model data, and a first retrieval result is correspondingly obtained; and if the indication identifier in the retrieval request indicates that the retrieval is carried out in the second field, namely the retrieval is carried out in the new characteristic value, the retrieval is carried out in the second field according to the new model data, and a second retrieval result is correspondingly obtained.
In one embodiment, as shown in fig. 7, after step 650, the method further comprises:
and 710, calculating test parameters according to the second retrieval result, wherein the test parameters are used for representing the performance of the new algorithm model indicated by the new model data.
And 730, if the test parameters meet the preset conditions, generating an upgrade confirmation instruction, and deleting the value of the first field and the old model data according to the upgrade confirmation instruction.
And step 750, if the test parameters do not meet the preset conditions, generating a rollback instruction, and deleting the value of the second field and the new model data according to the rollback instruction.
For algorithmic model upgrades, verification of a new algorithmic model is typically required, and the verification test performed is referred to as an AB test (ABTest).
And in the AB test of the new algorithm model, the testing party sends a retrieval request, the indication mark in the retrieval request indicates retrieval in the second field, and a second retrieval result is obtained on the basis of the retrieval request.
And after the second retrieval result is obtained, calculating the test parameters according to the second retrieval result. In one embodiment, the test parameters may be recall, accuracy of search results, and other parameters that can measure the performance of the new algorithm model.
In one embodiment, the predetermined condition may be a parameter range set for the test parameter. In one embodiment, the parameter range may be set according to the value of the test parameter calculated from the first search result. In other embodiments, the parameter range may also be set according to practical experience, and is not specifically limited herein.
If the test parameters meet the preset conditions, the new algorithm model is shown to be evolved compared with the old algorithm model or the new algorithm model meets the requirements, and therefore an upgrade confirmation instruction is correspondingly generated so as to delete the old data in the cache and retrieval layer according to the upgrade confirmation instruction. Specifically, the old data includes the value of the first field and the old model data.
In one embodiment, after the upgrade confirmation instruction is generated, the upgrade confirmation instruction is sent to the terminal, so that a user can confirm the upgrade confirmation instruction in the terminal, and therefore after the upgrade confirmation instruction is confirmed, old data in the cache and retrieval layer is deleted according to the upgrade confirmation instruction.
Further, after deleting old data in the cache and retrieval layer, assigning the first field as the value of the second field and clearing the value of the second field for the retrieval record in the feature library.
If the test parameters do not meet the preset conditions, the new algorithm model is degraded compared with the old algorithm model or the new algorithm model does not meet the requirements, so that a rollback instruction is correspondingly generated, and new data in the cache and the retrieval layer are deleted according to the rollback instruction. In particular, the new data includes the value of the second field and the new model data.
In one embodiment, the algorithm model includes at least two feature libraries, and after deleting the value of the first field and deleting the old model data according to the upgrade validation instruction, the method further includes:
and receiving an upgrading instruction sent for the next feature library.
If the full feature library of the algorithm model is large, and if the algorithm model is upgraded in full, the overhead of the cache layer is large. Therefore, in order to solve the problem, the features of the algorithm model are divided into at least two feature libraries, and then the library is used as granularity for upgrading, so that the overhead of a cache layer is reduced.
Further, the upgrading state of each feature library can be displayed in the terminal. FIG. 8 is an upgrade state diagram that illustrates a feature library, according to one embodiment.
As shown in fig. 8, before upgrading, the feature library required to be upgraded and the state of the feature library are displayed in the user interface of the terminal, where the state of the feature library includes the feature library to be upgraded and the upgrade is completed. As shown in I in fig. 8, the feature library to be upgraded includes a feature library a, a feature library B, and a feature library C, and the user interface further includes a status display box 810, where "upgrade" in the status display box 810 indicates that the feature library is in a state to be upgraded.
Further, the status display box 810 may also be used as an operation portal through which an upgrade instruction of a corresponding feature library is transmitted. As shown by I in fig. 8, the status display box 810 next to the displayed feature library a is regarded as an operation entry of the feature library a. If the user operates the operation entry, that is, the user regards as sending an upgrade instruction corresponding to the feature library a, after receiving the upgrade instruction, the server performs upgrade on the feature library a correspondingly according to the method disclosed by the present disclosure.
As shown in II in fig. 8, during the upgrade process of the feature library a, the upgrade progress of the feature library a is fed back through the progress bar 820.
And entering an upgrading switching state after the characteristic library finishes stock upgrading. The upgrade switching state is a state in which new and old model data is stored in the cache. As shown in III in fig. 8, the upgrade switching status is displayed through the second status display box 830.
In another embodiment, after the second status display box 810 is displayed, a prompt dialog box may be correspondingly displayed, where the prompt dialog box is used to prompt a user whether to confirm the upgrade confirmation instruction, and if the user indicates confirmation, step 730 is executed to implement the upgrade of the feature library a.
And after the characteristic library A is upgraded, correspondingly updating the upgrade state of the characteristic library A. As shown in IV in fig. 8, the upgrade state of feature library a is complete, and the states of feature libraries B and C are still to be upgraded.
The method of the present disclosure is described below in conjunction with a specific embodiment.
Fig. 9 is a schematic diagram illustrating the process of upgrading the feature library in this embodiment, and as shown in fig. 9, after receiving an upgrade instruction of the feature library, the processes of incremental upgrade 910 and inventory upgrade 920 are performed. It should be noted that there is no dependency between the incremental upgrade 910 and the inventory upgrade 920, and the incremental upgrade 910 may be performed while the inventory upgrade 920 is performed.
In the stock upgrading process, the upgrading of the features is involved, and the upgrading process of the features is represented as follows: and keeping the first field in the feature record unchanged, and newly adding the value of the second field.
In the process of incremental upgrading, feature adding, feature updating and feature deleting are involved. Wherein, the newly added process of the characteristics is represented as: adding a value of a first field of the characteristics and a value of a second field; the process of feature update is represented as: updating the value of the first field and adding/updating the value of the second field; the process of feature deletion is represented as: deleting the feature record of the feature.
After the inventory upgrade 920 is complete, an upgrade switch state 930 is entered. In the upgrade switch state 930, retrieval with new or old feature values, respectively, is supported. Further, during the upgrade switching state 930, a rollback may also be performed as per step 750. In the upgrade switching state 930, if the user confirms the upgrade confirmation instruction, the value of the first field and the old model data may be deleted according to the upgrade confirmation instruction, that is, the old data deletion process 940 is entered, and if the old data deletion is completed, the upgrade process of the feature library is completed, and the incremental upgrade process is correspondingly completed.
Embodiments of the apparatus of the present application are described below, which may be used to perform the methods of the above-described embodiments of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method described above in the present application.
Fig. 10 illustrates an upgrade apparatus 1000 for a feature library according to an embodiment, which includes, as shown in fig. 10:
the receiving module 1010 is configured to receive an upgrade instruction of a feature library, where the feature library includes a feature record of a feature, and the feature record includes a first field and a second field configured for the feature, and the first field is assigned as a pre-update feature value of the feature.
And a stock upgrade file obtaining module 1030, configured to obtain a stock upgrade file of the feature library according to the upgrade instruction, where the stock upgrade file indicates the feature to be updated.
The first feature value obtaining module 1050 is configured to obtain a first feature value of a feature to be updated from the inventory upgrade file.
The updating module 1070 is configured to assign the second field to the first feature value in the feature record of the feature to be updated, so as to obtain an updated feature record of the feature to be updated.
And a writing module 1090, configured to write the updated feature record into the feature library to replace the feature record of the feature to be updated in the feature library.
In one embodiment, the apparatus further comprises:
the incremental upgrade file acquisition module is used for acquiring an incremental upgrade file of the feature library, wherein the incremental upgrade file indicates the features to be written;
the new characteristic value acquisition module is used for acquiring a new characteristic value corresponding to the characteristic to be written according to the increment upgrading file;
the writing characteristic record generating module is used for generating a writing characteristic record for the characteristic to be written according to the new characteristic value, in the writing characteristic record, a first field of the characteristic to be written is assigned as an old characteristic value of the characteristic to be written, and a second field is assigned as a new characteristic value;
and the second writing module is used for writing the written characteristic record into the characteristic library.
In one embodiment, the feature library corresponds to the algorithm model, and the feature library is updated after the algorithm model is updated from an old algorithm model to a new algorithm model; the to-be-written features comprise newly-added features, and the new feature value acquisition module comprises:
a third characteristic value obtaining unit, configured to obtain a third characteristic value of the newly added feature in the new algorithm model according to the increment upgrade file, and use the third characteristic value as a new characteristic value;
in this embodiment, the write profile generation module includes:
the second characteristic value acquisition unit is used for acquiring a second characteristic value of the newly added characteristic under the old algorithm model, and taking the second characteristic value as an old characteristic value; and
and the initial characteristic record acquisition unit is used for acquiring the initial characteristic record configured for the newly added characteristic.
And the first write-in characteristic record obtaining unit is used for assigning the first field as an old characteristic value and assigning the second field as a new characteristic value in the initial characteristic record to obtain the write-in characteristic record of the newly added characteristic.
In another embodiment, the to-be-written feature includes an update feature, and the write feature record generation module includes:
and the characteristic record acquisition unit is used for acquiring the characteristic record of the updated characteristic from the characteristic library and taking the value of the first field in the characteristic record of the updated characteristic as the old characteristic value.
And the second writing characteristic record obtaining unit is used for assigning the second field as a new characteristic value in the characteristic record of the updating characteristic to obtain the writing characteristic record of the updating characteristic.
In one embodiment, the incremental upgrade file indicates a feature to be deleted, the apparatus further comprising:
and the deleting module is used for deleting the characteristic record of the characteristics to be deleted in the characteristic library.
In one embodiment, the apparatus further comprises:
and the traversing module is used for traversing the feature records in the feature library according to the stock upgrading file.
And the loading module is used for loading the new model data of the algorithm model corresponding to the feature library into the cache if traversing is carried out to determine that the features in the feature library complete stock upgrading.
In one embodiment, the cache includes old model data of the algorithm model, and the values of the first field and the second field are saved in the cache when the feature record is written into the feature library, and the apparatus further includes:
and the retrieval request receiving module is used for receiving a retrieval request, and the retrieval request comprises an indication identifier which is used for indicating retrieval in the first field or retrieval in the second field.
The first retrieval result obtaining module is used for retrieving in the first field according to the old model data if the indication identifier indicates that the first field is retrieved, and obtaining a first retrieval result;
and the second retrieval result obtaining module is used for retrieving in the second field according to the new model data if the indication identifier indicates that the second field is retrieved, so as to obtain a second retrieval result.
In one embodiment, the apparatus further comprises:
and the test parameter calculation module is used for calculating test parameters according to the second retrieval result, and the test parameters are used for representing the performance of the new algorithm model indicated by the new model data.
And the upgrading confirmation instruction generation module is used for generating an upgrading confirmation instruction if the test parameters meet the preset conditions, and deleting the value of the first field and the old model data according to the upgrading confirmation instruction.
And the rollback instruction generating module is used for generating a rollback instruction if the test parameter does not meet the preset condition, and deleting the value of the second field and the new model data according to the rollback instruction.
In one embodiment, the algorithmic model includes at least two feature libraries, and the apparatus further includes:
and the second upgrading instruction receiving module is used for receiving the upgrading instruction sent for the next feature library.
The implementation process of the functions and actions of each module/unit in the device is specifically described in the implementation process of the corresponding step in the method for upgrading the feature library, and is not described herein again.
It is understood that these modules may be implemented in hardware, software, or a combination of both. When implemented in hardware, these modules may be implemented as one or more hardware modules, such as one or more application specific integrated circuits. When implemented in software, the modules may be implemented as one or more computer programs executing on one or more processors.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
FIG. 11 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.
It should be noted that the computer system 1100 of the electronic device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 11, a computer system 1100 includes a Central Processing Unit (CPU)1101, which can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. In the RAM 1103, various programs and data necessary for system operation are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other by a bus 1104. An Input/Output (I/O) interface 1105 is also connected to bus 1104.
The following components are connected to the I/O interface 1105: an input portion 1106 including a keyboard, mouse, and the like; an output section 1107 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 1108 including a hard disk and the like; and a communication section 1109 including a network interface card such as a LAN (Local area network) card, a modem, or the like. The communication section 1109 performs communication processing via a network such as the internet. A driver 1110 is also connected to the I/O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is mounted into the storage section 1108 as necessary.
In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 1109 and/or installed from the removable medium 1111. When the computer program is executed by a Central Processing Unit (CPU)1101, various functions defined in the system of the present application are executed.
It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method described in the above embodiments.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A method for upgrading a feature library is characterized by comprising the following steps:
receiving an upgrade instruction of a feature library, wherein the feature library comprises a feature record of a feature, the feature record comprises a first field and a second field configured for the feature, and the first field is assigned as a pre-update feature value of the feature;
obtaining a stock upgrading file of the feature library according to the upgrading instruction;
acquiring a first characteristic value of the characteristic from the stock upgrading file;
in the feature record of the feature, assigning the second field as the first feature value to obtain an updated feature record of the feature;
and writing the updated feature record into the feature library to replace the feature record of the feature in the feature library.
2. The method of claim 1, wherein after receiving the upgrade instruction for the feature library, the method further comprises:
obtaining an increment upgrading file of the feature library, wherein the increment upgrading file indicates features to be written;
acquiring a new characteristic value corresponding to the to-be-written characteristic according to the increment upgrading file;
generating a writing characteristic record for the characteristic to be written according to the new characteristic value, wherein in the writing characteristic record, a first field of the characteristic to be written is assigned as an old characteristic value of the characteristic to be written, and a second field is assigned as the new characteristic value;
writing the written feature record into the feature library.
3. The method of claim 2, wherein the feature library corresponds to an algorithm model, and wherein the upgrading of the feature library is performed after the algorithm model is upgraded from an old algorithm model to a new algorithm model;
the step of obtaining a new feature value corresponding to the to-be-written feature according to the increment upgrade file includes:
acquiring a third characteristic value of the newly added feature under the new algorithm model according to the increment upgrading file, and taking the third characteristic value as a new characteristic value;
generating a writing feature record for the feature to be written according to the new feature value, including:
acquiring a second characteristic value of the newly added characteristic under the old algorithm model, and taking the second characteristic value as an old characteristic value; and
acquiring an initial feature record configured for the newly added feature;
and in the initial characteristic record, assigning the first field as the old characteristic value, assigning the second field as the new characteristic value, and obtaining the write-in characteristic record of the newly added characteristic.
4. The method of claim 2, wherein the feature to be written comprises an updated feature, and wherein generating a written feature record for the feature to be written according to the new feature value comprises:
acquiring the feature record of the updated feature from the feature library, and taking the value of a first field in the feature record of the updated feature as the old feature value;
and in the feature record of the updated feature, assigning the second field as the new feature value to obtain the write-in feature record of the updated feature.
5. The method of claim 2, wherein the incremental upgrade file indicates a feature to be deleted, and after obtaining the incremental upgrade file of the feature library, the method further comprises:
and deleting the characteristic record of the characteristic to be deleted in the characteristic library.
6. The method of any of claims 1-5, wherein after the writing the updated feature record to the feature library, the method further comprises:
traversing the feature records in the feature library according to the stock upgrading file;
and if the characteristics in the characteristic library are determined to finish the stock upgrading through traversal, loading the new model data of the algorithm model corresponding to the characteristic library into a cache.
7. The method according to claim 6, wherein the cache includes old model data of the algorithm model, when the feature record is written into the feature library, the value of the first field and the value of the second field are saved in the cache, and after the new model data of the algorithm model corresponding to the feature library is loaded into the cache, the method further comprises:
receiving a retrieval request, wherein the retrieval request comprises an indication identifier, and the indication identifier is used for indicating retrieval in the first field or retrieval in the second field;
if the indication mark indicates to search in the first field, searching in the first field according to the old model data to obtain a first search result;
and if the indication identifier indicates to search in the second field, searching in the second field according to the new model data to obtain a second search result.
8. The method of claim 7, wherein after retrieving in the second field according to the new model data to obtain a second retrieval result, the method further comprises:
calculating a test parameter according to the second retrieval result, wherein the test parameter is used for representing the performance of a new algorithm model indicated by the new model data;
if the test parameters meet preset conditions, an upgrade confirmation instruction is generated, and the value of the first field and the old model data are deleted according to the upgrade confirmation instruction;
and if the test parameters do not meet the preset conditions, generating a rollback instruction, and deleting the value of the second field and the new model data according to the rollback instruction.
9. The method of claim 8, wherein the algorithmic model includes at least two feature libraries, and wherein after deleting the value of the first field and deleting the old model data according to the upgrade validation instruction, the method further comprises:
and receiving an upgrading instruction sent for the next feature library.
10. An apparatus for upgrading a feature library, comprising:
the system comprises a receiving module, a judging module and a judging module, wherein the receiving module is used for receiving an upgrading instruction of a feature library, the feature library comprises a feature record of a feature, the feature record comprises a first field and a second field configured for the feature, and the first field is assigned as a feature value before the feature is updated;
the stock upgrading file acquisition module is used for acquiring the stock upgrading file of the feature library according to the upgrading instruction;
the first characteristic value acquisition module is used for acquiring a first characteristic value of the characteristic from the stock upgrading file;
the updating module is used for assigning the second field to be the first characteristic value in the characteristic record of the characteristic to obtain an updated characteristic record of the characteristic;
and the writing module is used for writing the updated feature record into the feature library so as to replace the feature record of the feature in the feature library.
CN201911262764.7A 2019-12-10 2019-12-10 Feature library upgrading method and device Active CN111176705B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911262764.7A CN111176705B (en) 2019-12-10 2019-12-10 Feature library upgrading method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911262764.7A CN111176705B (en) 2019-12-10 2019-12-10 Feature library upgrading method and device

Publications (2)

Publication Number Publication Date
CN111176705A true CN111176705A (en) 2020-05-19
CN111176705B CN111176705B (en) 2024-01-30

Family

ID=70650325

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911262764.7A Active CN111176705B (en) 2019-12-10 2019-12-10 Feature library upgrading method and device

Country Status (1)

Country Link
CN (1) CN111176705B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111522570A (en) * 2020-06-19 2020-08-11 杭州海康威视数字技术股份有限公司 Target library updating method and device, electronic equipment and machine-readable storage medium
CN112948246A (en) * 2021-02-26 2021-06-11 北京百度网讯科技有限公司 AB test control method, device, equipment and storage medium of data platform

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0307421D0 (en) * 2003-03-31 2003-05-07 Matsushita Electric Ind Co Ltd Method and apparatus for upgrading software
CN102156646A (en) * 2010-02-11 2011-08-17 华为技术有限公司 Feature library upgrading method and device thereof
CN106815049A (en) * 2016-12-29 2017-06-09 杭州迪普科技股份有限公司 The method and device of feature database upgrading

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0307421D0 (en) * 2003-03-31 2003-05-07 Matsushita Electric Ind Co Ltd Method and apparatus for upgrading software
CN102156646A (en) * 2010-02-11 2011-08-17 华为技术有限公司 Feature library upgrading method and device thereof
CN106815049A (en) * 2016-12-29 2017-06-09 杭州迪普科技股份有限公司 The method and device of feature database upgrading

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111522570A (en) * 2020-06-19 2020-08-11 杭州海康威视数字技术股份有限公司 Target library updating method and device, electronic equipment and machine-readable storage medium
CN111522570B (en) * 2020-06-19 2023-09-05 杭州海康威视数字技术股份有限公司 Target library updating method and device, electronic equipment and machine-readable storage medium
CN112948246A (en) * 2021-02-26 2021-06-11 北京百度网讯科技有限公司 AB test control method, device, equipment and storage medium of data platform
CN112948246B (en) * 2021-02-26 2023-08-04 北京百度网讯科技有限公司 AB test control method, device and equipment of data platform and storage medium

Also Published As

Publication number Publication date
CN111176705B (en) 2024-01-30

Similar Documents

Publication Publication Date Title
CN109766362B (en) Data processing method and device
CN104391725A (en) Page display method and page display device
CN110781658B (en) Resume analysis method, resume analysis device, electronic equipment and storage medium
CN110096302A (en) Software data storage, upgrade method, device, equipment and storage medium
CN111090663A (en) Transaction concurrency control method, device, terminal equipment and medium
CN107368513B (en) Method and device for updating client database
CN111176705B (en) Feature library upgrading method and device
CN102779088A (en) Transaction processing device, transaction processing method, and transaction processing program
CN115827162A (en) Method, device, medium and equipment for intelligently processing mirror image warehouse data
US9588884B2 (en) Systems and methods for in-place reorganization of device storage
CN113377770A (en) Data processing method and device
CN107391539B (en) Transaction processing method, server and storage medium
US20190050266A1 (en) Software application runtime having dynamic evaluation functions and parameters
CN109614383B (en) Data copying method and device, electronic equipment and storage medium
US10862761B2 (en) System and method for management of distributed systems
WO2017036197A1 (en) Method and device for managing network element model
US20110113001A1 (en) Information Integrity Rules Framework
CN115543967A (en) Data migration method and device, electronic equipment and computer readable medium
US7389515B1 (en) Application deflation system and method
US20130166603A1 (en) Consumer-related specialization of metadata models based on a template-projection approach
CN114995790A (en) Component development method and device, computer readable medium and electronic equipment
CN113722007A (en) Configuration method, device and system of VPN branch equipment
CN108733704B (en) Multi-database data processing method and device, storage medium and electronic equipment
CN113010115A (en) Data processing method in block chain node and related equipment
KR101737575B1 (en) Method and device for verifying data based on sql sentences generated automatically

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant