CN107291779B - Cache data management method and device - Google Patents

Cache data management method and device Download PDF

Info

Publication number
CN107291779B
CN107291779B CN201610224699.9A CN201610224699A CN107291779B CN 107291779 B CN107291779 B CN 107291779B CN 201610224699 A CN201610224699 A CN 201610224699A CN 107291779 B CN107291779 B CN 107291779B
Authority
CN
China
Prior art keywords
change event
cache data
change
cache
event type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610224699.9A
Other languages
Chinese (zh)
Other versions
CN107291779A (en
Inventor
杨蒙蒙
钟宇亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201610224699.9A priority Critical patent/CN107291779B/en
Publication of CN107291779A publication Critical patent/CN107291779A/en
Application granted granted Critical
Publication of CN107291779B publication Critical patent/CN107291779B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a cache data management method and a device, wherein the method comprises the following steps: acquiring a database change event message, wherein the change event message comprises a change event type; determining cache data matched with the change event message based on the change event message and configuration information of the cache data; determining a management strategy aiming at the obtained cache data according to the change event type; and processing the cache data according to the determined management strategy. The invention updates the corresponding cache data by acquiring the database change event message, so that the change of the database and the corresponding cache management are asynchronously carried out, the complexity of synchronously writing in the database can be reduced, and the effective utilization rate of the cache is improved.

Description

Cache data management method and device
Technical Field
The embodiment of the invention relates to the technical field of computer data processing, in particular to a cache data management method and device.
Background
With the rapid development of science and technology and internet technology, a novel business operation mode, namely electronic commerce, appears, and under the network environment of the open internet, based on a browser/server application mode, buyers and sellers can carry out various commercial and trade activities without conspiracy, so that the online shopping of consumers, the online transaction and the online electronic payment among merchants, various commercial activities, transaction activities, financial activities and related comprehensive service activities are realized.
At present, common electronic commerce platforms include hundred degree takeouts, hundred degree glutinous rice and the like, with the increase of electronic commerce users, attribute information of article data managed by the electronic commerce platform is increased sharply, and the electronic commerce platform is more complex when managing the data. In order to enable a user to acquire item data more quickly, part of the item data is usually placed in a cache, so that the user acquires the item data directly from the cache during acquisition.
Therefore, in the prior art, in order to enable a user to read the latest state of item data in real time, when the item data is written into the database, the corresponding item data in the cache is updated at the same time. That is, when the current item data is changed, the corresponding item data in the cache is updated while the database is updated. With the increasing growth of electronic commerce, the number of articles needing to be put on shelves is also rapidly increased, the operation of synchronously writing into a database is complex, and the cache is frequently operated, so that the effective utilization rate of the cache is low.
Disclosure of Invention
The invention provides a cache data management method and a cache data management device, which are used for improving the utilization rate of a cache.
In a first aspect, an embodiment of the present invention provides a cache data management method, including:
acquiring a database change event message, wherein the change event message comprises a change event type;
determining cache data matched with the change event message based on the change event message and configuration information of the cache data;
determining a management strategy aiming at the obtained cache data according to the change event type;
and processing the cache data according to the determined management strategy.
In a second aspect, an embodiment of the present invention further provides a cache data management apparatus, including:
the system comprises a change event acquisition module, a database change event processing module and a database change event processing module, wherein the change event acquisition module is used for acquiring a database change event message which contains a change event type;
the cache data determining module is used for determining cache data matched with the change event message based on the change event message and the configuration information of the cache data;
a management policy determining module, configured to determine a management policy for the obtained cache data according to the change event type;
and the cache data processing module is used for processing the cache data according to the determined management strategy.
According to the method and the device, the database change event message containing the change field list and the change event type is obtained, the cache data matched with the change event message is determined based on the change event message and the configuration information of the cache data, and the cache data is processed according to the determined management strategy, so that the technical problem that the cache utilization rate is low due to frequent cache data operation in the prior art when the database change and the cache data are synchronously managed is solved, the database change and the corresponding cache data management are asynchronously conducted, the complexity of synchronous writing of the database can be reduced, and the effective utilization rate of the cache is improved.
Drawings
Fig. 1 is a flowchart of a cache data management method according to a first embodiment of the present invention;
fig. 2 is a flowchart of a cache data management method according to a second embodiment of the present invention;
fig. 3 is a structural diagram of a cache data management apparatus according to a third embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a cache data management method according to an embodiment of the present invention, where this embodiment is applicable to a situation where cache data corresponding to a database change is asynchronously updated in each scenario, and the method may be executed by a cache data management device according to an embodiment of the present invention, where the device may be integrated in a mobile terminal, a fixed terminal, or a server, as shown in fig. 1, and specifically includes:
s101, obtaining a database change event message, wherein the change event message comprises a change event type.
When an item is stored in the database, the attribute corresponding to the item is stored as an attribute field (including the name and/or number of the item, the corresponding price, weight, quantity, etc.). For example, when storing apples, the corresponding attribute fields in the database may include "apple", "price", "quantity", and the like. When the database is changed, a change event message is generated, and the change event message contains the change type of the change event.
Specifically, the operation log of the database can be acquired in real time, the change information of the database is obtained by analyzing the operation log of the database, and the change event message is generated according to the change information. When detecting that the database is changed (such as adding new data, changing data information and deleting redundant data), respectively generating corresponding change event messages and adding corresponding change event types into the change event messages.
S102, determining cache data matched with the change event message based on the change event message and the configuration information of the cache data.
When the data in the database is put into the cache, the cache data is configured. Specifically, when the matched cache data is determined, the configuration information is compared with the change field list, and the data corresponding to the field with the intersection in the list is determined as the cache data matched with the change event message.
S103, determining a management strategy aiming at the obtained cache data according to the change event type.
Different management strategies are preset according to different change event types. The change event types include, but are not limited to, an add event type, a change event type, and a delete event type.
Specifically, when the change event type is an increase event type, setting a corresponding management policy as an increase policy; when the change event type is an update event type, setting a corresponding management strategy as an update strategy; and when the change event type is the deletion event type, setting the corresponding management strategy as a deletion strategy.
And S104, processing the cache data according to the determined management strategy.
The processing of the cached data for different management policies is also different. For example, when the management policy is a new policy, the attribute data of the change field list is written into a cache; when the management strategy is an updating strategy, modifying the corresponding attribute data into the attribute data of the updated field; and when the change event type is a deletion event type, deleting the cache data.
In this embodiment, by obtaining a database change event message including a change field list and a change event type, determining cache data matched with the change event message based on the change event message and configuration information of the cache data, and processing the cache data according to a determined management policy, a technical problem that cache utilization is low due to frequent operation of the cache data when database change and cache data management are performed synchronously in the prior art is solved, so that database change and corresponding cache data management are performed asynchronously, complexity of synchronous writing in the database can be reduced, and effective utilization of the cache can be improved.
On the basis of the above embodiment, the change event message further includes a change field list, the configuration information of the cache data includes a care field list, and determining the cache data matched with the change event message based on the change event message and the configuration information of the cache data includes:
and determining cache data of intersection of the concerned field list and the changed field list as cache data matched with the change event message.
The changed field list is composed of fields composed of attributes corresponding to changed data. For example, if the original data stored in the database is the price, weight and quantity of apple, the price, weight and quantity of grape, etc., when the data that the user needs to change is the price of apple, a change field list containing the attribute field "apple-price" is generated after changing the price of apple in the database, the change field list being in the form of a change event message, and when the data that the user needs to change is the price and weight of apple, a change field list containing the attribute field "apple-price-weight" is generated. And when the user writes new data into the database, generating a change field list according to the attribute field corresponding to the data written by the user. For example, a user writes the price and quantity of dragon fruit into a database, and a list of change fields is generated that contains the attribute field "dragon fruit-price-quantity". Similarly, when the user deletes the data in the database, the change field list is generated according to the attribute field corresponding to the data deleted by the user, for example, when the user deletes the number of apples from the database, the change field list containing the attribute field "apple-price-number" is generated.
That is, the change field list included in the change event message may be defined according to the attribute field corresponding to the changed data in the database, or may be the same as the attribute field corresponding to the changed attribute data, and may include information such as the name, identification, and attribute (e.g., price, weight, quantity, etc.) of the item. The change event message may also include changed data information, such as changed price, changed weight, and changed number.
Wherein, the configuration information includes a field of interest list, and the field of interest list includes, but is not limited to, the attribute field described in step S101. And when the matched cache data is determined, comparing the concerned field list with the change field list, and determining the data corresponding to the fields with intersection in the list as the cache data matched with the change event message.
On the basis of the above embodiment, the changing event type includes an adding event type, a changing event type, or a deleting event type, the change field list includes a pre-change field and a post-change field, and determining a management policy for the acquired cache data according to the changing event type includes:
when cache data matched with the change event message does not exist and the change event type in the change event message is an added event type, determining the management strategy as a newly added strategy, wherein the newly added strategy is to write the attribute data of the change field list into a cache;
when cache data matched with the change event message exists and the change event type is an update event type, determining the management strategy as an update strategy, wherein the update strategy is to modify the attribute data of the concerned field list into the attribute data of the updated field in the change field list; or when cache data matched with the change event message exists and the change event type is a deletion event type, determining the management strategy as a deletion strategy, wherein the deletion strategy is to delete the cache data.
On the basis of the foregoing embodiment, the configuration information further includes a change event type of interest, and correspondingly, determining cache data matching the change event message based on the change event message and the configuration information of the cache data includes:
and determining cache data which has intersection between the concerned field list and the changed field list and matches the concerned changed event type with the changed event type as cache data which matches the changed event message.
Wherein the interest field list includes but is not limited to the following attribute fields of the item: name and/or number of the item, corresponding price, weight and quantity.
Specifically, when obtaining the matched cache data, firstly, comparing the concerned field list with the changed field list, obtaining the fields with intersection, matching the concerned changed event type with the changed event type, obtaining the corresponding cache data when both the concerned changed event type and the changed event type are satisfied, and processing the matched cache data according to the changed event type. For example, first, a management policy for the obtained cache data is determined according to the change event type, and then the cache data is processed according to the determined management policy, and the detailed processing procedure is described in step S103 and step S104 in the first embodiment.
On the basis of the above embodiment, in order to further improve the cache utilization efficiency and reduce the data amount of the processed cache data, the present invention further includes the following steps:
filtering the acquired cache data according to the setting, and
processing the cached data according to the determined management policy comprises:
and processing the filtered cache data according to the determined management strategy.
Wherein the set filtering rules include, but are not limited to, traffic filtering rules or heat filtering rules.
Specifically, if the set rule is a time filtering rule, the latest update time is obtained from the configuration information in the cache data, if the latest update time is earlier than or equal to the occurrence time of the change event message, the cache data is processed according to the determined management policy, and if the latest update time is later than the occurrence time of the change event message, the cache data is rejected from being processed.
If the set rule is a service filtering rule, acquiring the current service state of the event from the configuration information in the cache data, if the current service state is an invalid service state, refusing to process the cache data, and if the current service type is the valid service state, processing the cache data according to the determined management strategy. For example, in the technical field of electronic commerce, the invalid business state may be an article that has been put on shelf, an article that has been put off production, or an article that has not been put on shelf or is on line, and for all the cache data corresponding to these articles, the cache data is in an invalid state, and no processing is needed, so that the utilization rate of the cache can be further improved.
If the set rule is a popularity filtering rule, counting popularity of the cached data, wherein the popularity is determined by at least one of click rate, purchase quantity, browsing quantity and score of the user on the event associated with the cached data; if the heat degree exceeds the preset heat degree, processing the cache data according to the determined management strategy; and if the heat is lower than or equal to the preset heat, refusing to process the cached data.
On the basis of the foregoing embodiment, if the configuration information includes the mask period information, determining the management policy for the obtained cache data according to the change event type further includes:
and determining the acquired management strategy according to the change event type and the shielding period information.
Specifically, obtaining shielded time period information from the configuration information of the cache data, and determining the management policy as a deferral policy when the occurrence time of a change event is in the shielded time period, where the deferral policy is to defer the pushing of the change event message in the asynchronous queue; and when the occurrence time of the change event is not in the shielding time period, determining the management strategy according to the change event type, and processing the cache data according to the determined management strategy. .
On the basis of the foregoing embodiment, if the cache data is stored in the form of a cache topic, determining the cache data that matches the change event message includes:
and determining a cache theme associated with the change field list, and acquiring corresponding cache data in the cache theme.
On the basis of the above embodiment, when the cache data is processed according to the determined management policy, the cache data corresponding to the unique change data identifier under the cache theme is processed according to the determined management policy and the unique change data identifier included in the change event message.
The cache subject comprises one or more attributes of the article, and the corresponding cache data comprises one or more attribute data or a group of data of a fixed structure calculated according to the one or more attribute data. For example, for the cache for storing fruits, a cache theme corresponding to the cache for storing prices separately may be established, a cache theme corresponding to the cache for storing prices and weights simultaneously may also be established, or a cache theme corresponding to the cache for the total amount that needs to be calculated according to the product of prices and weights may be established.
Specifically, the modified field list includes modified attribute fields, and for the modified attribute fields in the database, a corresponding relationship list between the attribute fields and corresponding cache topics needs to be established in advance, and the corresponding relationship list can be placed in a cache or a database separately. After a database change event message is acquired, searching a corresponding relation list according to an attribute field contained in a change field list, if a cache theme corresponding to the attribute field is found, determining that the cache theme associated with the change field list exists, and processing cache data corresponding to the unique identification of the change data under the cache theme according to the determined management strategy and the unique identification of the change data contained in the change event message.
Specifically, a cache theme may store attribute data corresponding to a large number of different articles, for example, for a "price" cache theme, price cache data corresponding to fruits such as apples, pears, grapes, and the like may be included, when an attribute field included in a change field list in an acquired database change event message is "apple-price", a corresponding relationship list between the attribute field and the cache theme is found by searching for a pre-established attribute field and cache theme, and if a cache theme (i.e., a "price" cache theme and a "total amount" cache theme) associated with the "price" field is found, cache data corresponding to "apples" under the "price" cache theme and the "total amount" cache theme is further changed according to the unique identifier of the article, i.e., the apple.
If the corresponding cache theme is not found in the corresponding relation list between the attribute field and the cache theme, establishing the cache theme associated with the attribute field, writing the corresponding attribute data into the cache corresponding to the cache theme, and updating the corresponding relation list between the attribute field and the cache theme.
For example, when the attribute field included in the change field list in the obtained database change event message is "singleton weight, that is, average weight", by searching that the attribute field does not exist in the correspondence list, a cache theme corresponding to the attribute field, that is, the singleton weight, is established, the singleton weight corresponding to the mango is placed in the corresponding cache, and the correspondence between the attribute field corresponding to the mango-singleton weight and the cache theme is added to the correspondence list. Or when the attribute field contained in the change field list in the obtained database change event message is the 'pear-total amount', searching a cache theme corresponding to the attribute field 'total amount' by searching the corresponding relation list, further inquiring whether total amount cache data corresponding to the article unique identifier 'pear' exists under the cache theme, if not, further checking the change event type contained in the database change event message, if the change type is determined to be the increase event type, determining that the corresponding management strategy is the new increase strategy, expanding the cache under the 'total amount' cache theme, and storing the total amount corresponding to the pear in the corresponding cache.
And aiming at the change event types contained in different change event messages of the database, the cache data in the cache subject are processed differently, and the update event type and the deletion event type are included besides the increase event type.
Also taking the database for storing the fruit prices as an example, when the obtained change event message is the price of an updated apple, determining that the management strategy is the updating strategy, and updating the cache data corresponding to the apple under the price cache theme into new price data according to the unique item identifier.
The following two situations exist for updating the cached data under the theme of the price-weight cache: when the cache data needing to be updated is only the price, the data corresponding to the weight needs to be acquired at the same time, the cache data corresponding to the theme of price-weight cache is updated together, and when the cache data needing to be updated is the price and the weight, the cache data corresponding to the theme of price-weight cache is updated by using the updated price data and weight data.
For the update of the cache data under the "total amount" cache theme (total amount is price amount), there are three cases as follows: when the changed database field is only the price, the updated price and the cache data corresponding to the quantity are required to be obtained at the same time, the total amount is obtained through calculation, and the cache data corresponding to the total amount under the total amount cache theme is updated by using the total amount obtained through calculation; when the changed database field is only quantity, the cache data of the price and the updated quantity data need to be acquired at the same time, the total amount is obtained through calculation, and the corresponding total amount cache data under the total amount cache theme is updated by using the total amount obtained through calculation; and when the cache data needing to be updated is the price and the weight, calculating to obtain the total amount by using the updated price data and weight data, and updating the corresponding cache data under the cache theme of the total amount by using the calculated total amount.
When the obtained change event message is the price of the deleted apple, determining the cache theme associated with the price, determining the management strategy to be the deletion strategy, deleting the cache corresponding to the apple under the cache theme, and updating the corresponding relation list of the attribute field and the cache theme, namely deleting the price corresponding to the apple from the cache of the cache theme.
In order to use the cache more effectively, when the cache is operated, the operation time should be considered, that is, according to the set filtering rule, the cache data under the obtained cache theme is filtered, and then according to the determined management policy, the filtered cache data under the cache theme is processed. The set filtering rules comprise time filtering rules, business filtering rules or heat filtering rules. The specific processing procedure is similar to that described in the first embodiment, and is not described herein again.
Example two
Fig. 2 is a flowchart of a cache data management method according to a second embodiment of the present invention, which is applicable to an article shelving process in the technical field of electronic commerce (for example, shelving an article of hundredth sticky rice), and specifically includes:
s201, writing the attribute field of the article to be shelved into the database, or deleting the article which is shelved from the database or updating the attribute data of the article which is shelved in the database, and generating a database operation log.
The attribute field of the article to be shelved specifically may include at least one of information attribute information such as article name, weight, price, color, and the like.
S202, acquiring an operation log of a database in real time or periodically, and analyzing the operation log to generate a change event message, wherein the change event message comprises a change field list and a change event type, and the change field list comprises a changed attribute field.
S203, searching a pre-established corresponding relation list of the attribute fields and the cache subjects, and determining whether the cache subjects associated with the changed field list exist.
If there is a cached subject associated with the change field list, then perform step 204; if there is no cached subject associated with the attribute field, step 206 is performed.
S204, determining the type of the change event in the change event message, and determining a management strategy aiming at the cache data under the acquired cache theme according to the type of the change event.
And S205, processing the cache data corresponding to the unique changed data identifier under the cache theme according to the determined management strategy.
S206, determining the change event type in the change event message as an increase event type, establishing a cache theme associated with the attribute field, writing the corresponding attribute data into a cache corresponding to the cache theme, and updating the corresponding relation list of the attribute field and the cache theme.
In the prior art (see description of background art), the shelving process of an article includes a writing operation of a database and an updating management operation of a cache, and by adopting the technical scheme of this embodiment, a change of the database and a change of cache data are performed asynchronously by acquiring a database change event message in real time or periodically, so that the shelving operation and the updating management operation of the cache can be decoupled effectively, time consumed for shelving the article is saved effectively, and shelving efficiency is improved effectively.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a cache data management device according to a third embodiment of the present invention, where the device may be implemented in a software or hardware manner, and the device may be integrated in a mobile terminal, a fixed terminal, or a server, as shown in fig. 3, the specific structure of the device is as follows: a change event acquisition module 31, a cache data determination module 32, a management policy determination module 33, and a cache data processing module 34.
The change event acquiring module 31 is configured to acquire a database change event message, where the change event message includes a change event type.
The cache data determination module 32 is configured to determine cache data matching the change event message based on the change event message and configuration information of the cache data.
The management policy determining module 33 is configured to determine a management policy for the obtained cache data according to the change event type.
The cache data processing module 34 is configured to process the cache data according to the determined management policy.
The cache data management apparatus of this embodiment is configured to execute the cache data management method of the foregoing embodiments, and the technical principle and the generated technical effect are similar, which are not described herein again.
On the basis of the foregoing embodiment, the change event message further includes a change field list, the configuration information of the cache data includes a field-of-interest list, and the cache data determining module 32 is specifically configured to determine cache data in which the field-of-interest list and the change field list intersect with each other, as cache data matched with the change event message.
On the basis of the foregoing embodiment, the configuration information includes a change event type of interest, and the cache data determination module 32 is specifically configured to determine cache data, in which the change event type of interest is matched with the change event message, and an intersection exists between the field list of interest and the change field list, and the change event type of interest is matched with the change event type of interest.
On the basis of the above embodiment, the apparatus further includes: a cache data filtering module 35.
The cache data filtering module 35 is configured to filter the acquired cache data according to a set filtering rule.
The cache data processing module 34 is specifically configured to process the filtered cache data according to the determined management policy.
On the basis of the foregoing embodiment, the change event type includes an add event type, a change event type, or a delete event type, the change field list includes a pre-change field and a post-change field, and the management policy determining module 33 is specifically configured to determine, when there is no cache data matching the change event message and the change event type in the change event message is the add event type, the management policy as an additional policy, where the additional policy is to write attribute data of the change field list into a cache; when cache data matched with the change event message exists and the change event type is an update event type, determining the management strategy as an update strategy, wherein the update strategy is to modify the attribute data of the concerned field list into the attribute data of the updated field in the change field list; or when cache data matched with the change event message exists and the change event type is a deletion event type, determining the management strategy as a deletion strategy, wherein the deletion strategy is to delete the cache data.
On the basis of the above embodiment, the set filtering rule includes a temporal filtering rule, a business filtering rule or a heat filtering rule.
On the basis of the foregoing embodiment, the configuration information includes masked period information, and the management policy determining module 33 is specifically configured to determine the acquired management policy according to the change event type and the masked period information.
On the basis of the foregoing embodiment, the change event message is in an asynchronous queue, and the management policy determining module 33 is specifically configured to determine the management policy as a deferral policy when the occurrence time of the change event is in the masked period, where the deferral policy is to defer the pushing of the change event message in the asynchronous queue, and determine the management policy according to the type of the change event when the occurrence time of the change event is not in the masked period.
On the basis of the foregoing embodiment, the cache data determining module 32 is specifically configured to determine a cache topic associated with the change field list, and obtain corresponding cache data in the cache topic.
On the basis of the foregoing embodiment, the cache data processing module 34 is specifically configured to process, according to the determined management policy and the unique change data identifier included in the change event message, the cache data corresponding to the unique change data identifier under the cache theme.
On the basis of the above embodiment, the change event acquiring module 31 is specifically configured to analyze an operation log of the database and generate a change event message.
The cache data management apparatus according to the foregoing embodiments is used for executing the cache data management method according to the foregoing embodiments, and the technical principle and the generated technical effect are similar, and are not described herein again.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (18)

1. A cache data management method, comprising:
acquiring a database change event message, wherein the change event message comprises a change event type;
determining cache data matched with the change event message based on the change event message and configuration information of the cache data;
determining a management strategy aiming at the obtained cache data according to the change event type;
processing the cache data according to the determined management strategy;
wherein, the change event message further includes a change field list, the change field list is composed of fields composed of attributes corresponding to changed data, the configuration information of the cache data includes a field of interest list and a change event type of interest, and the determining cache data matched with the change event message based on the change event message and the configuration information of the cache data includes:
and determining cache data which has intersection between the concerned field list and the changed field list and matches the concerned changed event type with the changed event type as cache data which matches the changed event message.
2. The method of claim 1, further comprising:
filtering the acquired cache data according to the set filtering rule, and
processing the cached data according to the determined management policy comprises:
and processing the filtered cache data according to the determined management strategy.
3. The method of claim 1, wherein the change event type comprises an add event type, a change event type, or a delete event type, wherein the list of change fields comprises a pre-change field and a post-change field, and wherein
Determining a management policy for the obtained cached data according to the change event type comprises:
when cache data matched with the change event message does not exist and the change event type in the change event message is an added event type, determining the management strategy as a newly added strategy, wherein the newly added strategy is to write the attribute data of the change field list into a cache;
when cache data matched with the change event message exists and the change event type is an update event type, determining the management strategy as an update strategy, wherein the update strategy is to modify the attribute data of the concerned field list into the attribute data of the updated field in the change field list; or when cache data matched with the change event message exists and the change event type is a deletion event type, determining the management strategy as a deletion strategy, wherein the deletion strategy is to delete the cache data.
4. The method of claim 2, wherein the set filtering rules comprise temporal filtering rules, traffic filtering rules, or thermal filtering rules.
5. The method of claim 1, wherein the configuration information comprises mask period information, and wherein
Determining a management policy for the obtained cached data according to the change event type comprises:
and determining the acquired management strategy according to the change event type and the shielding period information.
6. The method of claim 5, wherein the change event message is in an asynchronous queue, and wherein determining the management policy for the obtained management policy based on the change event type and the masking period information comprises:
determining the management policy as a deferral policy when an occurrence time of a change event is in the masked period, the deferral policy being to defer a push of the change event message in the asynchronous queue, an
And when the occurrence time of the change event is not in the shielding time period, determining the management strategy according to the change event type.
7. The method of any of claims 1 to 6, wherein determining cached data that matches the change event message comprises:
and determining a cache theme associated with the change field list, and acquiring corresponding cache data in the cache theme.
8. The method of claim 7, wherein processing the cached data according to the determined management policy comprises:
and processing the cache data corresponding to the unique changed data identification under the cache theme according to the determined management strategy and the unique changed data identification contained in the changed event message.
9. The method of any of claims 1 to 8, wherein obtaining a database change event message comprises:
and analyzing the operation log of the database to generate a change event message.
10. A cache data management apparatus, comprising:
the system comprises a change event acquisition module, a database change event processing module and a database change event processing module, wherein the change event acquisition module is used for acquiring a database change event message which contains a change event type;
the cache data determining module is used for determining cache data matched with the change event message based on the change event message and the configuration information of the cache data;
a management policy determining module, configured to determine a management policy for the obtained cache data according to the change event type;
the cache data processing module is used for processing the cache data according to the determined management strategy;
wherein, the change event message further includes a change field list, the change field list is composed of fields composed of attributes corresponding to changed data, the configuration information of the cache data includes a concerned field list and a concerned change event type, and the cache data determination module is specifically configured to:
and determining cache data which has intersection between the concerned field list and the changed field list and matches the concerned changed event type with the changed event type as cache data which matches the changed event message.
11. The apparatus of claim 10, further comprising:
the cache data filtering module is used for filtering the acquired cache data according to a set filtering rule;
the cache data processing module is specifically configured to:
and processing the filtered cache data according to the determined management strategy.
12. The apparatus of claim 10, wherein the change event type comprises an add event type, a change event type, or a delete event type, the change field list comprises a pre-change field and a post-change field, and the management policy determination module is specifically configured to:
when cache data matched with the change event message does not exist and the change event type in the change event message is an added event type, determining the management strategy as a newly added strategy, wherein the newly added strategy is to write the attribute data of the change field list into a cache;
when cache data matched with the change event message exists and the change event type is an update event type, determining the management strategy as an update strategy, wherein the update strategy is to modify the attribute data of the concerned field list into the attribute data of the updated field in the change field list; or when cache data matched with the change event message exists and the change event type is a deletion event type, determining the management strategy as a deletion strategy, wherein the deletion strategy is to delete the cache data.
13. The apparatus of claim 11, wherein the set filtering rules comprise temporal filtering rules, traffic filtering rules, or thermal filtering rules.
14. The apparatus of claim 10, wherein the configuration information comprises masking period information, and wherein the management policy determination module is specifically configured to:
and determining the acquired management strategy according to the change event type and the shielding period information.
15. The apparatus of claim 14, wherein the change event message is in an asynchronous queue, and wherein the management policy determination module is specifically configured to:
when the occurrence time of the change event is in the shielding period, determining the management policy as a deferral policy, wherein the deferral policy is to defer the pushing of the change event message in the asynchronous queue, and when the occurrence time of the change event is not in the shielding period, determining the management policy according to the change event type.
16. The apparatus according to any one of claims 10 to 15, wherein the cache data determining module is specifically configured to:
and determining a cache theme associated with the change field list, and acquiring corresponding cache data in the cache theme.
17. The apparatus of claim 16, wherein the cache data processing module is specifically configured to:
and processing the cache data corresponding to the unique changed data identification under the cache theme according to the determined management strategy and the unique changed data identification contained in the changed event message.
18. The apparatus according to any one of claims 10 to 17, wherein the change event acquiring module is specifically configured to:
and analyzing the operation log of the database to generate a change event message.
CN201610224699.9A 2016-04-12 2016-04-12 Cache data management method and device Active CN107291779B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610224699.9A CN107291779B (en) 2016-04-12 2016-04-12 Cache data management method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610224699.9A CN107291779B (en) 2016-04-12 2016-04-12 Cache data management method and device

Publications (2)

Publication Number Publication Date
CN107291779A CN107291779A (en) 2017-10-24
CN107291779B true CN107291779B (en) 2021-08-13

Family

ID=60093824

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610224699.9A Active CN107291779B (en) 2016-04-12 2016-04-12 Cache data management method and device

Country Status (1)

Country Link
CN (1) CN107291779B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108009029B (en) * 2017-11-30 2022-01-04 中电福富信息科技有限公司 Method and system for decoupling and persisting cache data based on Ignite grid
CN108512948B (en) * 2018-03-21 2022-05-17 平安普惠企业管理有限公司 Address book updating method and device, computer equipment and storage medium
CN108874403B (en) * 2018-05-30 2022-07-12 深圳市分期乐网络科技有限公司 Method, device and equipment for issuing rule file and storage medium
CN109828980A (en) * 2019-02-19 2019-05-31 北京字节跳动网络技术有限公司 Data cache method, device, terminal and storage medium
CN111382180B (en) * 2020-03-11 2024-04-26 网易(杭州)网络有限公司 Data clearing method and device for local cache
CN111694863B (en) * 2020-04-30 2023-11-28 武汉灯塔之光科技有限公司 Database cache refreshing method, system and device
CN114443777B (en) * 2022-04-08 2022-08-23 北京新氧科技有限公司 Multi-data aggregation management method, device, equipment and system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8775741B1 (en) * 2009-01-13 2014-07-08 Violin Memory Inc. Using temporal access patterns for determining prefetch suitability
CN102043653A (en) * 2010-12-23 2011-05-04 中国农业银行股份有限公司 Cache system and methods for modifying cache configuration and operating and querying cache data
CN103595776A (en) * 2013-11-05 2014-02-19 福建网龙计算机网络信息技术有限公司 Distributed type caching method and system

Also Published As

Publication number Publication date
CN107291779A (en) 2017-10-24

Similar Documents

Publication Publication Date Title
CN107291779B (en) Cache data management method and device
CN102722481B (en) The processing method of a kind of user's favorites data and searching method
US20200250732A1 (en) Method and apparatus for use in determining tags of interest to user
CN104992348A (en) Method and device for displaying information
CN104615721A (en) Method and system for recommending communities based on returned goods related information
EP3321824A1 (en) Data retrieval system
CN115423555A (en) Commodity recommendation method and device, electronic equipment and storage medium
CN109426998B (en) Information pushing method and device
US11294917B2 (en) Data attribution using frequent pattern analysis
CN110838019A (en) Method and device for determining trial supply distribution crowd
CN115796914A (en) Operation data analysis method, system, computer device and storage medium
CN107688581B (en) Data model processing method and device
CN112182071B (en) Data association relation mining method and device, electronic equipment and storage medium
CN108009245A (en) Value of the product acquisition methods, device, computer equipment and storage medium
CN114065015A (en) Search recommendation method, device and equipment
CN113158032A (en) Information pushing method and device
CN112819490A (en) Device and method for pre-notifying second-killing advertisement
CN111881148A (en) Object combination attribute determination method and device and electronic equipment
CN112131465A (en) Activity information matching method, device, medium and terminal equipment
CN112991033A (en) Method and device for determining value attribute of article
WO2020221022A1 (en) Service object recommendation method
WO2024140140A1 (en) Topic pushing processing method and apparatus, and device and medium
CN111861605B (en) Service object recommendation method
CN115442092A (en) Target account identification method and device, and computer equipment
CN114565398A (en) Electronic commerce data mining system based on internet

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant