CN113419976A - Self-adaptive segmented caching method and system based on classification prediction - Google Patents
Self-adaptive segmented caching method and system based on classification prediction Download PDFInfo
- Publication number
- CN113419976A CN113419976A CN202110723648.1A CN202110723648A CN113419976A CN 113419976 A CN113419976 A CN 113419976A CN 202110723648 A CN202110723648 A CN 202110723648A CN 113419976 A CN113419976 A CN 113419976A
- Authority
- CN
- China
- Prior art keywords
- cache
- segment
- module
- turning
- head
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000008569 process Effects 0.000 claims description 45
- 230000003044 adaptive effect Effects 0.000 claims description 14
- 238000013145 classification model Methods 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 13
- 238000010606 normalization Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 238000010801 machine learning Methods 0.000 abstract description 6
- 238000004422 calculation algorithm Methods 0.000 description 12
- 238000002474 experimental method Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0893—Caches characterised by their organisation or structure
- G06F12/0897—Caches characterised by their organisation or structure with two or more cache hierarchy levels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
- G06F12/0868—Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses a self-adaptive segmented cache method and a self-adaptive segmented cache system based on classification prediction, and particularly provides a self-adaptive segmented cache strategy, wherein a cache space is divided into three sections with different sizes and different priorities, the distribution proportion of each cache section is adjusted based on the cache hit rate in a certain time window, and a lifting rule is designed according to access frequency so as to provide longer cache residence time for an object with high access frequency for next hit. On the basis, the machine learning classifier is introduced to predict whether the object is revisited, and the object is placed in different positions of three sections of caches according to the prediction result.
Description
Technical Field
The invention belongs to the technical field of computer storage, and particularly relates to a self-adaptive segmented caching method and system based on classification prediction.
Background
With the rapid development of the mobile internet industry, object service providers are facing a serious challenge to store, transmit, and process a large number of objects. In practical applications, the most common conventional cache replacement policy not only makes it difficult to obtain a cache hit rate, but also may cause problems such as cache pollution, that is, the cache contains objects that are accessed infrequently.
In recent years, many researchers have attempted to make intelligent cache policies such as cache prefetching, cache admission, and eviction using machine learning techniques. The excellent caching strategy can not only reduce the access time of a user, but also improve the performance of the system in mass data transmission. The existing researchers mainly adopt machine learning algorithms (such as support vector machines, random forests and naive Bayes) to realize intelligent cache strategies such as cache prefetching, admission and elimination.
However, the existing intelligent cache policy based on the machine learning algorithm does not fully utilize the data access characteristics to optimize the configuration management of the cache space, and the system cache resources are not maximally utilized, so that the cache hit rate is low, the consumption of network bandwidth is increased, and the access delay of the user is increased.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides a self-adaptive segmented caching method and system based on classification prediction, and aims to optimize the configuration management of a cache space by fully utilizing the characteristics of data access and improve the cache hit rate and the cache byte hit rate of a cache management strategy, so that the technical problems that the cache hit rate is low, the consumption of network bandwidth is increased and the access delay of a user is prolonged due to the fact that the cache resources of a system are not maximally utilized in the existing intelligent caching strategy based on a machine learning algorithm are solved.
To achieve the above object, according to an aspect of the present invention, there is provided an adaptive segment caching method based on classification prediction, including the steps of:
(1) receiving an access request from a user, judging whether a corresponding object is in a cache or not according to the access request, if so, turning to the step (2), otherwise, turning to the step (7); the cache comprises three sections S1, S2 and S3;
(2) judging whether the object is located in the first segment S1 of the cache, if so, moving the object to the head of the first segment S1, and ending the process, otherwise, turning to the step (3);
(3) judging whether the object is located in the second segment S2 of the cache, if so, turning to the step (4), otherwise, turning to the step (5);
(4) and judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr1, if so, moving the object to the head of the first section S1 of the cache, and ending the process, otherwise, moving the object to the head of the second section S2 of the cache, and ending the process.
(5) Judging whether the object is located in a third section S3 of the cache, if so, turning to the step (6), otherwise, turning to the step (7);
(6) judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr2, if so, moving the object to the head of the second section S2 of the cache, and then ending the process, otherwise, moving the object to the head of the third section S3 of the cache, and ending the process;
(7) reading the object from the background database, determining whether enough space is left in the cache for storing the object, if so, turning to the step (10), otherwise, turning to the step (8);
(8) judging whether the cache size of the first segment S1 or the second segment S2 exceeds the corresponding preset length-to-ratio threshold, if so, moving the object at the tail of the first segment S1 or the second segment S2 to the head of the next segment, and turning to the step (9), otherwise, turning to the step (9);
(9) judging whether the cache size of the third section S3 of the cache exceeds a preset length ratio threshold, if so, deleting the object at the tail part of the third section S3 of the cache from the cache, and then turning to the step (10), otherwise, turning to the step (10);
(10) inputting the object corresponding to the access request obtained in the step (1) into a trained AdaBoost classification model to obtain a predicted value, and judging whether the predicted value is greater than 0, if so, moving the object to the head of the second section S2 of the cache, and ending the process, otherwise, moving the object to the head of the third section S3 of the cache, and ending the process.
Preferably, the preset access frequency threshold thr1 ranges from 5 to 10, preferably 5;
the preset access frequency threshold thr2 is in the range of 1 to 3, preferably 2.
In the steps (8) and (9), the value range of the preset length-to-ratio threshold is 0.1 to 0.9.
Preferably, the size setting of the three sections S1, S2, S3 buffered in step (1) is obtained by the following procedure: firstly, the size ratio sr of the first segment cache S1 in the whole cache is set1Is between 0.1 and 0.9, then the size ratio sr of the second segment cache S2 in the whole cache is set2Between 0.1 and 0.9, subsequently according to sr1And sr2To obtain the size of the third segment buffer S3 in the whole buffer is equal to sr3＝1-sr1-sr2Then observing the cache hit rate in a certain time window along with the change of the size of the three-segment cache, and then adjusting sr1、sr2And repeating the above processes until sr corresponding to the maximum cache hit rate is finally found1、sr2And sr3。
Preferably, the AdaBoost classification model used in step (10) is trained by the following steps:
(10-1) obtaining a training data set S { (x)1,y1),(x2,y2),…,(xn,yn) In which xiIs a feature of the ith object, and yiIs the class label of the ith object, if the ith object is actually accessed again, yi1, otherwise yi-1, wherein i ∈ [1, n ]]N is the total number of objects in the training dataset;
(10-2) training each pair in the data set SWeight D of image i1(i) Initializing to 1/n;
(10-3) setting a counter t equal to 1;
(10-4) judging whether T is larger than a preset iteration threshold value T, if so, turning to the step (10-10), otherwise, turning to the step (10-5);
(10-5) training the weight D of each object i in the data set S1(i) And feature xiInputting AdaBoost classification model to obtain each feature xiCorresponding weak classification result ft(xi) The weak classification results corresponding to all the features form the tth weak classification result set ft(x) Where t represents the t-th weak classifier result.
(10-6) the t-th weak classification result set f obtained according to the step (10-5)t(x) Calculating an error estimate epsilon thereoft：
(10-7) obtaining an error estimation value epsilon according to the step (10-6)tObtaining a weight update parameter alphat：
(10-8) obtaining an error estimation value epsilon according to the step (10-6)tAnd the weight update parameter alpha obtained in the step (10-7)tObtaining the weight D of each object i in the t +1 iteration processt+1(i)：
(10-9) setting t as t +1, and returning to the step (10-4);
and (10-10) generating a final strong classification result F (x) according to the T weak classification result sets, and taking the final strong classification result F (x) as a trained AdaBoost classification model.
Preferably, in step (10-6), the error estimation value ε is calculated by using the following formulat：
Preferably, in step (10-7), the weight update parameter α is calculated by using the following formulat：
Preferably, the weight D of each object in the t +1 th iteration in step (10-8)t+1(i) Equal to:
wherein Z istIs a normalization factor and has
etIs the base of the natural logarithm.
Preferably, the final strong classification result f (x) in step (10-10) is equal to:
according to another aspect of the present invention, there is provided an adaptive segment caching system based on classification prediction, including:
the first module is used for receiving an access request from a user, judging whether a corresponding object is in a cache or not according to the access request, if so, switching to the second module, and otherwise, switching to the seventh module; the cache comprises three sections S1, S2 and S3;
the second module is used for judging whether the object is positioned in the first segment S1 of the cache, if so, the object is moved to the head of the first segment S1, the process is finished, otherwise, the process is shifted to the third module;
the third module is used for judging whether the object is positioned in the second segment S2 of the cache, if so, the fourth module is switched, otherwise, the fifth module is switched;
and the fourth module is used for judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr1, if so, moving the object to the head of the first section S1 of the cache, and ending the process, otherwise, moving the object to the head of the second section S2 of the cache, and ending the process.
A fifth module, configured to determine whether the object is located in a third segment S3 of the cache, if so, switch to the sixth module, otherwise, switch to the seventh module;
a sixth module, configured to determine whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr2, if so, move the object to the head of the second segment S2 of the cache, and then end the process, otherwise, move the object to the head of the third segment S3 of the cache, and end the process;
a seventh module, configured to read the object from the background database, and determine whether there is enough space in the cache to store the object, if so, switch to the tenth module, otherwise, switch to the eighth module;
an eighth module, configured to determine whether a buffer size of the first segment S1 or the second segment S2 of the buffer exceeds a corresponding preset length-to-ratio threshold, if so, move an object at the tail of the first segment S1 or the second segment S2 of the buffer to the head of the next segment, and turn to the ninth module, otherwise, turn to the ninth module;
a ninth module, configured to determine whether a buffer size of the third segment S3 of the buffer exceeds a preset length-to-ratio threshold, if so, delete the object at the tail of the third segment S3 of the buffer from the buffer, and then switch to the tenth module, otherwise, switch to the tenth module;
and a tenth module, configured to input the object corresponding to the access request obtained by the first module into the trained AdaBoost classification model to obtain a predicted value, and determine whether the predicted value is greater than 0, if so, move the object to the head of the second segment S2 of the cache, where the process is ended, otherwise, move the object to the head of the third segment S3 of the cache, and end the process.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
1. the invention adopts the steps (1) to (6) to divide the cache into three sections of S1, S2 and S3 with different lengths, designs the lifting rule by combining the access frequency, stores the objects with high access frequency in the high-level cache section as much as possible, and prolongs the residence time of the objects in the cache, thereby improving the cache hit rate, and reducing the consumption of network bandwidth and the access delay of users.
2. The method adopts the step (10), the object predicted to be revisited by the AdaBoost classifier is placed at the head of the second section of the cache, and the object is given longer cache residence time to be hit next time; the object predicted by the AdaBoost classifier not to be accessed again is placed at the head of the third section of the cache, so that the object is given shorter cache residence time so as to be driven out of the cache more quickly, and the cache hit rate is further improved.
Drawings
FIG. 1 is a flow chart of the adaptive segmented caching method based on classification prediction according to the present invention;
FIG. 2 is a graph comparing cache performance of the present invention with a prior art classic cache policy, where FIG. 2(a) is a performance comparison of hit rates and FIG. 2(b) is a performance comparison of byte hit rates.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
. With the coming of big data era, data access modes are diversified, the traditional cache replacement strategy is easy to cause the problems of cache pollution and the like, the invention aims to provide a self-adaptive (namely different sizes and different priorities) segmented cache method based on classification prediction, and the method aims to optimize the configuration management of cache space by fully utilizing the characteristics of data access.
As shown in fig. 1, the present invention provides a classification prediction based adaptive segment caching method, which includes the following steps:
(1) receiving an access request from a user, judging whether a corresponding object is in a cache or not according to the access request, if so, turning to the step (2), otherwise, turning to the step (7); the cache comprises three sections S1, S2 and S3;
(2) judging whether the object is located in the first segment S1 of the cache, if so, moving the object to the head of the first segment S1, and ending the process, otherwise, turning to the step (3);
(3) judging whether the object is located in the second segment S2 of the cache, if so, turning to the step (4), otherwise, turning to the step (5);
(4) judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold value thrl, if so, moving the object to the head of the first section of the cache S1, and ending the process, otherwise, moving the object to the head of the second section of the cache S2, and ending the process;
specifically, the preset access frequency threshold value thrl ranges from 5 to 10, and is preferably 5.
(5) Judging whether the object is located in a third section S3 of the cache, if so, turning to the step (6), otherwise, turning to the step (7);
(6) judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr2, if so, moving the object to the head of the second section S2 of the cache, and then ending the process, otherwise, moving the object to the head of the third section S3 of the cache, and ending the process;
specifically, the preset access frequency threshold thr2 has a value ranging from 1 to 3, preferably 2.
(7) Reading the object from the background database, determining whether enough space is left in the cache for storing the object, if so, turning to the step (10), otherwise, turning to the step (8);
(8) judging whether the cache size of the first segment S1 or the second segment S2 exceeds the corresponding preset length-to-ratio threshold, if so, moving the object at the tail of the first segment S1 or the second segment S2 to the head of the next segment, and turning to the step (9), otherwise, turning to the step (9);
specifically, the preset length-to-length ratio threshold is set to range from 0.1 to 0.9, and preferably, the length-to-length ratio threshold corresponding to the first segment S1 of the cache is 0.1, and the length-to-length ratio threshold corresponding to the second segment S2 of the cache is 0.7.
(9) Judging whether the cache size of the third section S3 of the cache exceeds a preset length ratio threshold, if so, deleting the object at the tail part of the third section S3 of the cache from the cache, and then turning to the step (9), otherwise, turning to the step (10);
specifically, the value range of the preset length-to-ratio threshold is 0.1 to 0.9, and preferably 0.2.
(10) Inputting the object corresponding to the access request obtained in the step (1) into a trained AdaBoost classification model to obtain a predicted value, and judging whether the predicted value is greater than 0, if so, moving the object to the head of the second section S2 of the cache, and ending the process, otherwise, moving the object to the head of the third section S3 of the cache, and ending the process.
Specifically, if the predicted value is 0, it indicates that the object is not to be accessed again, and if the predicted value is 1, it indicates that the object is to be accessed here.
Preferably, the size setting of the three sections S1, S2, S3 buffered in step (1) is obtained by the following experiment: the size ratio of the first segment cache S1 in the entire cache (highest-level cache segment) sr is set first1Is between 0.1 and 0.9, then the size ratio (middle-level cache segment) sr of the second segment cache S2 in the whole cache is set2Between 0.1 and 0.9, subsequently according to sr1And sr2The value of (c) obtains the size of the third segment cache S3 (lowest level cache segment) in the whole cache to be equal to sr3＝1-sr1-sr2Then observe the cache hit rate within a certain time window (the time window ranges from half a day to one day, preferably one day) as the size of the three-segment cache changes, and then adjust sr1、sr2And repeating the above processes until the value pair which makes the cache hit rate maximum is finally foundCorresponding sr1、sr2And sr3。
Preferably, the access frequency thresholds thr1 and thr2 used in step (4) and step (6) are obtained by statistics as follows: collecting historical cache log data of a certain time window (the range of the time window is from the previous day to the previous week, preferably the previous day), dividing all historical cache log data into a plurality of access levels according to the access frequency, and respectively counting the total request number, the total object number and the hit contribution rate corresponding to each access level. The hit contribution rate is defined as the number of hits corresponding to each access level divided by the total number of hits. The access frequency thresholds thr1 and thr2 are generally chosen according to the following rules: the access frequency exceeding the threshold thr1 is the object with high access frequency, and the sum of the total hit contribution rate should reach 70%. The access frequency is lower than the threshold thr2, namely the low access frequency object, and the sum of the total hit contribution rate is lower than 10%.
Preferably, the AdaBoost classification model used in step (10) is trained by the following steps:
(10-1) obtaining a training data set S { (x)1，y1)，(x2，y2)，...，(xn，yn) In which xiIs a feature of the ith object, and yiIs the class label of the ith object, if the ith object is actually accessed again, yi1, otherwise yi-1, wherein i ∈ [1, n ]]N is the total number of objects in the training dataset;
(10-2) training the weight D of each object i in the data set S1(i) Initializing to 1/n;
(10-3) setting a counter t equal to 1;
(10-4) judging whether T is larger than a preset iteration threshold value T, if so, turning to the step (10-10), otherwise, turning to the step (10-5);
t ranges from 100 to 1000, preferably 100.
(10-5) training the weight D of each object i in the data set S1(i) And feature xiInputting AdaBoost classification model to obtain each feature xiCorresponding weak classification result ft(xi) The weak classification results corresponding to all the features form the tth weak classification result set ft(x) Where t represents the t-th weak classifier result.
(10-6) the t-th weak classification result set f obtained according to the step (10-5)t(x) Calculating an error estimate epsilon thereoft：
(10-7) obtaining an error estimation value epsilon according to the step (10-6)tObtaining a weight update parameter alphat：
(10-8) obtaining an error estimation value epsilon according to the step (10-6)tAnd the weight update parameter alpha obtained in the step (10-7)tObtaining the weight D of each object i in the t +1 iteration processt+1(i)：
(10-9) setting t as t +1, and returning to the step (10-4);
(10-10) generating a final strong classification result F (x) according to the T weak classification result sets, and taking the final strong classification result F (x) as a trained AdaBoost classification model:
specifically, in step (10-6), the error estimation value ε is calculated by using the following equationt：
Specifically, in step (10-7), the weight update parameter α is calculated by the following formulat：
Specifically, the weight D of each object in the t +1 th iteration process in step (10-8)t+1(i) Equal to:
wherein Z istIs a normalization factor and has
etIs the base of the natural logarithm.
Specifically, the final strong classification result f (x) in step (10-10) is equal to:
compared with the traditional cache working process, the cache space is divided into three sections with different sizes and priorities, the allocation proportion of each cache section is adjusted based on the cache hit rate in a certain time window, and the lifting rule is designed according to the access frequency, so that the object with high access frequency is given longer cache residence time for the next hit; on the basis, the invention further introduces a machine learning classifier to predict whether the object is to be accessed again, and the object is placed in different positions of three sections of caches according to the prediction result, so that different decisions are made according to the prediction result, the cache hit rate is improved, the miss rate is reduced, and the time and the network bandwidth required by searching the background storage are reduced.
Results of the experiment
In order to illustrate the superiority of the adaptive segmented caching method provided by the invention, the invention performs related tests on the Tencent QQ album data set, selects five classic cache replacement algorithms of LRU, FIFO, S3LRU, ARC and OPT to compare with the invention, and records the cache hit rate and the cache byte hit rate. The best permutation algorithm OPT is also called Belady, and the rejected pages selected by the best permutation algorithm OPT are pages that are never used later or are no longer accessed within the longest (future) time. Since it is not predictable which object will no longer be accessed for the longest time in the future, the algorithm represents only the best case in the ideal state, and OPT is often used to evaluate other algorithms.
The experimental environment of the invention is as follows: the CPU is Intel (R) core (TM) i5-8265U CPU @1.80GHz, the memory is 8GB, the hard disk capacity is 4TB, the network card selects a hundred million network card, and the system is operated in Win10And then, a Python language is mainly adopted for simulation, Sklearn packets are used in the training process of the classification algorithm, statistical analysis and playback test are carried out according to the collected data set, and specific parameters in the experiment are set as follows: thr1 ═ 5, thr2 ═ 2, sr1＝0.1，sr20.7. The cache size actually specified by the Tencent QQ album server is recorded as X (10% of the total size of all pictures), and the abscissa in the graph takes 0.1X-2.0X.
From the comparison result of fig. 2(a), when the cache size is between 0.1X and 1.0X, the cache hit rate of the present invention is higher than 8.64% to 29.66% of the FIFO and higher than 4.12% to 19.10% of the LRU, compared to the FIFO algorithm and the LRU algorithm. As is evident from the figure: when the cache size is 0.1X, the performance difference between the invention and FIFO and LRU is obviously larger, so the invention can exert better effect under the condition of limited cache space. Meanwhile, when the cache size is in the range of 0.1X-1.0X, the cache hit rate of the invention is improved by 2.87% -7.77% compared with S3LRU and improved by 2.98% -7.43% compared with ARC.
From the comparison result of fig. 2(b), it can be seen that when the cache size is between 0.1X and 1.0X, the byte hit rate of the present invention is higher than about 9.72% to 33.02% of FIFO, compared to FIFO algorithm and LRU algorithm; compared with LRU, the promotion is 4.97% -21.68%; compared with S3LRU, the improvement is 2.87% -7.77%; compared with ARC, the ARC is improved by 2.98-7.43%. In summary, the adaptive segmented cache strategy based on classification prediction provided by the invention can more effectively manage the cache space, and further improve the cache hit rate and the cache byte hit rate.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A self-adaptive segmented caching method based on classification prediction is characterized by comprising the following steps:
(1) receiving an access request from a user, judging whether a corresponding object is in a cache or not according to the access request, if so, turning to the step (2), otherwise, turning to the step (7); the cache comprises three sections S1, S2 and S3;
(2) judging whether the object is located in the first segment S1 of the cache, if so, moving the object to the head of the first segment S1, and ending the process, otherwise, turning to the step (3);
(3) judging whether the object is located in the second segment S2 of the cache, if so, turning to the step (4), otherwise, turning to the step (5);
(4) and judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr1, if so, moving the object to the head of the first section S1 of the cache, and ending the process, otherwise, moving the object to the head of the second section S2 of the cache, and ending the process.
(5) Judging whether the object is located in a third section S3 of the cache, if so, turning to the step (6), otherwise, turning to the step (7);
(6) judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr2, if so, moving the object to the head of the second section S2 of the cache, and then ending the process, otherwise, moving the object to the head of the third section S3 of the cache, and ending the process;
(7) reading the object from the background database, determining whether enough space is left in the cache for storing the object, if so, turning to the step (10), otherwise, turning to the step (8);
(8) judging whether the cache size of the first segment S1 or the second segment S2 exceeds the corresponding preset length-to-ratio threshold, if so, moving the object at the tail of the first segment S1 or the second segment S2 to the head of the next segment, and turning to the step (9), otherwise, turning to the step (9);
(9) judging whether the cache size of the third section S3 of the cache exceeds a preset length ratio threshold, if so, deleting the object at the tail part of the third section S3 of the cache from the cache, and then turning to the step (10), otherwise, turning to the step (10);
(10) inputting the object corresponding to the access request obtained in the step (1) into a trained AdaBoost classification model to obtain a predicted value, and judging whether the predicted value is greater than 0, if so, moving the object to the head of the second section S2 of the cache, and ending the process, otherwise, moving the object to the head of the third section S3 of the cache, and ending the process.
2. The adaptive segment caching method based on classification prediction according to claim 1 or 2,
the preset access frequency threshold thr1 is in the range of 5 to 10, preferably 5;
the preset access frequency threshold thr2 is in the range of 1 to 3, preferably 2.
In the steps (8) and (9), the value range of the preset length-to-ratio threshold is 0.1 to 0.9.
3. The adaptive segmented caching method based on classified prediction according to any one of claims 1 to 3, wherein the size setting of the three segments S1, S2 and S3 cached in the step (1) is obtained by the following process: firstly, the size ratio sr of the first segment cache S1 in the whole cache is set1Is between 0.1 and 0.9, then the size ratio sr of the second segment cache S2 in the whole cache is set2Between 0.1 and 0.9, subsequently according to sr1And sr2To obtain the size of the third segment buffer S3 in the whole buffer is equal to sr3＝1-sr1-sr2Then observing the cache hit rate in a certain time window along with the change of the size of the three-segment cache, and then adjusting sr1、sr2And repeating the above processes until sr corresponding to the maximum cache hit rate is finally found1、sr2And sr3。
4. The adaptive segmented caching method based on classification prediction according to claim 1, wherein the AdaBoost classification model used in the step (10) is trained by the following steps:
(10-1) obtaining a training data set S { (x)1,y1),(x2,y2),…,(xn,yn) } of whichIn xiIs a feature of the ith object, and yiIs the class label of the ith object, if the ith object is actually accessed again, yi1, otherwise yi-1, wherein i ∈ [1, n ]]N is the total number of objects in the training dataset;
(10-2) training the weight D of each object i in the data set S1(i) Initializing to 1/n;
(10-3) setting a counter t equal to 1;
(10-4) judging whether T is larger than a preset iteration threshold value T, if so, turning to the step (10-10), otherwise, turning to the step (10-5);
(10-5) training the weight D of each object i in the data set S1(i) And feature xiInputting AdaBoost classification model to obtain each feature xiCorresponding weak classification result ft(xi) The weak classification results corresponding to all the features form the tth weak classification result set ft(x) Where t represents the t-th weak classifier result.
(10-6) the t-th weak classification result set f obtained according to the step (10-5)t(x) Calculating an error estimate epsilon thereoft：
(10-7) obtaining an error estimation value epsilon according to the step (10-6)tObtaining a weight update parameter alphat：
(10-8) obtaining an error estimation value epsilon according to the step (10-6)tAnd the weight update parameter alpha obtained in the step (10-7)tObtaining the weight D of each object i in the t +1 iteration processt+1(i)：
(10-9) setting t as t +1, and returning to the step (10-4);
and (10-10) generating a final strong classification result F (x) according to the T weak classification result sets, and taking the final strong classification result F (x) as a trained AdaBoost classification model.
5. The adaptive segment caching method based on classification prediction according to claim 4,
in step (10-6), the error estimation value epsilon is calculated by using the following formulat：
6. The adaptive segmented caching method based on classification prediction as claimed in claim 4, wherein the weight update parameter α is calculated in step (10-7) by using the following formulat：
7. The adaptive segmented caching method based on classification prediction according to claim 4, wherein the weight D of each object in the t +1 th iteration in the step (10-8)t+1(i) Equal to:
wherein Z istIs a normalization factor and has
etIs the base of the natural logarithm.
8. The adaptive segment caching method based on classification prediction according to claim 4, wherein the final strong classification result F (x) in the steps (10-10) is equal to:
9. an adaptive segmented cache system based on classification prediction, comprising:
the first module is used for receiving an access request from a user, judging whether a corresponding object is in a cache or not according to the access request, if so, switching to the second module, and otherwise, switching to the seventh module; the cache comprises three sections S1, S2 and S3;
the second module is used for judging whether the object is positioned in the first segment S1 of the cache, if so, the object is moved to the head of the first segment S1, the process is finished, otherwise, the process is shifted to the third module;
the third module is used for judging whether the object is positioned in the second segment S2 of the cache, if so, the fourth module is switched, otherwise, the fifth module is switched;
and the fourth module is used for judging whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr1, if so, moving the object to the head of the first section S1 of the cache, and ending the process, otherwise, moving the object to the head of the second section S2 of the cache, and ending the process.
A fifth module, configured to determine whether the object is located in a third segment S3 of the cache, if so, switch to the sixth module, otherwise, switch to the seventh module;
a sixth module, configured to determine whether the access frequency of the object is greater than or equal to a preset access frequency threshold thr2, if so, move the object to the head of the second segment S2 of the cache, and then end the process, otherwise, move the object to the head of the third segment S3 of the cache, and end the process;
a seventh module, configured to read the object from the background database, and determine whether there is enough space in the cache to store the object, if so, switch to the tenth module, otherwise, switch to the eighth module;
an eighth module, configured to determine whether a buffer size of the first segment S1 or the second segment S2 of the buffer exceeds a corresponding preset length-to-ratio threshold, if so, move an object at the tail of the first segment S1 or the second segment S2 of the buffer to the head of the next segment, and turn to the ninth module, otherwise, turn to the ninth module;
a ninth module, configured to determine whether a buffer size of the third segment S3 of the buffer exceeds a preset length-to-ratio threshold, if so, delete the object at the tail of the third segment S3 of the buffer from the buffer, and then switch to the tenth module, otherwise, switch to the tenth module;
and a tenth module, configured to input the object corresponding to the access request obtained by the first module into the trained AdaBoost classification model to obtain a predicted value, and determine whether the predicted value is greater than 0, if so, move the object to the head of the second segment S2 of the cache, where the process is ended, otherwise, move the object to the head of the third segment S3 of the cache, and end the process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110723648.1A CN113419976B (en) | 2021-06-29 | 2021-06-29 | Self-adaptive segmented caching method and system based on classification prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110723648.1A CN113419976B (en) | 2021-06-29 | 2021-06-29 | Self-adaptive segmented caching method and system based on classification prediction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113419976A true CN113419976A (en) | 2021-09-21 |
| CN113419976B CN113419976B (en) | 2024-04-26 |
Family
ID=77716970
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110723648.1A Active CN113419976B (en) | 2021-06-29 | 2021-06-29 | Self-adaptive segmented caching method and system based on classification prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113419976B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114419875A (en) * | 2021-12-08 | 2022-04-29 | 斑马网络技术有限公司 | Vehicle travel segmentation method and device and storage medium |
| CN115130032A (en) * | 2022-07-05 | 2022-09-30 | 华中科技大学 | Method and system for constructing deficiency rate curve in heterogeneous granularity storage system |
| CN117555933A (en) * | 2024-01-12 | 2024-02-13 | 深圳市智百威科技发展有限公司 | Method and system for solving high concurrency data access |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6463508B1 (en) * | 1999-07-19 | 2002-10-08 | International Business Machines Corporation | Method and apparatus for caching a media stream |
| US20050268037A1 (en) * | 2004-05-27 | 2005-12-01 | International Business Machines Corporation | Cache hit ratio estimating apparatus, cache hit ratio estimating method, program, and recording medium |
| CN107247675A (en) * | 2017-05-31 | 2017-10-13 | 华中科技大学 | A kind of caching system of selection and system based on classification prediction |
| CN109739780A (en) * | 2018-11-20 | 2019-05-10 | 北京航空航天大学 | Dynamic secondary based on the mapping of page grade caches flash translation layer (FTL) address mapping method |
-
2021
- 2021-06-29 CN CN202110723648.1A patent/CN113419976B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6463508B1 (en) * | 1999-07-19 | 2002-10-08 | International Business Machines Corporation | Method and apparatus for caching a media stream |
| US20050268037A1 (en) * | 2004-05-27 | 2005-12-01 | International Business Machines Corporation | Cache hit ratio estimating apparatus, cache hit ratio estimating method, program, and recording medium |
| CN107247675A (en) * | 2017-05-31 | 2017-10-13 | 华中科技大学 | A kind of caching system of selection and system based on classification prediction |
| CN109739780A (en) * | 2018-11-20 | 2019-05-10 | 北京航空航天大学 | Dynamic secondary based on the mapping of page grade caches flash translation layer (FTL) address mapping method |
Non-Patent Citations (1)
| Title |
|---|
| 赵中全;刘丹;: "基于树扩展朴素贝叶斯分类器的Web代理服务器缓存优化", 计算机工程, no. 01, 15 January 2017 (2017-01-15) * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114419875A (en) * | 2021-12-08 | 2022-04-29 | 斑马网络技术有限公司 | Vehicle travel segmentation method and device and storage medium |
| CN114419875B (en) * | 2021-12-08 | 2023-12-22 | 斑马网络技术有限公司 | Vehicle travel segmentation method, device and storage medium |
| CN115130032A (en) * | 2022-07-05 | 2022-09-30 | 华中科技大学 | Method and system for constructing deficiency rate curve in heterogeneous granularity storage system |
| CN115130032B (en) * | 2022-07-05 | 2024-07-09 | 华中科技大学 | Deletion rate curve construction method and system in heterogeneous granularity storage system |
| CN117555933A (en) * | 2024-01-12 | 2024-02-13 | 深圳市智百威科技发展有限公司 | Method and system for solving high concurrency data access |
| CN117555933B (en) * | 2024-01-12 | 2024-03-22 | 深圳市智百威科技发展有限公司 | Method and system for solving high concurrency data access |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113419976B (en) | 2024-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113419976B (en) | Self-adaptive segmented caching method and system based on classification prediction | |
| Cheng et al. | LRU-SP: a size-adjusted and popularity-aware LRU replacement algorithm for web caching | |
| Wu et al. | Dynamic content update for wireless edge caching via deep reinforcement learning | |
| KR101443231B1 (en) | Cache memory capable of adjusting burst length of write-back data in write-back operation | |
| CN100395750C (en) | Buffer store management method | |
| CN110795363B (en) | Hot page prediction method and page scheduling method of storage medium | |
| CN110532200B (en) | Memory system based on hybrid memory architecture | |
| CN104077242A (en) | Cache management method and device | |
| US11593268B2 (en) | Method, electronic device and computer program product for managing cache | |
| CN112051968A (en) | Kafka-based distributed data stream hierarchical cache automatic migration algorithm | |
| US8250302B2 (en) | Cache management using sampled values assigned to a request | |
| CN113268458B (en) | Caching method and system based on cost sensitive classification algorithm | |
| CN111506517B (en) | Flash memory page level address mapping method and system based on access locality | |
| CN113271631A (en) | Novel content cache deployment scheme based on user request possibility and space-time characteristics | |
| CN115827511A (en) | Method and device for setting cache | |
| CN113225380B (en) | Content distribution network caching method and system based on spectral clustering | |
| Meint et al. | From FIFO to predictive cache replacement | |
| CN114860168A (en) | Cache placement method, system and medium for long and short time slot combination optimization | |
| CN113050894A (en) | Agricultural spectrum hybrid storage system cache replacement algorithm based on cuckoo algorithm | |
| CN117971503B (en) | Data caching method and system based on edge calculation | |
| CN116467353B (en) | Self-adaptive adjustment caching method and system based on LRU differentiation | |
| Wu et al. | Objective-greedy algorithms for long-term Web prefetching | |
| CN109857680B (en) | LRU flash memory cache management method based on dynamic page weight | |
| Abdel-Baset et al. | Cache Policies for Smartphone in Flexible Learning: A Comparative Study. | |
| Zhao et al. | A buffer algorithm of flash database based on LRU |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |