CN115964337A - Eight-path rack server based on access heat analysis file storage method - Google Patents
Eight-path rack server based on access heat analysis file storage method Download PDFInfo
- Publication number
- CN115964337A CN115964337A CN202210234923.8A CN202210234923A CN115964337A CN 115964337 A CN115964337 A CN 115964337A CN 202210234923 A CN202210234923 A CN 202210234923A CN 115964337 A CN115964337 A CN 115964337A
- Authority
- CN
- China
- Prior art keywords
- file
- access
- value
- heat
- statistical period
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an eight-path rack server based on an access heat analysis file storage method, which comprises the steps of calculating the access frequency of a file, calculating the access heat of the file and creating a dynamic copy of the file; the file access frequency is the number of times of access by a user in unit time and is determined by the number of times of file access in a statistical period and the time of the statistical period; the file access heat is the required degree of the file, and the value of the file access heat is determined by the file access frequency; during the file access process, the access frequency of the file in a statistical period is actually calculated, the current access requirement of the file is calculated according to the historical access frequency, and finally the number of the file copies is dynamically adjusted. The invention has the beneficial effects that the dynamic copy creating algorithm is provided, the current access requirement is predicted by analyzing the historical access requirement of the file, the copy number of the file is dynamically adjusted, and the data service performance and the operation efficiency of the file are improved while the disk space is saved.
Description
Technical Field
The invention relates to the field of server data storage, in particular to an eight-path rack server based on an access heat analysis file storage method.
Background
The explosive growth of global data volume and the promotion of data asset value have become the basic features of the development of the current digital economic era, and the increase of mobile devices and intelligent sensing devices in the aspect of data volume enables the global data volume to rapidly increase, and according to IDC prediction, the global data volume reaches 163ZB by 2025, which is equivalent to ten times of 16.1ZB data volume in 2016. In the aspect of data asset value, a digital technology application scene represented by cloud computing, big data and artificial intelligence is combined with enterprise production and resident life acceleration, the value of the data asset in the enterprise production, resident life and social treatment is increasingly highlighted, and the enterprise can realize cloud application intelligence by using a digital technology to realize fine production and intelligent marketing; the users can enjoy digital technical achievements through digital applications such as online banking, electronic shopping malls, online conferences, short videos, unmanned driving and the like.
The prior art CN110188084B discloses a distributed file storage system and a file storage method. A distributed file storage method is realized based on a distributed file storage system, and comprises the following steps: presetting a unique storage unit number for each storage unit, and establishing association between the storage units which are mutually prepared; after receiving a file writing request, a receiving layer of the file storage system firstly judges whether the file requested to be written is written for the first time; for the file which is written for the first time, the file storage system allocates a storage unit for the file which is requested to be written, and the allocated storage unit allocates a unique file number for the file; if the file to be written already exists, the file number is directly returned to the service party; the storage unit feeds back the key information of the file index of the storage file to the cache unit in real time. The invention has the advantages of a common distributed file storage system, and has the characteristics of no center and easy data recovery. However, a large number of data users cannot accurately estimate the using heat of the data before storage, and it is difficult to manually select a proper storage strategy.
Therefore, the storage method for the access heat analysis file allows a user to dynamically select and use the copy strategy for storage according to the estimation of the data heat.
Disclosure of Invention
The invention aims to provide an eight-path rack server based on an access heat analysis file storage method, so as to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme: an eight-path rack server based on an access heat analysis file storage method comprises the steps of calculating file access frequency, calculating file access heat and creating a dynamic file copy;
the file access frequency is the number of times of access by a user in unit time and is determined by the number of times of file access in a statistical period and the time of the statistical period;
the file access heat is the required degree of the file, and the value of the file access heat is determined by the file access frequency;
and during the file access process, the access frequency of the file in a statistical period is actually calculated, the current access requirement of the file is calculated according to the historical access frequency, and finally the number of the file copies is dynamically adjusted.
Preferably, the file access frequency calculation procedure is as follows:
s11: the initial value of the counter is 0, the file is accessed, the value of the counter is added with 1, and the beginning of the counting period is when the value of the counter is 1; when the counter value is a, the end of the statistical period is determined;
s12: judging whether the counting period is started or ended;
s13: if the statistical period is started or ended, judging whether the statistical period is started or ended;
s14: if the counting period is started, timing is started, and the step S1 is returned; if the counting period is over, the timer finishes timing, and the step S7 is carried out;
s15: if the counting period is not the beginning or the end of the counting period, judging whether the recording time of the timer is overtime or not;
s16: if the judgment result is overtime, the step S7 is carried out; if the judgment result is that the time is not overtime, returning to the step S1;
s17: and calculating the file access frequency, and setting a counter to be 0 to enter the next statistical period.
Preferably, the counter determines whether a statistical period starts or ends, the counter starts the statistical period when the counter value is 1, ends the statistical period when the counter value is a, and a is the number of times a file is accessed in the statistical period, and the timer calculates the duration t of the statistical period.
Preferably, the file access frequency is:
in the formula: f. of k The frequency of file access in the kth statistical period is obtained; a is the number of times of accessing the file in the access cycle; t is the duration of the access period.
Preferably, the judgment timer is used for judging whether the recording time is overtime or not, and the overtime critical time is b in response to the condition that the file cannot finish a statistical period after delay, namely the access frequency is extremely low, and if t is larger than or equal to b, the file access is overtime.
Preferably, the file access heat calculation formula is as follows:
in the formula: h is ij For the heat of file i at time jA value of the metric; alpha is a constant and has the function of normalizing the value of the heat degree of the file; s i The influence of the size of the file i on the heat degree of the file i; f j The influence of the accessed frequency of the file on the j moment of the file is shown; i and j are both positive integers.
Preferably, said S i Is determined by the size s of the file i And size s of system memory data block 0 And determining the expression as follows:
in the formula: s i Is the size, s, of the file i 0 Store the size of the data block for the system, [ s ] i /s 0 ]Denotes a value of not more than s i /s 0 Maximum integer of (1), s i ％s 0 Denotes s i And s 0 The remainder between.
Preferably, said F j The value of (b) is determined by the frequency and weight of the file accessed in l statistical cycles, and the expression is as follows:
in the formula: when the formula (1) is satisfied, j is located in the k-th statistical period, k =1; when the formula (2) is satisfied, j is located in the kth statistical period, k is not less than 2 and is an integer, l is a positive integer, and k is less than l; when the formula (3) is satisfied, j is located in the kth statistical period, k is not less than 2 and is an integer, l is a positive integer, and k is greater than l;
wherein f is k-1 Representing the frequency magnitude in the k-1 th statistical period; beta is a k-1 Denotes f k-1 The influence value on the heat of the file at the moment j is satisfied
Preferably, the process of creating the dynamic copy of the file is as follows:
s21: when a file is accessed, inputting historical access frequency of the file;
s22: calculating the file access heat;
s23: calculating a file copy dynamic creation value;
s24: converting the required quantity of the copies of the files;
s25: judging whether the number of the existing copies needs to be adjusted or not;
s26: if the number of the copies needs to be adjusted, copy or delete operation of the copies is executed, and the process is ended; if the number of copies does not need to be adjusted, the process is directly ended.
Preferably, the method for calculating the dynamic creation value of the file copy comprises the following steps:
v ij ＝h ij ·q i
in the formula: v. of ij Dynamically creating values for the file copies; h is ij Represents the heat value of the file i at the moment j, q i The static impact factor calculation formula for file i is as follows:
q i ＝β/s i
in the formula: beta is a constant and has the function of normalizing the numerical value of the static influence factor of the file; s i Is the size of file i.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a dynamic copy creating algorithm aiming at the dynamism of file access requirements, the algorithm predicts the current access requirements by analyzing the historical access requirements of the files, dynamically adjusts the copy number of the files, saves the disk space and improves the data service performance of the files.
2. In order to ensure correctness of data access, a copy management policy generally covers several aspects of copy creation, copy location, copy selection, copy consistency maintenance, and the like. As important content in mass data distributed storage, the copy creating technology can ensure the high-reliability storage of the files, reduce the access delay of the files, improve the capacity of the files for providing data services and optimize the overall performance of the system by reasonably deciding the number of the copies and selecting the storage positions of the copies.
Drawings
FIG. 1 is a flow chart of file access frequency calculation according to the present invention;
FIG. 2 is a flow chart of creating a dynamic copy of a file according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention relates to a storage method of access heat analysis files, which comprises the steps of calculating the access frequency of the files, calculating the access heat of the files and creating dynamic copies of the files;
the file access frequency is the number of times of access by a user in unit time and is determined by the number of times of file access in a statistical period and the time of the statistical period;
the file access heat is the required degree of the file, and the value of the file access heat is determined by the file access frequency;
during the file access process, the access frequency of the file in a statistical period is actually calculated, the current access requirement of the file is calculated according to the historical access frequency, and finally the number of the file copies is dynamically adjusted.
As shown in fig. one, the file access frequency calculation flow is as follows:
s11: the initial value of the counter is 0, the file is accessed, the value of the counter is added with 1, and the beginning of the counting period is when the value of the counter is 1; when the counter value is a, the end of the counting period is determined;
s12: judging whether the counting period is started or ended;
s13: if the statistical period is started or ended, judging whether the statistical period is started or ended;
s14: if the counting period is started, timing is started, and the step S1 is returned; if the counting period is over, the timer finishes timing, and the step S7 is carried out;
s15: if the counting period is not the beginning or the end of the counting period, judging whether the recording time of the timer is overtime or not;
s16: if the judgment result is overtime, the step S7 is carried out; if the judgment result is that the time is not overtime, returning to the step S1;
s17: and calculating the file access frequency, and setting a counter to be 0 to enter the next statistical period.
The counter judges whether a statistical period starts or ends, the counter starts the statistical period when the counter value is 1, the counter ends the statistical period when the counter value is a, and a is the number of times of accessing files in the statistical period, and the timer calculates the duration time t of the statistical period.
The file access frequency is as follows:
in the formula: f. of k The frequency of file access in the kth statistical period is obtained; a is the number of times of accessing the file in the access cycle; t is the duration of the access period.
And judging whether the recording time of the timer is overtime or not, and responding to the condition that the file cannot finish a statistical period after delay, namely the access frequency is extremely low, wherein the overtime critical time is b, and if t is more than or equal to b, the file access is overtime.
The file access heat calculation formula is as follows:
in the formula: h is ij The heat value of the file i at the moment j is obtained; alpha is a constant and has the function of normalizing the value of the heat degree of the file; s i The influence of the size of the file i on the heat degree of the file i; f j The influence of the accessed frequency of the file on the j moment of the file is taken as the influence; i and j are both positive integers.
Said S i Is determined by the size s of the file i And size s of system memory data block 0 And determining the expression as follows:
in the formula: s i Is the size, s, of the file i 0 Store the size of the data block for the system, [ s ] i /s 0 ]Denotes a value of not more than s i /s 0 Maximum integer of (1), s i ％s 0 Denotes s i And s 0 The remainder between.
Said F j The value of (a) is determined by the frequency and weight of the accessed file in l statistical periods, and the expression is as follows:
in the formula: when the formula (1) is satisfied, j is located in the kth statistical period, and k =1; when the formula (2) is satisfied, j is located in the kth statistical period, k is not less than 2 and is an integer, l is a positive integer, and k is less than l; when the formula (3) is satisfied, j is located in the kth statistical period, k is not less than 2 and is an integer, l is a positive integer, and k is greater than l;
wherein f is k-1 Representing the frequency magnitude in the k-1 statistical period; beta is a k-1 Denotes f k-1 The influence value on the heat of the file at the moment j is satisfied
As shown in FIG. 2, the creation flow of the dynamic copy of the file is as follows:
s21: when a file is accessed, inputting historical access frequency of the file;
s22: calculating the access heat of the file;
s23: calculating a dynamic creation value of the file copy;
s24: converting the required number of the file copies;
s25: judging whether the number of the existing copies needs to be adjusted or not;
s26: if the number of the copies needs to be adjusted, copy or delete operation of the copies is executed, and the process is ended; if the number of copies does not need to be adjusted, the process is directly ended.
The method for calculating the dynamic creation value of the file copy comprises the following steps:
v ij ＝h ij ·q i
in the formula: v. of ij Dynamically creating values for the file copies; h is ij Representing the heat value of the file i at time j, q i The static impact factor calculation formula for file i is as follows:
q i ＝β/s i
in the formula: beta is a constant and has the function of normalizing the numerical value of the static influence factor of the file; s i Is the size of file i.
The dynamic copy creating algorithm predicts the current access requirement by analyzing the historical access requirement of the file, dynamically adjusts the copy number of the file, saves the disk space and improves the data service performance of the file.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. The eight-path rack server based on the access heat analysis file storage method is characterized in that the access heat analysis file storage method comprises the steps of calculating the access frequency of a file, calculating the access heat of the file and creating a dynamic copy of the file;
the file access frequency is the number of times of access by a user in unit time and is determined by the number of times of file access in a statistical period and the time of the statistical period;
the file access heat is the required degree of the file, and the value of the file access heat is determined by the file access frequency;
during the file access process, the access frequency of the file in a statistical period is actually calculated, the current access requirement of the file is calculated according to the historical access frequency, and finally the number of the file copies is dynamically adjusted.
2. The eight-way rack server based on the access heat analysis file storage method according to claim 1, wherein the file access frequency calculation process is as follows:
s11: the initial value of the counter is 0, the file is accessed, the value of the counter is added with 1, and the beginning of the counting period is when the value of the counter is 1; when the counter value is a, the end of the counting period is determined;
s12: judging whether the counting period is started or ended;
s13: if the statistical period is started or ended, judging whether the statistical period is started or ended;
s14: if the counting period is started, timing is started, and the step S1 is returned; if the counting period is over, the timer finishes timing, and the step S7 is carried out;
s15: if the counting period is not the beginning or the end of the counting period, judging whether the recording time of the timer is overtime or not;
s16: if the judgment result is overtime, the step S7 is carried out; if the judgment result is that the time is not overtime, returning to the step S1;
s17: and calculating the file access frequency, and setting a counter to be 0 to enter the next statistical period.
3. The eight-way rack server for analyzing the file storage method based on access heat according to claim 2, wherein the counter determines whether a statistical period starts or ends, the start of the statistical period is when the counter value is 1, the end of the statistical period is when the counter value is a, a is a number of times that a file is accessed in the statistical period, a is a positive integer, and the timer calculates a time t for which the statistical period lasts.
4. The eight-way rack-mounted server for analyzing the file storage method based on the access heat according to claim 3, wherein the file access frequency is:
in the formula: f. of k The frequency of file access in the kth statistical period is obtained; a is the number of times of accessing the file in the access period; t is the duration of the access period.
5. The eight-path rack server for analyzing the file storage method based on the access heat as claimed in claim 2, wherein the determination timer is configured to determine whether the recording time of the timer is overtime, in response to a situation that the file cannot complete a statistical period after a delay, that is, the access frequency is extremely low, the overtime threshold time is b, b is a constant, and if t ≧ b, the file access is overtime.
6. The eight-way rack server based on the access heat analysis file storage method according to claim 1, wherein the file access heat calculation formula is as follows:
in the formula: h is a total of ij The heat value of the file i at the moment j is obtained; gamma is a constant and has the function of normalizing the value of the heat degree of the file; s. the i Is the influence of the size of the file i on its heat; f j The influence of the accessed frequency of the file on the j moment of the file is shown; i and j are bothA positive integer.
7. The eight-way rack server for accessing the hotness analysis file storage method according to claim 6, wherein S is a server with eight racks i Is determined by the size s of the file i And size s of system memory data block 0 And determining the expression as follows:
in the formula: s i Is the size, s, of the file i 0 Store the size of the data block for the system, [ s ] i /s 0 ]Denotes a value of not more than s i /s 0 Maximum integer of (1), s i ％s 0 Denotes s i And s 0 The remainder between.
8. The eight-way rack server based on access heat analysis file storage method according to claim 1, wherein F is j The value of (a) is determined by the frequency and weight of the accessed file in l statistical periods, and the expression is as follows:
in the formula: when the formula (1) is satisfied, j is located in the k-th statistical period, k =1; when the formula (2) is satisfied, j is located in the kth statistical period, k is not less than 2 and is an integer, l is a positive integer, and k is less than l; when the formula (3) is satisfied, j is located in the kth statistical period, k is not less than 2 and is an integer, l is a positive integer, and k is greater than l;
wherein f is k-1 Representing the frequency magnitude in the k-1 th statistical period; beta is a k-1 Denotes f k-1 The influence value on the heat of the file at the moment j is satisfied
9. The eight-way rack-mounted server for the file storage method based on access heat analysis according to claim 1, wherein the creation process of the file dynamic copy is as follows:
s21: when a file is accessed, inputting historical access frequency of the file;
s22: calculating the file access heat;
s23: calculating a dynamic creation value of the file copy;
s24: converting the required number of the file copies;
s25: judging whether the number of the existing copies needs to be adjusted or not;
s26: if the number of the copies needs to be adjusted, copy or delete operation of the copies is executed, and the process is ended; if the number of copies does not need to be adjusted, the process is directly ended.
10. The eight-way rack-mounted server based on the access heat analysis file storage method according to claim 9, wherein the file copy dynamic creation value calculating method comprises the following steps:
v ij ＝h ij ·q i
in the formula: v. of ij Dynamically creating values for the file copies; h is a total of ij Representing the heat value of the file i at time j, q i The static impact factor calculation formula for file i is as follows:
q i ＝β/s i
in the formula: beta is a constant and has the function of normalizing the numerical value of the static influence factor of the file; s is i Is the size of file i.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210234923.8A CN115964337A (en) | 2022-03-10 | 2022-03-10 | Eight-path rack server based on access heat analysis file storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210234923.8A CN115964337A (en) | 2022-03-10 | 2022-03-10 | Eight-path rack server based on access heat analysis file storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115964337A true CN115964337A (en) | 2023-04-14 |
Family
ID=87351599
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210234923.8A Pending CN115964337A (en) | 2022-03-10 | 2022-03-10 | Eight-path rack server based on access heat analysis file storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115964337A (en) |
-
2022
- 2022-03-10 CN CN202210234923.8A patent/CN115964337A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111881133B (en) | Storage bucket management method and device, computer equipment and readable storage medium | |
| CN110018788A (en) | It is classified storage method, device, electronic equipment and computer readable storage medium | |
| CN107249035B (en) | Shared repeated data storage and reading method with dynamically variable levels | |
| EP4020153A1 (en) | Cache space management method and device | |
| CN111858403B (en) | Cache data heat management method and system based on probability to access frequency counting | |
| CN114564457B (en) | Storage space optimization method and system for database files | |
| CN108628769A (en) | A kind of cache allocation method and equipment | |
| CN106933868A (en) | A kind of method and data server for adjusting data fragmentation distribution | |
| CN111737168A (en) | Cache system, cache processing method, device, equipment and medium | |
| CN108846114A (en) | Distributed system control method, device, equipment and readable storage medium storing program for executing | |
| US20230009375A1 (en) | Data prefetching method and apparatus, and storage device | |
| CN109582649A (en) | A kind of metadata storing method, device, equipment and readable storage medium storing program for executing | |
| CN112148690A (en) | File caching method, file access request processing method and device | |
| CN116578409A (en) | Method, system and medium for identifying and migrating memory hot page | |
| CN107025223A (en) | A kind of buffer management method and server towards multi-tenant | |
| CN116700634B (en) | Garbage recycling method and device for distributed storage system and distributed storage system | |
| CN111427920B (en) | Data acquisition method, device, system, computer equipment and storage medium | |
| CN110413226B (en) | Storage pooling capacity utilization method, system, electronic equipment and storage medium | |
| CN115964337A (en) | Eight-path rack server based on access heat analysis file storage method | |
| CN108959614A (en) | A kind of snapshot management method, system, device, equipment and readable storage medium storing program for executing | |
| CN111913913A (en) | Access request processing method and device | |
| CN116027982A (en) | Data processing method, device and readable storage medium | |
| CN112579576B (en) | Data processing method, device, medium and computing equipment | |
| Ha et al. | Dynamic hot data identification using a stack distance approximation | |
| CN114579061A (en) | Data storage method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |