CN111813740A - File layered storage method and server - Google Patents
File layered storage method and server Download PDFInfo
- Publication number
- CN111813740A CN111813740A CN201910291050.2A CN201910291050A CN111813740A CN 111813740 A CN111813740 A CN 111813740A CN 201910291050 A CN201910291050 A CN 201910291050A CN 111813740 A CN111813740 A CN 111813740A
- Authority
- CN
- China
- Prior art keywords
- target file
- access
- file
- storage
- access heat
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000004590 computer program Methods 0.000 claims description 18
- 230000008859 change Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000005012 migration Effects 0.000 description 3
- 238000013508 migration Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000010076 replication Effects 0.000 description 3
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003362 replicative effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000020169 heat generation Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1727—Details of free space management performed by the file system
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a file layered storage method, which is used for solving the problems that in the prior art, the cold and hot files are roughly judged, the cold and hot degrees of the files cannot be truly reflected, the files are inaccurately stored in a layered mode, storage equipment with different performances cannot be reasonably distributed and effectively utilized, and the practicability is low. The method comprises the following steps: determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file; and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file. The application also discloses a server.
Description
Technical Field
The embodiment of the invention relates to the technical field of storage, in particular to a file layered storage method and a server.
Background
With the development of society, the amount of data required to be stored by each enterprise is larger and larger, and the types of the stored data are more and more. Usually, each enterprise will purchase other storage devices to achieve the purpose of capacity expansion urgently, and since the performances of the purchased storage devices are not uniform, how to reasonably allocate and effectively utilize the storage devices with different performances is very important.
At present, storage equipment with different performances is reasonably distributed and effectively utilized by judging the cold and hot degree of files of an enterprise system and then storing the judged cold and hot files in a layered manner. However, in the existing file hierarchical storage method, the determination of the cold and hot degrees of the file is based on the last access time, which specifically includes: if the last access time atime of the file a is 201805031400, if the file hierarchy is scheduled to be executed at 201805031500, the cold-hot degree of the file a is 60; if the last access time atime of the file b is 201805031000, the file b is 300 when the file hierarchy is scheduled to be executed at 201805031500. It can be seen that the method of calculating the time since the last access to a file is: the time of execution of the hierarchical command minus the last access time atime of the file. However, this method of determining the degree of cooling and heating of the document has the following disadvantages:
first, if some files are open before the hierarchical command is executed, the time since the last access is one day, even though the files may be cold files that were inactive the previous month. In the process of storing and layering the files, the cold files are obviously judged as hot files and are not stored in low-performance storage equipment, so that a large amount of data corresponding to the files which are not frequently used continuously occupy the high-performance storage equipment.
Second, if some files are subject to a large number of read/write (I/O) operations at certain fixed times of the week/month, obviously, these files are hot files. However, designating files that were not active 3 days ago as cold files when these file storage hierarchies were made may re-migrate these files to low performance storage devices.
Therefore, the file layered storage method in the prior art roughly judges cold and hot files, cannot truly reflect the cold and hot degrees of the files, causes inaccurate file layered storage, cannot reasonably distribute and effectively utilize storage devices with different performances, and has low practicability.
Disclosure of Invention
The embodiment of the invention provides a file layered storage method and a server, which are used for solving the problems that in the prior art, the cold and hot files are roughly judged, the cold and hot degrees of the files cannot be truly reflected, the file layered storage is inaccurate, storage devices with different performances cannot be reasonably distributed and effectively utilized, and the practicability is low.
The embodiment of the invention adopts the following technical scheme:
in a first aspect, a file hierarchical storage method is provided, where the method includes:
determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file;
and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file.
In a second aspect, a server is provided, the server comprising:
the first determining module is used for determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current moment and the average access proportion of the target file;
and the storage module is used for storing the target file on a storage level matched with the access heat degree based on the access heat degree of the target file.
In a third aspect, a server is provided, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor performing the operations of:
determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file;
and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file.
In a fourth aspect, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the operations of:
determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file;
and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file.
The embodiment of the invention adopts at least one technical scheme which can achieve the following beneficial effects:
according to the file layered storage method provided by the embodiment of the invention, the access heat of the target file is determined based on the average access frequency of the target file in the preset time period before the current moment and the average access proportion of the target file, so that the access cold and hot degree of the target file can be truly and accurately reflected, and a basis is provided for layered storage of the target file; based on the access heat of the target file, the target file is stored on a storage level matched with the access heat, and layered storage is performed based on the access heat of the target file, so that storage devices with different performances can be reasonably distributed and effectively utilized, and the practicability is high.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic flowchart of a file hierarchical storage method according to an embodiment of the present specification;
fig. 2 is one of schematic diagrams of practical application scenarios of a file hierarchical storage method provided in an embodiment of the present specification;
fig. 3 is a second schematic view of an actual application scenario of the file hierarchical storage method according to an embodiment of the present disclosure;
fig. 4 is a third schematic view of a practical application scenario of the file hierarchical storage method according to an embodiment of the present disclosure;
fig. 5 is a fourth schematic view of an actual application scenario of the file hierarchical storage method according to an embodiment of the present specification;
FIG. 6 is a block diagram of a server according to an embodiment of the present disclosure;
fig. 7 is a second schematic diagram of a server structure according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be clearly and completely described below with reference to the specific embodiments of the present specification and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step are within the scope of the present application.
The embodiment of the invention provides a file layered storage method and a server, and aims to solve the problems that in the prior art, cold and hot files are roughly judged, the cold and hot degrees of the files cannot be truly reflected, the files are inaccurately stored in a layered mode, storage devices with different performances cannot be reasonably distributed and effectively utilized, and the practicability is low. Embodiments of the present invention provide a file hierarchical storage method, where an execution subject of the method may be, but is not limited to, an application program, a server, or an apparatus or system capable of being configured to execute the method provided by the embodiments of the present invention.
For convenience of description, the following description will be made of an embodiment of the method, taking an execution subject of the method as a server capable of executing the method as an example. It is understood that the implementation of the method by the server is merely an exemplary illustration and should not be construed as a limitation of the method.
Fig. 1 is a flowchart of a file hierarchical storage method according to an embodiment of the present invention, where the method in fig. 1 may be executed by a server, and as shown in fig. 1, the method may include:
The predetermined time period before the current time may be set according to actual requirements, and the embodiment of the present invention is not particularly limited. The current time may be a user-defined time, and the predetermined period of time may be in units of "days".
For example, the predetermined period of time may be a unit of time, such as 1 day; alternatively, the predetermined period of time may be 0.5 times the unit time, such as 0.5 days; still alternatively, the predetermined period of time may be 5 times the unit time, such as 5 days.
The average access frequency may refer to the total number of times of reading and writing the file in a unit time, i.e., the total number of times of reading and writing the file is divided by the unit time.
Illustratively, there are 20 read/write (I/O) requests for a file within 2 days, with the file having an average access frequency of 20 ÷ 2 days ═ 10;
the average access ratio may refer to the total size of the file read or the total size of the file write in a unit time, which is the ratio of the total size of the file read/write in a unit time divided by the size of the total file.
Illustratively, if the total size of a file is 2MB, and the file has been completely read or written 15 times over the last three days, the average access ratio is 15 × 2 MB/3 days 5.
The access heat degree can be divided into three levels of cold, warm and hot, and of course, can also be divided into a plurality of levels, and the embodiments of the present invention are not listed.
The steps can be realized as follows:
if the average access frequency is greater than or equal to a first threshold value and the average access proportion is greater than or equal to a second threshold value, determining that the access heat of the target file is hot;
if the average access frequency is greater than or equal to a first threshold value and the average access proportion is smaller than a second threshold value, determining that the access heat of the target file is temperature;
if the average access frequency is smaller than a first threshold value and the average access proportion is larger than or equal to a second threshold value, determining the access heat of the target file as temperature;
and if the average access frequency is smaller than a first threshold value and the average access proportion is smaller than a second threshold value, determining that the access heat of the target file is cold.
The first threshold and the second threshold can be set by a user according to the actual specific environment and requirements.
For example, assuming that the first threshold is a and the second threshold is b, the specific implementation of this step may be:
if the average access frequency > of the file is a and the average access proportion > of the file is b, the access heat of the file is hot;
if the average access frequency of the file is a and the average access proportion of the file is < b or the average access frequency of the file is < a and the average access proportion of the file is b, the access heat of the file is temperature;
if the average access frequency of the file is < a and the average access proportion of the file is < b, the access heat of the file is cold.
In practical applications, for example: data cold and hot layering is performed on the app file system once a week, a reference value a of the average access frequency of the file is 10, a reference value b of the average access proportion of the file is 50, the unit time is set to be 7 days, the size of the file alpha, the size of the file beta, the size of the file gamma, the number of times of complete reading in 7 days, the number of times of partial reading, the total reading amount and the access heat judgment of the corresponding file are specified, and as shown in table 1:
TABLE 1
And 102, storing the target file on a storage level matched with the access heat degree based on the access heat degree of the target file.
Assuming that there may be 3 or more storage devices in a system, and the storage performances of these storage devices are different, the IOPS is generally used as a criterion for determining the storage performance, and the larger the IOPS value is, the better the storage performance is. For example,
the IOPS is higher than or equal to 100 ten thousand times and is high-end storage equipment which is used for storing hot data, and then files with hot access degrees are stored on the high-end storage equipment;
the medium-end storage equipment with the IOPS of more than 100 ten thousand times and more than 10 ten thousand times is used for storing temperature data, and files with the access heat of temperature are stored on the medium-end storage equipment;
IOPS <10 ten thousand times is a low-end storage device for storing cold data, and files with cold access degrees are stored on the low-end storage device.
Illustratively, a storage device 1, a storage device 2 and a storage device 3 are arranged on a certain app system of the host. The IOPS of storage device 1, storage device 2, and storage device 3 are 2000000, 500000, 80000, respectively.
Then, after the access heat of the object file is confirmed, the object file with the hot access heat is stored in the storage device 1, the object file with the warm access heat is stored in the storage device 2, and the object file with the cold access heat is stored in the storage device 3.
According to the file layered storage method provided by the embodiment of the invention, the access heat of the target file is determined based on the average access frequency of the target file in the preset time period before the current moment and the average access proportion of the target file, so that the access cold and hot degree of the target file can be truly and accurately reflected, and a basis is provided for layered storage of the target file; based on the access heat of the target file, the target file is stored on a storage level matched with the access heat, and layered storage is performed based on the access heat of the target file, so that storage devices with different performances can be reasonably distributed and effectively utilized, and the practicability is high.
As an embodiment, step 102 may be specifically implemented as:
determining the current storage level of the target file based on the access heat of the target file;
judging whether the current storage level is the same as the original storage level of the target file or not;
if not, executing the operation of migrating and storing the target file from the original storage level to the current storage level;
if so, the migration operation does not need to be executed.
For example, following the above example, suppose that there are storage device 1(IOPS 2000000), storage device 2(IOPS 500000), and storage device 3(IOPS 80000) on a certain app system of the host, and file α, file β, and file γ are all currently stored on storage 1 (as shown in fig. 2).
According to this step can be realized as: determining the levels of the files alpha, beta and gamma according to the access heat of the files alpha, beta and gamma, judging whether the original storage levels of the files alpha, beta and gamma are the same according to the levels of the files alpha, beta and gamma, and if so, not migrating the files alpha, beta and gamma; if not, the file α, the file β, and the file γ need to be migrated, as shown in table 2 and table 3:
TABLE 2
| Filename | Access heat of file | Original storage hierarchy | Whether to migrate | The hierarchy of the genus |
| α | Heat generation | Storage device 1 | Whether or not | Storage device 1 |
| β | Temperature of | Storage device 1 | Is that | Storage device 2 |
| γ | Cold | Storage device 1 | Is that | Storage device 3 |
TABLE 3
| Filename | Addressing | Data block | Addressing | Data block | Addressing | Data block | Addressing | Data | |
| α | 0x1 | ||||||||
| 00D | 0x2 | E05 | 0x3 | A32 | 0x4 | | |||
| β | 0x5 | ||||||||
| 00D | 0x6 | E05 | 0x7 | DD3 | 0x8 | A38 | |||
| γ | 0x9 | E05 | | DD3 | 0x11 | 32F | …… | …… |
As an embodiment, the executing the operation of migrating the target file from the original storage hierarchy to the current storage hierarchy may specifically be implemented as:
synchronizing data blocks of the target file on the original storage level on the current storage level;
and changing a file pointer of the target file to enable the file pointer to point to an address corresponding to the data block of the current storage level.
Illustratively, taking file β as an example: through the copying and synchronization of the data blocks, the data blocks of the same beta file on the storage device 2 (simplified to storage 2) as on the storage device 1 (simplified to storage 1) are created, and after the creation is completed, the file pointer of the beta file is changed, that is, the migration is completed, as shown in fig. 3.
As an embodiment, before synchronizing the data block of the target file on the original storage hierarchy on the current storage hierarchy, the method includes:
determining the access heat degree change condition of the target file;
if the access heat of the target file becomes small (for example, the access heat is changed from hot to warm, or the access heat is changed from warm to cold, or the access heat is changed from hot to cold), the synchronizing the data blocks of the target file on the original storage hierarchy on the current storage hierarchy includes:
determining whether the data blocks of the target file on the current storage level are the same as the data blocks on the original storage level;
if so, changing a file pointer of the target file to enable the file pointer to point to an address corresponding to the data block of the current storage level;
and if not, synchronously copying the data block of the target file on the original storage level on the current storage level.
Illustratively, following the above example, if the access heat of the target file γ becomes small, the target file γ migrates from the storage device 1 (storage 1 for short) to the storage device 3 (storage 3 for short). Then it needs to determine whether the data block of the target file γ at the belonged level (i.e. storage 3) is the same as the data block at the original storage level (i.e. storage 1), such as the table of storage 1 and storage 3 in fig. 3, and determine that the block data E05 and the data block DD3 of the target file γ already exist at storage 3, so it only needs to change the file pointer of the target file γ; it is determined that data block 32F of destination file γ does not have a corresponding data block on store 3, so the data block is synchronized and the file pointer of destination file γ is changed when replication is needed.
According to the embodiment of the invention, by determining the change condition of the access heat of the target file, if the access heat of the target file is reduced, whether the data block of the target file on the current storage level is the same as the data block on the original storage level is determined, and if the access heat of the target file is reduced, the file pointer of the target file is changed to point to the address corresponding to the data block of the current storage level, so that the target file is migrated and compressed, and the target file with cold access heat is stored on the storage equipment with low performance requirements in a compressed manner, so that the available space of the storage equipment is improved, the utilization rate of the storage space is improved, and the cost is reduced.
As an embodiment, before synchronizing the data block of the target file on the original storage hierarchy on the current storage hierarchy, the method includes:
determining the access heat degree change condition of the target file;
and if the access heat of the target file becomes large (for example, the access heat is changed from cold to warm, or the access heat is changed from warm to warm, or the access heat is changed from cold to warm), synchronously copying the data block of the target file on the original storage level on the current storage level. The concrete implementation is as follows:
illustratively, following the above example, if the access heat of the target file γ becomes large, the target file γ is migrated from the storage device 3 (storage 3 for short) to the storage device 2 (storage 2 for short), as shown in fig. 5. Then, only the data block identical to that in storage 3 needs to be created in storage 2 according to the address corresponding to the database pointed by the file pointer of the target file. Wherein the file addressing is the actual location of the data block.
According to the embodiment of the invention, by determining the change condition of the access heat degree of the target file, if the access heat degree of the target file is increased, the data block of the target file on the original storage hierarchy is synchronized on the current storage hierarchy, the file pointer is changed to point to the address corresponding to the data block of the current storage hierarchy, so that the target file is migrated and decompressed, the target file with the hot access heat degree is decompressed and stored on the storage device with high performance requirement, the read-write performance of the target file with the hot access heat degree is ensured, and the normal high-speed operation of the service is ensured.
The above description section introduces an embodiment of a file hierarchical storage method in detail, as shown in fig. 6, and the present specification further provides a server, as shown in fig. 6, where the server 600 includes:
a determining module 601, configured to determine an access heat of a target file based on an average access frequency of the target file in a predetermined time period before a current time and an average access proportion of the target file;
a storage module 602, configured to store the target file on a storage hierarchy matched with the access heat based on the access heat of the target file.
Optionally, as an embodiment, the determining module 601 includes:
a first determining unit, configured to determine that the access heat of the target file is hot if the average access frequency is greater than or equal to a first threshold and the average access ratio is greater than or equal to a second threshold;
a second determining unit, configured to determine that the access heat of the target file is warm if the average access frequency is greater than or equal to a first threshold and the average access proportion is smaller than a second threshold;
a third determining unit, configured to determine that the access heat of the target file is warm if the average access frequency is less than a first threshold and the average access ratio is greater than or equal to a second threshold;
and the fourth determining unit is used for determining that the access heat of the target file is cold if the average access frequency is less than a first threshold and the average access proportion is less than a second threshold.
Optionally, as an embodiment, the storage module 602 includes:
a fifth determining unit, configured to determine, based on the access heat of the target file, a storage tier to which the target file belongs;
the judging unit is used for judging whether the current storage level is the same as the original storage level of the target file or not;
and the first execution unit is used for executing the operation of migrating and storing the target file from the original storage hierarchy to the current storage hierarchy if the current storage hierarchy to which the target file belongs is different from the original storage hierarchy of the target file.
Optionally, as an embodiment, the storage module 602 includes:
and the second execution unit is used for not executing the migration operation if the current storage level to which the target file belongs is the same as the original storage level of the target file.
Optionally, as an embodiment, the first execution unit includes:
a synchronization subunit, configured to synchronize, on the current storage hierarchy to which the target file belongs, a data block of the target file on the original storage hierarchy;
and the changing subunit is used for changing the file pointer of the target file to enable the file pointer to point to the address corresponding to the data block of the current storage hierarchy.
Optionally, as an embodiment, the first execution unit includes:
the first determining subunit is used for determining the access heat degree change condition of the target file;
if the access heat of the target file becomes smaller, the synchronization subunit includes:
the second determining subunit is used for determining whether the data block of the target file on the current storage hierarchy is the same as the data block on the original storage hierarchy;
and the changing subunit is configured to, if it is determined that the data block of the target file on the current storage hierarchy is the same as the data block on the original storage hierarchy, change the file pointer of the target file so that the file pointer points to an address corresponding to the data block of the current storage hierarchy.
Optionally, as an embodiment, if the access heat of the target file becomes large, the synchronization subunit includes:
and the first synchronous replication sub-unit is used for synchronously replicating the data blocks of the target file on the original storage hierarchy on the current storage hierarchy.
Optionally, as an embodiment, if the access heat of the target file is changed from high to low, the synchronization subunit includes:
and the second synchronous replication sub-unit is used for synchronously replicating the data block of the target file on the original storage hierarchy on the current storage hierarchy if the data block of the target file on the current storage hierarchy is determined to be different from the data block on the original storage hierarchy.
According to the file layered storage method provided by the embodiment of the invention, the access heat of the target file is determined based on the average access frequency of the target file in the preset time period before the current moment and the average access proportion of the target file, so that the access cold and hot degree of the target file can be truly and accurately reflected, and a basis is provided for layered storage of the target file; based on the access heat of the target file, the target file is stored on a storage level matched with the access heat, and layered storage is performed based on the access heat of the target file, so that storage devices with different performances can be reasonably distributed and effectively utilized, and the practicability is high.
A server according to an embodiment of the present invention will be described in detail below with reference to fig. 7. Referring to fig. 7, at the hardware level, the server includes a processor, optionally an internal bus, a network interface, and a memory. As shown in fig. 7, the Memory may include a Memory, such as a Random-Access Memory (RAM), and may also include a non-volatile Memory, such as at least 1 disk Memory. Of course, the server may also include the hardware needed to implement other services.
The processor, the network interface, and the memory may be interconnected by an internal bus, which may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an extended EISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 7, but this does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form a device for forwarding the chat information on a logic level. The processor executes the program stored in the memory and is specifically configured to perform the operations of the method embodiments described herein.
The method and the method executed by the server according to the embodiments shown in fig. 1 to 6 may be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The server shown in fig. 7 may also execute the methods shown in fig. 1 to fig. 5, and implement the functions of the file hierarchical storage method in the embodiments shown in fig. 1 to fig. 5, which are not described herein again in the embodiments of the present invention.
Of course, besides the software implementation, the server of the present application does not exclude other implementations, such as a logic device or a combination of software and hardware, and the like, that is, the execution main body of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the processes of the method embodiments, and can achieve the same technical effects, and in order to avoid repetition, the details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A file hierarchical storage method is characterized by comprising the following steps:
determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file;
and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file.
2. The method of claim 1,
the determining the access heat of the target file based on the average access frequency of the target file in a predetermined time period before the current time and the average access proportion of the target file comprises:
if the average access frequency is greater than or equal to a first threshold value and the average access proportion is greater than or equal to a second threshold value, determining that the access heat of the target file is hot;
if the average access frequency is greater than or equal to a first threshold value and the average access proportion is smaller than a second threshold value, determining that the access heat of the target file is temperature;
if the average access frequency is smaller than a first threshold value and the average access proportion is larger than or equal to a second threshold value, determining the access heat of the target file as temperature;
and if the average access frequency is smaller than a first threshold value and the average access proportion is smaller than a second threshold value, determining that the access heat of the target file is cold.
3. The method according to claim 2, wherein the storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file comprises:
determining the current storage level of the target file based on the access heat of the target file;
judging whether the level to which the target file belongs is the same as the original storage level of the target file or not;
if not, the operation of migrating and storing the target file from the original storage level to the current storage level is executed.
4. The method of claim 3, wherein the performing the operation of migrating the target file from the original storage hierarchy to the current storage hierarchy comprises:
synchronizing data blocks of the target file on the original storage level on the current storage level;
and changing a file pointer of the target file to enable the file pointer to point to an address corresponding to the data block of the current storage level.
5. The method of claim 4, wherein before synchronizing the data blocks of the target file on the original storage hierarchy on the current storage hierarchy, the method comprises:
determining the access heat degree change condition of the target file;
if the access heat of the target file is increased, synchronizing the data block of the target file on the original storage hierarchy on the current storage hierarchy, including:
synchronously copying the data blocks of the target file on the original storage level on the current storage level.
6. The method of claim 5,
if the access heat of the target file is reduced, synchronizing the data block of the target file on the original storage hierarchy on the current storage hierarchy, including:
determining whether the data blocks of the target file on the current storage level are the same as the data blocks on the original storage level;
and if so, changing a file pointer of the target file to enable the file pointer to point to an address corresponding to the data block of the current storage level.
7. The method of claim 6,
if the access heat of the target file is reduced, synchronizing the data block of the target file on the original storage hierarchy on the current storage hierarchy, including:
and if not, synchronously copying the data block of the target file on the original storage level on the current storage level.
8. A server, comprising:
the determining module is used for determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current moment and the average access proportion of the target file;
and the storage module is used for storing the target file on a storage level matched with the access heat degree based on the access heat degree of the target file.
9. A server, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of:
determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file;
and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file.
10. A computer-readable storage medium having a computer program stored thereon, which when executed by a processor, performs the steps of:
determining the access heat of the target file based on the average access frequency of the target file in a preset time period before the current time and the average access proportion of the target file;
and storing the target file on a storage hierarchy matched with the access heat degree based on the access heat degree of the target file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910291050.2A CN111813740A (en) | 2019-04-11 | 2019-04-11 | File layered storage method and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910291050.2A CN111813740A (en) | 2019-04-11 | 2019-04-11 | File layered storage method and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111813740A true CN111813740A (en) | 2020-10-23 |
Family
ID=72843855
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910291050.2A Pending CN111813740A (en) | 2019-04-11 | 2019-04-11 | File layered storage method and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111813740A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022134345A1 (en) * | 2020-12-25 | 2022-06-30 | 平安科技（深圳）有限公司 | File access method, apparatus, device, and readable storage medium |
| CN115543930A (en) * | 2022-03-01 | 2022-12-30 | 荣耀终端有限公司 | Method, device and related equipment for locking file in memory |
| CN117236236A (en) * | 2023-11-10 | 2023-12-15 | 杭州行芯科技有限公司 | Chip design data management method and device, electronic equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521260A (en) * | 2011-11-18 | 2012-06-27 | 华为技术有限公司 | Data preheating method and device |
| CN102523279A (en) * | 2011-12-12 | 2012-06-27 | 云海创想信息技术（无锡）有限公司 | Distributed file system and hot file access method thereof |
| CN103198027A (en) * | 2013-02-27 | 2013-07-10 | 天脉聚源(北京)传媒科技有限公司 | Method and device for storing and providing files |
| CN104199784A (en) * | 2014-08-20 | 2014-12-10 | 浪潮(北京)电子信息产业有限公司 | Data migration method and device based on hierarchical storage |
| CN105653524A (en) * | 2014-11-10 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Data storage method, device and system |
| CN105763508A (en) * | 2014-12-16 | 2016-07-13 | 阿里巴巴集团控股有限公司 | Data access method and application server |
| CN106502576A (en) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | Migration strategy method of adjustment, capacity change suggesting method and device |
| CN107844269A (en) * | 2017-10-17 | 2018-03-27 | 华中科技大学 | A kind of layering mixing storage system and method based on uniformity Hash |
| CN109144411A (en) * | 2018-07-24 | 2019-01-04 | 中国电子科技集团公司第三十八研究所 | Data center's hybrid magnetic disc array and its data dynamic migration strategy |
| CN109344092A (en) * | 2018-09-11 | 2019-02-15 | 天津易华录信息技术有限公司 | A kind of method and system improving cold storing data reading speed |
-
2019
- 2019-04-11 CN CN201910291050.2A patent/CN111813740A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521260A (en) * | 2011-11-18 | 2012-06-27 | 华为技术有限公司 | Data preheating method and device |
| CN102523279A (en) * | 2011-12-12 | 2012-06-27 | 云海创想信息技术（无锡）有限公司 | Distributed file system and hot file access method thereof |
| CN103198027A (en) * | 2013-02-27 | 2013-07-10 | 天脉聚源(北京)传媒科技有限公司 | Method and device for storing and providing files |
| CN104199784A (en) * | 2014-08-20 | 2014-12-10 | 浪潮(北京)电子信息产业有限公司 | Data migration method and device based on hierarchical storage |
| CN105653524A (en) * | 2014-11-10 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Data storage method, device and system |
| CN105763508A (en) * | 2014-12-16 | 2016-07-13 | 阿里巴巴集团控股有限公司 | Data access method and application server |
| CN106502576A (en) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | Migration strategy method of adjustment, capacity change suggesting method and device |
| CN107844269A (en) * | 2017-10-17 | 2018-03-27 | 华中科技大学 | A kind of layering mixing storage system and method based on uniformity Hash |
| CN109144411A (en) * | 2018-07-24 | 2019-01-04 | 中国电子科技集团公司第三十八研究所 | Data center's hybrid magnetic disc array and its data dynamic migration strategy |
| CN109344092A (en) * | 2018-09-11 | 2019-02-15 | 天津易华录信息技术有限公司 | A kind of method and system improving cold storing data reading speed |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022134345A1 (en) * | 2020-12-25 | 2022-06-30 | 平安科技（深圳）有限公司 | File access method, apparatus, device, and readable storage medium |
| CN115543930A (en) * | 2022-03-01 | 2022-12-30 | 荣耀终端有限公司 | Method, device and related equipment for locking file in memory |
| CN115543930B (en) * | 2022-03-01 | 2023-10-24 | 荣耀终端有限公司 | Method, device and related equipment for locking file in memory |
| CN117236236A (en) * | 2023-11-10 | 2023-12-15 | 杭州行芯科技有限公司 | Chip design data management method and device, electronic equipment and storage medium |
| CN117236236B (en) * | 2023-11-10 | 2024-04-16 | 杭州行芯科技有限公司 | Chip design data management method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107450979B (en) | Block chain consensus method and device | |
| CN107391526B (en) | Data processing method and device based on block chain | |
| CN104679661B (en) | hybrid storage control method and hybrid storage system | |
| US9298389B2 (en) | Operating a memory management controller | |
| US8856440B2 (en) | Volatile memory representation of nonvolatile storage device set | |
| CN110879800B (en) | Data writing, compressing and reading method, data processing method and device | |
| CN111813740A (en) | File layered storage method and server | |
| CN108628688B (en) | Message processing method, device and equipment | |
| US10628200B2 (en) | Base state for thin-provisioned volumes | |
| US20160274819A1 (en) | Memory system including host and a plurality of storage devices and data migration method thereof | |
| CN109669640A (en) | A kind of date storage method, device, electronic equipment and medium | |
| US20140280397A1 (en) | Heterogenic volume generation and use system | |
| CN111708787A (en) | Multi-center service data management system | |
| WO2024082787A1 (en) | Method and apparatus for creating tablespace, storage medium, and electronic device | |
| CN107632779B (en) | Data processing method and device and server | |
| CN111158858B (en) | Cloning method and device of virtual machine and computer readable storage medium | |
| CN110955720A (en) | Data loading method, device and system | |
| CN113434470B (en) | Data distribution method and device and electronic equipment | |
| CN114442931A (en) | Data deduplication method and system, electronic device and storage medium | |
| CN111475099A (en) | Data storage method, device and equipment | |
| CN111078127A (en) | Data migration method, system and device | |
| US10901972B2 (en) | Table partition configuration method, apparatus and system for database system | |
| US11200210B2 (en) | Method of efficient backup of distributed file system files with transparent data access | |
| CN107102898B (en) | Memory management and data structure construction method and device based on NUMA (non Uniform memory Access) architecture | |
| CN109558073A (en) | A kind of disk based on educational system extends the method and electronic equipment in service life |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201023 |