CN112306901B - Disk refreshing method and device based on layered storage system, electronic equipment and medium - Google Patents
Disk refreshing method and device based on layered storage system, electronic equipment and medium Download PDFInfo
- Publication number
- CN112306901B CN112306901B CN202011281870.2A CN202011281870A CN112306901B CN 112306901 B CN112306901 B CN 112306901B CN 202011281870 A CN202011281870 A CN 202011281870A CN 112306901 B CN112306901 B CN 112306901B
- Authority
- CN
- China
- Prior art keywords
- brushing
- disk
- storage system
- service
- hierarchical storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
- G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0893—Caches characterised by their organisation or structure
- G06F12/0897—Caches characterised by their organisation or structure with two or more cache hierarchy levels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The application discloses a method and a device for brushing a disk based on a layered storage system, electronic equipment and a medium. By applying the technical scheme of the application, the service operation parameter of at least one service object associated with the hierarchical storage system can be acquired, the service processing pressure of the hierarchical storage system in a first preset time period is determined based on the service operation parameter of the at least one service object, and if the service processing pressure exceeds a preset threshold value, the first disk brushing grade of the hierarchical storage system is adjusted to the second disk brushing grade. By applying the technical scheme of the application, the service pressure of the storage system in the future time period can be determined by detecting the service operation parameters of the service objects associated with the hierarchical storage system, and the corresponding brushing grade can be dynamically adjusted. Therefore, the condition that the disk refreshing process conflicts with the service processing process in the process of performing disk refreshing in a layered storage mechanism in the related technology is avoided, and the service processing efficiency is influenced.
Description
Technical Field
The present application relates to data communication technologies, and in particular, to a method, an apparatus, an electronic device, and a medium for flushing a disk based on a hierarchical storage system.
Background
Due to the rise of the communication era and society, data storage by using a hierarchical storage mechanism has been continuously applied along with the development of each business platform.
Further, Hierarchical Storage (also called Hierarchical Storage Management) may store data in a cache region and hard disk data at a rear end in different Storage manners according to indexes such as importance, access frequency, retention time, capacity, and performance of the data. For the hierarchical storage mechanism, because the hot spot condition of the business object is constantly changing, for example, some data has been converted from cold data to hot data, some hot data has been converted into cold data, then these data need to be interacted, cache and backend data need to be migrated, and the operation of removing the data stored in the cache area to the backend hard disk data is a disk-flushing operation.
However, in the related art, during the disk-flushing process of the hierarchical storage mechanism, the disk-flushing operation is usually performed by using a fixed disk-flushing manner, and such a manner cannot adapt to a multi-service-state service platform.
Disclosure of Invention
The embodiment of the application provides a disk-refreshing method, a disk-refreshing device, an electronic device and a medium based on a layered storage system, and is used for solving the problem that a disk-refreshing process conflicts with a service processing process in the process of refreshing a layered storage mechanism in the related art.
According to an aspect of an embodiment of the present application, a disk-scrubbing method based on a hierarchical storage system is provided, where the method includes:
acquiring service operation parameters of at least one service object associated with the hierarchical storage system;
determining the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of the at least one business object;
and if the service processing pressure in the first preset time period is determined to exceed a preset threshold value, adjusting the first disk brushing grade of the hierarchical storage system to a second disk brushing grade, wherein the disk brushing grade corresponds to a disk brushing frequency and a disk brushing force.
Optionally, in another embodiment based on the foregoing method of the present application, the adjusting the first disk-brushing level of the hierarchical storage system to the second disk-brushing level includes:
Reducing a first disk-brushing frequency of the hierarchical storage system to a second disk-brushing frequency;
and increasing the first disk brushing force of the layered storage system to a second disk brushing force.
Optionally, in another embodiment based on the foregoing method of the present application, the increasing the first disk-brushing strength of the hierarchical storage system to the second disk-brushing strength includes:
acquiring a first brushing data quantity under the first brushing grade;
determining a ratio of the first disk refreshing data quantity to a total data quantity, wherein the total data quantity is a data quantity located in a cache region in the hierarchical storage system;
determining a second brushing data quantity under a second brushing grade based on the ratio of the first brushing data quantity to the total data quantity;
and determining the second disc brushing force according to the second disc brushing data quantity.
Optionally, in another embodiment based on the foregoing method of the present application, the determining the second brushing force according to the second brushing data amount includes:
acquiring a first disc brushing time length under the first disc brushing grade, and reducing the first disc brushing time length to a second disc brushing time length;
and determining the second disk brushing force based on the second disk brushing data quantity and the second disk brushing time length.
Optionally, in another embodiment based on the foregoing method of the present application, the determining, based on the service operation parameter of the at least one service object, a service processing pressure of the hierarchical storage system in a first preset time period includes:
acquiring the quantity and the type of service requests generated by the at least one service object in the hierarchical storage system within a second preset time period;
determining the quantity of data to be accessed by the at least one business object in the first preset time period based on the quantity of the business requests and the type of the business requests;
and determining the service processing pressure of the hierarchical storage system in the first preset time period based on the matching relation between the data quantity to be accessed by the at least one service object and the residual storage capacity of the cache region in the hierarchical storage system.
Optionally, in another embodiment based on the foregoing method of the present application, after the adjusting the first disk-brushing level of the hierarchical storage system to the second disk-brushing level, the method further includes:
and if the service processing pressure in a third preset time period is detected not to exceed the preset threshold value, adjusting the second disk brushing grade of the hierarchical storage system to the first disk brushing grade.
Optionally, in another embodiment based on the foregoing method of the present application, before the obtaining the service operation parameter of at least one service object associated with the hierarchical storage system, the method further includes:
determining that a number of business objects associated with the tiered storage system exceeds a first number;
and/or the presence of a gas in the atmosphere,
and determining the current time period as the traffic peak time period of the hierarchical storage system based on the historical processing traffic information.
According to an aspect of the embodiments of the present application, a disk-brushing method and apparatus based on a hierarchical storage system are provided, where the method includes:
an acquisition module configured to acquire a business operation parameter of at least one business object associated with the hierarchical storage system;
the determining module is configured to determine the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of the at least one business object;
an adjusting module configured to adjust a first disk brushing level of the hierarchical storage system to a second disk brushing level if it is determined that the service processing pressure within the first preset time period exceeds a preset threshold, where the disk brushing level corresponds to a disk brushing frequency and a disk brushing force.
According to another aspect of the embodiments of the present application, there is provided an electronic device including:
a memory for storing executable instructions; and
a display for displaying with the memory to execute the executable instructions to complete the operation of any of the above hierarchical storage system based disk refreshing methods.
According to a further aspect of the embodiments of the present application, there is provided a computer-readable storage medium for storing computer-readable instructions, which, when executed, perform the operations of any one of the above-mentioned hierarchical storage system-based disk brushing methods.
In the application, a service operation parameter of at least one service object associated with the hierarchical storage system may be acquired, and based on the service operation parameter of the at least one service object, a service processing pressure of the hierarchical storage system within a first preset time period may be determined, and if it is determined that the service processing pressure within the first preset time period exceeds a preset threshold, a first disk-brushing level of the hierarchical storage system may be adjusted to a second disk-brushing level, where the disk-brushing level corresponds to a disk-brushing frequency and a disk-brushing force. By applying the technical scheme of the application, the service pressure of the storage system in the future time period can be determined by detecting the service operation parameters of the service objects associated with the hierarchical storage system, and the corresponding brushing grade can be dynamically adjusted. Therefore, the condition that the disk refreshing process conflicts with the service processing process in the process of performing disk refreshing in a layered storage mechanism in the related technology is avoided, and the service processing efficiency is influenced.
The technical solution of the present application is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.
The present application may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
FIG. 1 is a system architecture diagram of a hierarchical storage mechanism proposed in the present application;
FIG. 2 is a schematic diagram of a disk-refreshing method based on a layered storage system according to the present application;
FIG. 3 is a schematic diagram of another disk-refreshing method based on a hierarchical storage system according to the present application;
fig. 4 is a flowchart of a disk-refreshing method based on a layered storage system according to the present application;
FIG. 5 is a schematic structural diagram of an electronic device for a disk refreshing method based on a hierarchical storage system according to the present application;
fig. 6 is a schematic view of an electronic device according to the present application.
Detailed Description
Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present application unless specifically stated otherwise.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
In addition, technical solutions in the embodiments of the present application may be combined with each other, but it is necessary to be based on the realization of the technical solutions by a person skilled in the art, and when the technical solutions are contradictory to each other or cannot be realized, such a combination of technical solutions should not be considered to exist, and is not within the protection scope claimed in the present application.
It should be noted that all the directional indicators (such as upper, lower, left, right, front and rear … …) in the embodiment of the present application are only used to explain the relative position relationship between the components, the motion situation, etc. in a specific posture (as shown in the drawings), and if the specific posture is changed, the directional indicator is changed accordingly.
A method for performing a disk scrubbing based on a tiered storage system according to an exemplary embodiment of the present application is described below in conjunction with fig. 1-4. It should be noted that the following application scenarios are merely illustrated for facilitating understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in any way in this respect. Rather, embodiments of the present application may be applied to any scenario where applicable.
FIG. 1 shows a schematic diagram of a system architecture 100 to which the hierarchical storage mechanism of the embodiments of the present application may be applied.
As can be seen from fig. 1, in the system architecture of the tiered storage mechanism, at least a plurality of business objects that interact with the tiered storage system may be included. For example, it may be a client, a service server, etc. Further, for a tiered storage system, it includes at least one tier of cache memory and at least one tier of readable storage media. The readable storage medium includes at least one of a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In an embodiment, the storage areas with different storage performances may be storage media with different storage performances, that is, the storage may include storage media with different storage performances such as SSD and SATA. In another embodiment, the plurality of storage areas of different storage capabilities may be storage spaces of different storage capabilities within the same storage medium.
Further, the application also provides a method and a device for refreshing a disk based on the hierarchical storage system, a target terminal and a medium.
Fig. 2 schematically shows a flowchart of a disk brushing method based on a hierarchical storage system according to an embodiment of the present application. As shown in fig. 2, the method includes:
s101, acquiring service operation parameters of at least one service object associated with the hierarchical storage system.
Further, a storage hierarchy is a type of storage architecture that allocates different classes of data to different types of storage regions. The purpose of storage layering is to reduce the storage cost of service operation while meeting performance requirements. The classification of the hierarchy is based primarily on performance requirements, frequency of use, and level of protection required.
Furthermore, most of the existing hierarchical storage is two-layer hierarchical storage, a hard disk with better performance is used as a cache of a data disk, namely, a cache, hot data which is likely to be accessed by a user more is placed on the cache with higher performance, and if IO is hit when the user performs reading and writing (for the write IO, if a corresponding target object exists in the cache, the write is considered to be hit, and for the read IO, if a corresponding object exists in the cache, and all data to be read exists in the object, the read is considered to be hit, and it is noted that the hit is not calculated when only part of the data exists), the write is directly performed on the cache, so that the performance of the whole storage system can be improved. But typically the size of the buffer is much smaller than the size of the back-end data disc, so that the user's data is not always buffered. Meanwhile, the hot spot condition of the user data may be in constant change, some data have been converted from cold data to hot data, some hot data have been converted into cold data, and then the data need to be interacted, and the cache needs to perform data migration with the back-end data, and the operation of the cache to brush the hot spot data down to the back-end data pool is the disk-flushing operation.
However, in the current storage hierarchical system, the same flash disk manner is often adopted to perform hierarchical migration of data. This often results in a delay or an advance of the migration action, which may cause cold data to occupy space in the cache memory for a longer time, resulting in wasted resources. In addition, it may happen that the hot data will be frequently read in the space of the lower-performance storage area, and the working efficiency of the system is reduced. Based on the above problem, the present application can adopt a scheme of dynamically adjusting the disk brushing mode according to the service pressure condition.
The service object is not specifically limited in the present application, and may be, for example, a client, a service server, or the like.
Similarly, the service operation parameters are not specifically limited in this application, and may be, for example, the number of service requests, the service category, the service response time, and the like.
S102, determining the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of at least one business object.
It should be noted that the business object in the present application should be a business object interacting with the hierarchical storage system. For example, taking a service object as a client as an example, when a user uses a service using the client, the user may frequently read data required by the user. Therefore, the client frequently accesses the storage area to read data. When the number of the clients is too large or the number of access times is too large, great business processing pressure is brought to the hierarchical storage system within a period of time.
The first preset time period is not specifically limited, and may be, for example, 1 day, 1 month, or the like.
In addition, as for how to determine the business processing pressure of the hierarchical storage system in the first preset time period based on the business operation parameters of the business object, there may be a variety of ways. For example, the number of service objects, the number of service requests, the type of service request, etc.
It can be understood that when the business processing pressure of the business hierarchical storage system is too large, the cache area of the first storage hierarchy is frequently accessed by the business object to read data. Then if frequent disk-brushing actions also occur at this time, it is likely to affect the progress of the business process. Therefore, in order to avoid conflict between the two, the method and the device can reduce the frequency and the strength of disk refreshing when the service pressure of the hierarchical storage system is overlarge.
And S103, if the service processing pressure in the first preset time period is determined to exceed the preset threshold, adjusting the first disk brushing grade of the hierarchical storage system to be a second disk brushing grade, wherein the disk brushing grade corresponds to the disk brushing frequency and the disk brushing strength.
Furthermore, the problem to be solved in the present application is to avoid adopting the same disk-scrubbing mode to perform hierarchical migration of data. This tends to have a lag or lead in which migration action occurs. Therefore, the corresponding flash disk grade can be dynamically adjusted according to the estimated business processing pressure of the hierarchical storage system.
Wherein the brushing rating may correspond to a brushing frequency and a brushing force. For example, the frequency of swabbing may be increased or decreased depending on the size of the estimated traffic handling pressure of the tiered storage system. And the data amount of the disk brushing of each disk brushing can be increased or decreased according to the estimated business processing pressure of the hierarchical storage system.
For example, in one mode, when it is detected that the service processing pressure of the hierarchical storage system in the first preset time period exceeds a preset threshold, the frequency of flushing the disk of the hierarchical storage system is reduced from three times a day to once a day, so that the problem that the working efficiency of the system is reduced because hot data caused by frequent flushing of the disk is frequently read in a lower-performance storage area space is solved. And the disk brushing force of the layered storage system can be improved from the data volume of 1G for brushing the disk to the data volume of 10G for brushing the disk at one time, and the problem that the working efficiency of the system is reduced because the hot data is frequently read in the space of a storage area with lower performance due to frequent disk brushing can be solved.
In this embodiment, the number of the first brush plate grade and the second brush plate grade is not particularly limited, and in one embodiment, the first brush plate grade and the second brush plate grade may include only one corresponding brush plate grade. In another embodiment, the first brush plate grade and the second brush plate grade may include a plurality of corresponding brush plate grades. For example, for a second brush plate grade, a plurality of second brush plate grades may be included. For example, the adjustment to the corresponding sub-level can be selected according to the level of the business processing pressure.
Furthermore, the first disk-brushing level in the present application corresponds to the current disk-brushing level of the hierarchical storage system. Furthermore, a plurality of brush plate grades corresponding to different brush plate dynamics and brush plate frequency can be contained in the scheme of the application, and the first brush plate grade and the second brush plate grade are only one of the different brush plate grades.
In the application, a service operation parameter of at least one service object associated with the hierarchical storage system may be acquired, and based on the service operation parameter of the at least one service object, a service processing pressure of the hierarchical storage system within a first preset time period may be determined, and if it is determined that the service processing pressure within the first preset time period exceeds a preset threshold, a first disk-brushing level of the hierarchical storage system may be adjusted to a second disk-brushing level, where the disk-brushing level corresponds to a disk-brushing frequency and a disk-brushing force. By applying the technical scheme of the application, the service pressure of the storage system in the future time period can be determined by detecting the service operation parameters of the service objects associated with the hierarchical storage system, and the corresponding brushing grade can be dynamically adjusted. Therefore, the condition that the disk refreshing process conflicts with the service processing process in the process of performing disk refreshing in a layered storage mechanism in the related technology is avoided, and the service processing efficiency is influenced.
Alternatively, in one possible implementation manner of the present application, in S13 (adjusting the first disk-brushing level of the hierarchical storage system to the second disk-brushing level), the following steps may be implemented:
reducing a first disk-brushing frequency of the hierarchical storage system to a second disk-brushing frequency;
and increasing the first disk brushing force of the layered storage system to a second disk brushing force.
As described above, since the high-frequency disk refreshing process inevitably affects the service processing process, when it is determined that the service pressure of the storage system in a future time period is high, the first disk refreshing frequency of the hierarchical storage system may be reduced to the second disk refreshing frequency. For example, the frequency of the disk-flushing of the tiered storage system may be reduced from three times a day to once a day, and so on.
It should be noted that the first brush plate frequency and the second brush plate frequency are not specifically limited in this application, and for example, the second brush plate frequency may be half or one third of the first brush plate frequency.
Similarly, in order to reduce the conflict between the disk-brushing process and the service processing process, the disk-brushing strength of the hierarchical storage system at each time can be improved, specifically, the first disk-brushing strength can be improved to the second disk-brushing strength. For example, the data amount of one-time disk brushing 1G is increased to the data amount of 10G-time disk brushing, so that the problem that the hot data is frequently read in the lower-performance storage area space due to frequent disk brushing, and the working efficiency of the system is reduced can be solved.
It should also be noted that, the first brushing force and the second brushing force are not specifically limited in this application, for example, the second brushing force may be twice, three times, or the like as the first brushing force.
In one mode, the second disk-brushing frequency and the second disk-brushing force may be determined according to a difference between a service pressure of the hierarchical storage system and a preset threshold. It can be understood that, when the difference between the service pressure of the hierarchical storage system and the preset threshold value is larger, the second disk brushing frequency is lower, and the second disk brushing strength is larger.
Optionally, in one possible implementation manner of the present application, at S13 (increasing the first disk-brushing strength of the hierarchical storage system to the second disk-brushing strength), at least any one of the following two manners may be implemented:
the first mode is as follows:
acquiring a first brushing data quantity under a first brushing grade;
determining the ratio of the first disk refreshing data quantity to the total data quantity, wherein the total data quantity is the data quantity located in a cache region in the hierarchical storage system;
determining a second brushing data quantity under a second brushing grade based on the ratio of the first brushing data quantity to the total data quantity;
And determining second brushing force according to the second brushing data quantity.
Further, in the process of determining the second brushing force, the determination can be performed according to the ratio of the first brushing data quantity to the total data quantity. Specifically, the first disk-brushing data amount at the first disk-brushing level and the total data amount currently located in the cache region in the hierarchical storage system may be obtained first. And determining the second brushing data quantity under the second brushing grade according to the ratio of the first brushing grade to the second brushing grade. It is understood that, when the ratio of the two is higher, the total data amount representing the cache region currently located in the hierarchical storage system is relatively not high, and therefore, a relatively smaller second flash data amount may be determined. In other words, when the ratio of the two is lower, the total data amount representing the cache region currently located in the hierarchical storage system is relatively higher, and therefore, a relatively larger second flash data amount can be determined.
For example, when the first disk-brushing data amount at the first disk-brushing level is determined to be 1G, and the total data amount currently located in the cache region in the hierarchical storage system is determined to be 100G, the ratio of the two may be determined to be one percent, that is, the data amount per disk-brushing is too small relative to the total data amount of the current cache region at the first disk-brushing level. Therefore, in the embodiment of the present application, the second brushing data amount at the second brushing level may be determined to be 10G. Thereby achieving the purpose of increasing the disc brushing force.
Or, when the first disk-brushing data amount under the first disk-brushing level is determined to be 2G, and the total data amount currently located in the cache region in the hierarchical storage system is determined to be 10G, the ratio of the two is determined to be one fifth, that is, under the first disk-brushing force, the data amount per disk-brushing is less than the total data amount of the current cache region. Therefore, in the embodiment of the present application, it can be determined that the number of second scrub pad data at the second scrub pad level is increased from 2G to 5G. Thereby achieving the purpose of increasing the disc brushing force.
The second mode is as follows:
acquiring a first disc brushing time length under the first disc brushing grade, and reducing the first disc brushing time length to a second disc brushing time length;
and determining second disk brushing force based on the second disk brushing data quantity and the second disk brushing duration.
Further, in the process of determining the second disc brushing force, the disc brushing force can be determined according to the first disc brushing time. Specifically, the first tray brushing duration at the first tray brushing level may be obtained first, and the tray brushing duration may be reduced to the second duration. It can be understood that the time with different time durations can be selected as the second flash duration according to the magnitude of the business processing pressure.
For example, the first brushing duration at the first brushing level is determined to be 60 minutes, and the first brushing duration may be selected to be reduced to 30 minutes or 20 minutes, etc., depending on, for example, the traffic handling pressure matching a preset threshold. And determining the second brushing force together with the reduced second brushing time length and the second brushing data quantity.
It should be noted that, the first tray brushing time period and the second tray brushing time period are not specifically limited in this application, for example, the second tray brushing time period may be half of the first tray brushing time period, or may be one third of the first tray brushing time period, and the like.
Optionally, in a possible implementation manner of the present application, in S102 (determining a business processing pressure of the hierarchical storage system in the first preset time period based on the business operation parameter of the at least one business object), the following steps may be implemented:
acquiring the quantity and the type of service requests generated by at least one service object in a second preset time period;
determining the quantity of data to be accessed by at least one business object in a first preset time period based on the quantity of the business requests and the type of the business requests;
and determining the business processing pressure of the hierarchical storage system in a first preset time period based on the matching relation between the data quantity required to be accessed by at least one business object and the residual storage capacity of a cache region in the hierarchical storage system.
In one way, in the process of determining the business processing pressure of the hierarchical storage system, one way may be determined based on the number of business requests of the business object and the type of the business requests. It is understood that when the number of service requests of the service object is larger, or the type of the service request is more complex, it can be determined that the number of data (for example, the amount of read data or the amount of stored data, etc.) to be accessed by the plurality of service objects in the future time period is larger. Further, if the business object has a larger amount of data to access the hierarchical storage system in a future time period, it may be determined that the business processing pressure of the hierarchical storage system in the future time period is larger.
Specifically, a threshold comparing the number of service requests may be preset in the embodiment of the present application, for example, when the number of service requests generated by a plurality of service objects within the last 10 minutes is higher than the threshold, the number of service requests representing a future time period is larger. In addition, the service request type generated by the current service object may also be detected, and if the service request type is a data download class or a service data provision class, the number of the service requests may be combined to comprehensively determine the number of data (for example, 100G) that the current multiple service objects need to access in the first preset time period. Further, the 100G may be matched with the current remaining storage capacity data (for example, 105G) of the cache region in the hierarchical storage system, and when the data of the two are found to be close, the business processing pressure of the hierarchical storage system in the first preset time period may be determined, so as to compare the business processing pressure with the preset threshold value subsequently.
The second preset time period is not specifically limited, and may be, for example, 1 hour, 1 day, or the like. In addition, the second preset time period may be the same as or different from the first preset time period.
Further, the present application also provides a disk-brushing method based on a hierarchical storage system, wherein fig. 3 schematically shows a flow chart of a disk-brushing method based on a hierarchical storage system according to an embodiment of the present application. As shown in fig. 3, the method includes:
s201, acquiring service operation parameters of at least one service object associated with the hierarchical storage system.
Optionally, before obtaining the service operation parameters of the service object, the following steps may be further implemented:
determining that a number of business objects associated with the tiered storage system exceeds a first number;
and/or the presence of a gas in the gas,
and determining the current time period as the traffic peak time period of the hierarchical storage system based on the historical processing traffic information.
In order to avoid the problem of unnecessary consumption of system resources caused by real-time monitoring of the service operation parameters of the service objects, a corresponding condition can be set in a mode, so that the service operation parameters of at least one service object associated with the hierarchical storage system can be obtained only when the condition is triggered.
For example, one condition may be that acquisition of business operations parameters is not initiated until it is determined that the number of business objects associated with the tiered storage system exceeds a first number. It will be appreciated that as the number of business objects added to the tiered storage system increases, the business process pressure on behalf of the tiered storage system in the future may increase.
Still alternatively, another condition may be that acquisition of the service operation parameter is not started until the current time period is determined to be a service peak time period of the tiered storage system. It will be appreciated that when the tiered storage system is located during peak business processing hours, the greater the traffic processing pressure may be on behalf of the tiered storage system in the future. Specifically, whether the current time period is the service peak time period of the hierarchical storage system may be determined according to pre-stored historical processing service information including service processing pressures at various time periods.
S202, determining the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of at least one business object.
And S203, if the service processing pressure in the first preset time period is determined to exceed the preset threshold, adjusting the first disk brushing grade of the hierarchical storage system to be the second disk brushing grade.
And S204, if the fact that the service processing pressure in the third preset time period does not exceed the preset threshold value is detected, adjusting the second disk brushing grade of the layered storage system to be the first disk brushing grade.
Furthermore, the problem to be solved in the present application is to avoid adopting the same disk-scrubbing mode to perform hierarchical migration of data. This tends to have a lag or lead in which migration action occurs. Therefore, the corresponding flash disk grade can be dynamically adjusted according to the estimated business processing pressure of the hierarchical storage system. Furthermore, when the business processing pressure of the hierarchical storage system is low, the disk brushing level of the hierarchical storage system can be returned to be the same as the first disk brushing level.
The third preset time period is not specifically limited, and may be the same as or different from the first preset time period.
Further, as shown in fig. 4, for a flow chart of a disk-flushing method based on a tiered storage system proposed by the present application, first, the present application may determine to acquire a service operation parameter of at least one service object associated with the tiered storage system when it is detected that a number of service objects currently associated with the tiered storage system exceeds a first number, and/or when it is determined that a current time period is a service peak time period of the tiered storage system based on historical processing service information. And determining the service processing pressure of the hierarchical storage system in a first preset time period based on the service operation parameters of at least one service object, wherein the process of determining the service processing pressure can be obtained by acquiring the service request quantity and the service request type of the at least one service object generated in the hierarchical storage system in a second preset time period, determining the data quantity of the at least one service object to be accessed in the first preset time period based on the service request quantity and the service request type, and determining the service processing pressure based on the matching relationship between the data quantity and the residual storage capacity of a cache region in the hierarchical storage system.
Furthermore, in the embodiment of the present application, when it is determined that the service processing pressure in the first preset time period exceeds the preset threshold, the first disk brushing level of the hierarchical storage system is adjusted to the second disk brushing level, and when it is detected that the service processing pressure in the third preset time period does not exceed the preset threshold, the second disk brushing level of the hierarchical storage system is adjusted to the first disk brushing level.
In the application, a service operation parameter of at least one service object associated with the hierarchical storage system may be acquired, and based on the service operation parameter of the at least one service object, a service processing pressure of the hierarchical storage system within a first preset time period may be determined, and if it is determined that the service processing pressure within the first preset time period exceeds a preset threshold, a first disk-brushing level of the hierarchical storage system may be adjusted to a second disk-brushing level, where the disk-brushing level corresponds to a disk-brushing frequency and a disk-brushing force. By applying the technical scheme of the application, the service pressure of the storage system in the future time period can be determined by detecting the service operation parameters of the service objects associated with the hierarchical storage system, and the corresponding brushing grade can be dynamically adjusted. Therefore, the condition that the disk refreshing process conflicts with the service processing process in the disk refreshing process of the layered storage mechanism in the related technology is avoided, and the service processing efficiency is influenced.
In another embodiment of the present application, as shown in fig. 5, the present application further provides a disk brushing method and apparatus based on a hierarchical storage system. The device comprises an acquisition module 301, a determination module 302, and an adjustment module 303, wherein,
an obtaining module 301 configured to obtain a service operation parameter of at least one service object associated with the hierarchical storage system;
a determining module 302 configured to determine a business processing pressure of the hierarchical storage system within a first preset time period based on the business operation parameter of the at least one business object;
an adjusting module 303, configured to adjust a first disk brushing level of the hierarchical storage system to a second disk brushing level if it is determined that the service processing pressure in the first preset time period exceeds a preset threshold, where the disk brushing level corresponds to a disk brushing frequency and a disk brushing force.
In another embodiment of the present application, the obtaining module 301 further includes:
an obtaining module 301 configured to reduce a first disk-brushing frequency of the hierarchical storage system to a second disk-brushing frequency;
the obtaining module 301 is configured to increase a first disk-brushing strength of the hierarchical storage system to a second disk-brushing strength.
In another embodiment of the present application, the obtaining module 301 further includes:
an obtaining module 301, configured to obtain a first brushing data amount at the first brushing level;
an obtaining module 301, configured to determine a ratio of the first disk-brushing data quantity to a total data quantity, where the total data quantity is a data quantity in a cache region in the hierarchical storage system;
an obtaining module 301, configured to determine a second brushing data amount at a second brushing level based on a ratio of the first brushing data amount to the total data amount;
an obtaining module 301 configured to determine the second brushing force according to the second brushing data amount.
In another embodiment of the present application, the obtaining module 301 further includes:
an obtaining module 301, configured to obtain a first disk brushing duration at the first disk brushing level, and reduce the first disk brushing duration to a second disk brushing duration;
an obtaining module 301, configured to determine the second brushing strength based on the second brushing data amount and the second brushing duration.
In another embodiment of the present application, the obtaining module 301 further includes:
An obtaining module 301, configured to obtain, in a second preset time period of history, a number of service requests and a type of the service requests, which are generated by the at least one service object in the hierarchical storage system;
an obtaining module 301, configured to determine, based on the number of service requests and the type of service requests, the number of data that the at least one service object needs to access in the first preset time period;
the obtaining module 301 is configured to determine, based on a matching relationship between the amount of data that needs to be accessed by the at least one service object and the remaining storage capacity of a cache region in the hierarchical storage system, a service processing pressure of the hierarchical storage system in the first preset time period.
In another embodiment of the present application, the obtaining module 301 further includes:
an obtaining module 301, configured to adjust the second disk-brushing level of the hierarchical storage system to the first disk-brushing level if it is detected that the service processing pressure in a third preset time period does not exceed the preset threshold.
In another embodiment of the present application, the obtaining module 301 further includes:
an acquisition module 301 configured to determine that a number of business objects associated with the tiered storage system exceeds a first number;
And/or the presence of a gas in the atmosphere,
an obtaining module 301 configured to determine a current time period as a traffic peak time period of the hierarchical storage system based on historical processing traffic information.
FIG. 6 is a block diagram illustrating a logical configuration of an electronic device in accordance with an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
In an exemplary embodiment, there is also provided a non-transitory computer readable storage medium, such as a memory, including instructions executable by an electronic device processor to perform the method of network monitoring described above, the method comprising: acquiring service operation parameters of at least one service object associated with the hierarchical storage system; determining the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of the at least one business object; and if the service processing pressure in the first preset time period is determined to exceed a preset threshold value, adjusting the first disk brushing grade of the hierarchical storage system to a second disk brushing grade, wherein the disk brushing grade corresponds to a disk brushing frequency and a disk brushing force. Optionally, the instructions may also be executable by a processor of the electronic device to perform other steps involved in the exemplary embodiments described above. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided an application/computer program product including one or more instructions executable by a processor of an electronic device to perform the above-described method of network monitoring, the method comprising: acquiring service operation parameters of at least one service object associated with the hierarchical storage system; determining the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of the at least one business object; and if the service processing pressure in the first preset time period is determined to exceed a preset threshold value, adjusting the first disk brushing grade of the hierarchical storage system to a second disk brushing grade, wherein the disk brushing grade corresponds to a disk brushing frequency and a disk brushing force. Optionally, the instructions may also be executable by a processor of the electronic device to perform other steps involved in the exemplary embodiments described above.
Fig. 6 is an exemplary diagram of the computer device 40. It will be understood by those skilled in the art that the schematic diagram 5 is merely an example of the computer device 40, and does not constitute a limitation of the computer device 40, and may include more or less components than those shown, or combine certain components, or different components, for example, the computer device 40 may also include input output devices, network access devices, buses, etc.
The Processor 402 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor 402 may be any conventional processor or the like, the processor 402 being the control center for the computer device 40 and connecting the various parts of the overall computer device 40 using various interfaces and lines.
The modules integrated by the computer device 40 may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by hardware related to computer readable instructions, which may be stored in a computer readable storage medium, and when the computer readable instructions are executed by a processor, the steps of the method embodiments may be implemented.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (8)
1. A disk refreshing method based on a hierarchical storage system is characterized by comprising the following steps:
acquiring service operation parameters of at least one service object associated with a hierarchical storage system;
predicting the service processing pressure of the hierarchical storage system in a first preset time period based on the service operation parameters of the at least one service object;
if the service processing pressure in the first preset time period is determined to exceed a preset threshold value, adjusting a first disk brushing grade of the hierarchical storage system to a second disk brushing grade, wherein the disk brushing grade corresponds to a disk brushing frequency and a disk brushing force;
the adjusting the first disk-cleaning level of the hierarchical storage system to a second disk-cleaning level includes:
reducing a first disk-brushing frequency of the hierarchical storage system to a second disk-brushing frequency;
increasing a first disk-brushing force of the tiered storage system to a second disk-brushing force, comprising: acquiring a first brushing data quantity under the first brushing grade; determining a ratio of the first disk refreshing data quantity to a total data quantity, wherein the total data quantity is a data quantity located in a cache region in the hierarchical storage system; determining a second brushing data quantity under a second brushing grade based on the ratio of the first brushing data quantity to the total data quantity; and determining the second disc brushing force according to the second disc brushing data quantity.
2. The method of claim 1, wherein determining the second brushing force based on the second brushing data amount comprises:
acquiring a first disc brushing time length under the first disc brushing grade, and reducing the first disc brushing time length to a second disc brushing time length;
and determining the second disk brushing force based on the second disk brushing data quantity and the second disk brushing time length.
3. The method of claim 1, wherein the determining the business process pressure of the tiered storage system for a first preset time period based on the business operation parameters of the at least one business object comprises:
acquiring the quantity and the type of service requests generated by the at least one service object in the hierarchical storage system within a second preset time period;
determining the quantity of data which needs to be accessed by the at least one business object in the first preset time period based on the quantity of the business requests and the type of the business requests;
and determining the business processing pressure of the hierarchical storage system in the first preset time period based on the matching relation between the data quantity which needs to be accessed by the at least one business object and the residual storage capacity of the cache region in the hierarchical storage system.
4. The method of claim 1, wherein after the adjusting the first brush class of the tiered storage system to the second brush class, further comprising:
and if the service processing pressure in a third preset time period is detected not to exceed the preset threshold value, adjusting the second disk brushing grade of the hierarchical storage system to the first disk brushing grade.
5. The method of claim 1, wherein prior to said obtaining business operational parameters for at least one business object associated with a tiered storage system, further comprising:
determining that a number of business objects associated with the tiered storage system exceeds a first number;
and/or the presence of a gas in the gas,
and determining the current time period as the traffic peak time period of the hierarchical storage system based on the historical processing traffic information.
6. A disk brushing device based on a layered storage system is characterized by comprising:
an acquisition module configured to acquire a business operation parameter of at least one business object associated with the hierarchical storage system;
the determining module is configured to determine the business processing pressure of the hierarchical storage system in a first preset time period based on the business operation parameters of the at least one business object;
An adjusting module configured to adjust a first disk brushing grade of the hierarchical storage system to a second disk brushing grade if it is determined that the service processing pressure within the first preset time period exceeds a preset threshold, where the disk brushing grade corresponds to a disk brushing frequency and a disk brushing force;
the acquisition module is configured to reduce a first disk-brushing frequency of the hierarchical storage system to a second disk-brushing frequency; increasing a first disk brushing force of the tiered storage system to a second disk brushing force, comprising: acquiring a first brushing data quantity under the first brushing grade; determining a ratio of the first disk refreshing data quantity to a total data quantity, wherein the total data quantity is a data quantity located in a cache region in the hierarchical storage system; determining a second brushing data quantity under a second brushing grade based on the ratio of the first brushing data quantity to the total data quantity; and determining the second disc brushing force according to the second disc brushing data quantity.
7. An electronic device, comprising:
a memory for storing executable instructions; and the number of the first and second groups,
a processor for displaying with the memory to execute the executable instructions to perform the operations of the hierarchical storage system based disk scrubbing method of any of claims 1-5.
8. A computer-readable storage medium storing computer-readable instructions that, when executed, perform the operations of the hierarchical storage system-based disk brushing method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011281870.2A CN112306901B (en) | 2020-11-16 | 2020-11-16 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011281870.2A CN112306901B (en) | 2020-11-16 | 2020-11-16 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112306901A CN112306901A (en) | 2021-02-02 |
| CN112306901B true CN112306901B (en) | 2022-07-29 |
Family
ID=74335816
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011281870.2A Active CN112306901B (en) | 2020-11-16 | 2020-11-16 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112306901B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508789A (en) * | 2011-10-14 | 2012-06-20 | 浪潮电子信息产业股份有限公司 | Grading storage method for system |
| CN102508619A (en) * | 2011-11-21 | 2012-06-20 | 成都市华为赛门铁克科技有限公司 | Memory system, and method and system for controlling service quality of memory system |
| CN107193500A (en) * | 2017-05-26 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of distributed file system Bedding storage method and system |
| CN107807796A (en) * | 2017-11-17 | 2018-03-16 | 北京联想超融合科技有限公司 | A kind of data hierarchy method, terminal and system based on super fusion storage system |
| CN109189578A (en) * | 2018-09-06 | 2019-01-11 | 北京京东尚科信息技术有限公司 | Storage server distribution method, device, management server and storage system |
| CN110058786A (en) * | 2018-01-18 | 2019-07-26 | 伊姆西Ip控股有限责任公司 | For controlling the method, apparatus and computer program product of the write request in storage system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160004475A1 (en) * | 2013-02-28 | 2016-01-07 | Hitachi, Ltd | Management system and method of dynamic storage service level monitoring |
| CN107870916A (en) * | 2016-09-23 | 2018-04-03 | 伊姆西Ip控股有限责任公司 | Memory management method and equipment |
-
2020
- 2020-11-16 CN CN202011281870.2A patent/CN112306901B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508789A (en) * | 2011-10-14 | 2012-06-20 | 浪潮电子信息产业股份有限公司 | Grading storage method for system |
| CN102508619A (en) * | 2011-11-21 | 2012-06-20 | 成都市华为赛门铁克科技有限公司 | Memory system, and method and system for controlling service quality of memory system |
| CN107193500A (en) * | 2017-05-26 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of distributed file system Bedding storage method and system |
| CN107807796A (en) * | 2017-11-17 | 2018-03-16 | 北京联想超融合科技有限公司 | A kind of data hierarchy method, terminal and system based on super fusion storage system |
| CN110058786A (en) * | 2018-01-18 | 2019-07-26 | 伊姆西Ip控股有限责任公司 | For controlling the method, apparatus and computer program product of the write request in storage system |
| CN109189578A (en) * | 2018-09-06 | 2019-01-11 | 北京京东尚科信息技术有限公司 | Storage server distribution method, device, management server and storage system |
Non-Patent Citations (1)
| Title |
|---|
| 基于云存储的图片管理***移动端的设计与实现;许婧;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20190115;第2019卷(第1期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112306901A (en) | 2021-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8521986B2 (en) | Allocating storage memory based on future file size or use estimates | |
| CN105205014B (en) | A kind of date storage method and device | |
| CN105474180B (en) | Method, calculating equipment and computer readable storage medium for storage management | |
| US20150317246A1 (en) | Memory Reclamation Method and Apparatus | |
| US8001323B2 (en) | Network storage system, management method therefor, and control program product therefor | |
| CN111078408B (en) | Memory allocation method and device, storage medium and electronic equipment | |
| US9710283B2 (en) | System and method for pre-storing small data files into a page-cache and performing reading and writing to the page cache during booting | |
| CN110196681B (en) | Disk data write-in control method and device for business write operation and electronic equipment | |
| CN108292162A (en) | Software definition fifo buffer for multi-thread access | |
| US20160065663A1 (en) | Dynamic load-based merging | |
| US10789170B2 (en) | Storage management method, electronic device and computer readable medium | |
| CN111815228A (en) | Pricing method and device for distribution order, electronic equipment and storage medium | |
| CN108874324B (en) | Access request processing method, device, equipment and readable storage medium | |
| US9514162B2 (en) | Smart polling frequency | |
| CN112231242A (en) | Data caching method, storage control device and storage equipment | |
| EP3964973A1 (en) | File loading method and apparatus, electronic device, and storage medium | |
| CN111104178A (en) | Dynamic library loading method, terminal device and storage medium | |
| CN109582649A (en) | A kind of metadata storing method, device, equipment and readable storage medium storing program for executing | |
| CN109508150B (en) | Method and device for allocating storage space | |
| CN112306901B (en) | Disk refreshing method and device based on layered storage system, electronic equipment and medium | |
| CN115934002A (en) | Solid state disk access method, solid state disk, storage system and cloud server | |
| CN113609074B (en) | Method and related device for warehousing and archiving medical image file | |
| CN112711726B (en) | Method, device, computer equipment and storage medium for caching video data | |
| US9223689B2 (en) | Apparatus and method for managing memory | |
| CN110717743A (en) | Distribution method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |