CN113971137A - Garbage recovery method and device - Google Patents
Garbage recovery method and device Download PDFInfo
- Publication number
- CN113971137A CN113971137A CN202010711871.XA CN202010711871A CN113971137A CN 113971137 A CN113971137 A CN 113971137A CN 202010711871 A CN202010711871 A CN 202010711871A CN 113971137 A CN113971137 A CN 113971137A
- Authority
- CN
- China
- Prior art keywords
- time period
- storage system
- data
- time
- pressure value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/10—Address translation
- G06F12/1009—Address translation using page tables, e.g. page table structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Memory System (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a garbage recycling method, a storage system can determine a first time period according to a service pressure value of the storage system in a historical time period, wherein the service pressure value of the storage system in the first time period is lower than a set threshold, and the service pressure value of the storage system in the historical time period is obtained according to any one or more of IOPS, IO size, data read-write proportion, deduplication compression proportion and throughput, so that the storage system can recycle garbage in the first time period with a smaller service pressure value. Therefore, the storage system can have more blank logic block groups to support the storage system to write a large amount of new data in other time periods, so that the influence of garbage collection in the other time periods on the performance of the host such as IOPS (input/output system) and the like can be reduced, and the performance of the host with a large amount of data writing tasks is improved. In addition, this application embodiment still provides a rubbish recovery unit.
Description
Technical Field
The present application relates to the field of storage, and in particular, to a garbage recycling method and apparatus.
Background
In a storage medium of a partial storage system, such as a Solid State Disk (SSD), a Hard Disk Drive (HDD), etc., a read-on-write (ROW) mechanism is usually used to write new data. That is, when data in the storage system is modified, new data will be written to the set of blank logic blocks in the storage system, while the original data will be marked as garbage data for reclamation. In the Garbage Collection (GC) process, valid data in a logical block group (CKG) can be moved to a new logical block group, and then the valid data and invalid data in the logical block group are erased to form a new blank logical block group for storing other new data.
However, the garbage collection process is accompanied by certain calculation and data writing, and generates corresponding system performance consumption, which affects the number of times of performing read/write Operations Per Second (IOPS) of the host in the storage system.
Disclosure of Invention
The embodiment of the application provides a garbage recovery method and device, so as to improve the performance of a host in a storage system when a large amount of data is written into a task.
In a first aspect, an embodiment of the present application provides a garbage collection method, which may be applied to a storage system. Specifically, the storage system may determine a first time period in which a future service pressure value of the storage system is smaller according to a service pressure value of the storage system in a historical time period, specifically, the service pressure value of the storage system in the first time period may be lower than a set threshold, and then the storage system may perform garbage collection in the first time period. Because the storage system performs garbage collection in the first time period with a smaller service pressure value, the blank logic block groups of the storage system can be increased, and thus, the storage system can have more blank logic block groups to support the storage system to write a large amount of new data in other time periods (for example, the time period with a higher service pressure value) quickly, so that the concurrency of garbage collection performed by the storage system in order to increase the blank logic block groups in other time periods can be reduced, the influence of garbage collection performed in other time periods on the performance of the host, such as the IOPS of the host, is also reduced, and the performance of the host with a large amount of data writing tasks is improved.
The service pressure value of the storage system in the historical time period can be determined according to any one or more of the IOPS, the IO size, the data read-write proportion, the deduplication compression proportion and the throughput. Of course, in other possible embodiments, the service pressure value may also be determined according to other data of the storage system in the historical time period, which is not limited in this embodiment.
In a possible implementation manner, when determining the first time period, the storage system may specifically determine, according to the traffic pressure value of the storage system in the historical time period, an ending time of the first time period, which is not later than a starting time of the second time period, for example, the ending time of the first time period may be the starting time of the second time period, or the ending time of the first time period may be before the starting time of the second time period. Meanwhile, the set threshold value required for determining the first time period may be determined according to the service pressure value in the second time period, for example, the set threshold value may be an average value of the service pressure values in the second time period, or a minimum service pressure value in the second time period, and the like. Of course, in other possible embodiments, the set threshold may be a fixed value. The specific implementation process of how to determine the set threshold is not limited in the embodiment of the present application.
In a possible implementation manner, the storage system may further determine a traffic pressure value of the storage system in a second time period in the future according to the traffic pressure value of the storage system in the historical time period, so that the storage system may determine the garbage collection rate of the storage system in the first time period according to the traffic pressure value of the storage system in the second time period. For example, when the traffic pressure value of the storage system in the second time period is larger, garbage collection may be performed at a larger garbage collection rate in the first time period, and when the traffic pressure value of the storage system in the second time period is smaller, garbage collection may be performed at a smaller garbage collection rate in the first time period. In this way, the garbage collection rate of the storage system in the first time period can be dynamically planned and adjusted, so that the flexibility of implementation of the scheme can be increased.
In one possible embodiment, the storage system can perform garbage collection not only in the first time period, but also in the second time period at a smaller garbage collection rate. Of course, the garbage collection rate of the storage system in the second time period is less than the garbage collection rate of the storage system in the first time period. Because the garbage collection rate of the storage system in the second time period is low, the influence of the garbage collection process of the storage system in the second time period on the performance of the host in the storage system is low, and therefore the performance of the host in the second time period can still meet the requirement of the front-end service. Meanwhile, the storage system performs garbage collection in the second time period, and releases the blank logic block group while writing data, so that the storage system can have more available storage space for a long time in the second time period.
In a possible implementation manner, the storage system may further determine target data for performing garbage collection by the storage system, where the target data includes data that a predicted failure time is later than a preset time in the second time period, and the preset time may be any time in the second time period, for example, a start time of the second time period, or a time that is a certain time later than the start time of the second time period, and the like, which is not limited in this embodiment. Therefore, in the first time period, the storage system can not perform garbage collection on the data which is invalid before the preset time, and can perform garbage collection on the data which is invalid after the preset time, so that partial data of the storage system which is subjected to garbage collection in the first time period can be reduced as far as possible, the data is invalid when entering the second time period, and invalid write amplification is reduced.
In one possible implementation, the target data determined by the storage system may include first data and second data, where a failure time of the first data is later than a failure time of the second data. Therefore, when the storage system performs garbage collection, the first data can be preferentially subjected to garbage collection, and after all the first data are subjected to garbage collection, if the blank logic block group still needs to be released through garbage collection, the storage system can continue to perform garbage collection on the second garbage collection, and after all the first data are subjected to garbage collection or part of the first data are subjected to garbage collection, the blank logic block group of the storage system can meet the current data writing requirement, the storage system can be free from performing garbage collection on the second data, so that the situation that the second data are subjected to data failure soon after the second data are subjected to garbage collection can be avoided as much as possible, and therefore invalid write amplification in garbage collection can be reduced as much as possible.
In a second aspect, an embodiment of the present application further provides a garbage recycling apparatus, configured to execute the method described in any implementation manner of the first aspect.
In a third aspect, an embodiment of the present application further provides a garbage collection apparatus, where the garbage collection apparatus includes a memory and a processor, and the processor is configured to execute instructions stored in the memory to perform the method described in any implementation manner of the first aspect.
A fourth aspect of the present application provides a computer-readable medium having stored therein instructions, which when run on a computer, cause the computer to perform the method of the above-described aspects.
A fifth aspect of the present application provides a computer program product which, when run on a computer, causes the computer to perform the method of the above-described aspects.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a schematic structural diagram of a storage system according to an embodiment of the present application;
fig. 2 is a diagram showing a configuration example of the controller 11;
fig. 3 is a schematic flow chart of a garbage recycling method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of the traffic pressure variation of the storage system at different time periods;
fig. 5 is a schematic structural diagram of a garbage recycling device according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a garbage recovery method and a garbage recovery device, so as to improve the performance of a host such as IOPS (input/output system) when a large amount of data are written into a task.
Fig. 1 depicts a composition diagram of a storage system provided by an embodiment of the present application. The storage system shown in fig. 1 includes a host 10, at least one controller (e.g., controller 11 shown in fig. 1), and a plurality of solid state disks 22. The controller 11 may be connected to the host 10 via a Storage Area Network (SAN). The controller 11 may be a computing device such as a server, desktop computer, etc. An operating system and an application program are installed on the controller 11. The controller 11 may receive an input/output (I/O) request from the host 10. The controller 11 may also store data carried in the I/O request (if any) and write the data to the solid state disk 22. The Solid State disk is a memory using a flash memory (flash memory) chip as a medium, and is also called a Solid State Drive (SSD).
Fig. 1 is merely an illustration, and in practical applications, the storage system may include two or more controllers, each of which has a similar physical structure and function as the controller 11, and the present embodiment does not limit the connection manner between the controllers and between any one of the controllers and the solid state disk 22. As long as the respective controllers, and the respective controllers and the solid state disk 22, can communicate with each other. In addition, in the present embodiment, the controller 11 often sends instructions to the solid state disk to the plurality of solid state disks, and for convenience of description, the set including the plurality of solid state disks 22 is referred to as a solid state disk group.
It should be noted that fig. 1 illustrates an example in which the storage medium is a solid state disk, and in other possible application scenarios, the storage medium in the storage system may also be another storage medium such as a mechanical hard disk (HDD), which is not limited in this embodiment.
Fig. 2 is a diagram showing an example of the configuration of the controller 11, and as shown in fig. 2, the controller 11 includes an interface card 110, a processor 112, and an interface card 113.
The interface card 110 is used to communicate with the host 10, and the controller 11 may receive an operation instruction of the host 10 through the interface card 110. Processor 112 may be a Central Processing Unit (CPU). In the embodiment of the present application, the processor 112 may be configured to receive an I/O request from the host 10 and process the I/O request. The I/O request may be a write data request or a read data request, and the processor 112 may further send data in the write data request to the solid state disk 22. In addition, the processor 112 may also be used to perform garbage collection operations. The interface card 113 is used for communicating with the solid state disk 22, and the controller 11 may send a request for writing data (including data and a lifecycle level of the data) to the solid state disk 22 through the interface card 113 for storage.
Optionally, the controller 11 may further include a memory 111. The memory 111 is used to temporarily store data received from an external device such as the host 10 or data read from the solid state disk 22. When receiving a plurality of write data requests transmitted from an external device such as the host 10, the controller 11 may temporarily store data in the plurality of write data requests in the memory 111. When the capacity of the memory 111 reaches a certain threshold, the data stored by the memory 111 and the logical address allocated to the data are sent to the solid state disk 22. The solid state disk 22 stores data. The memory 111 includes a volatile memory, a flash memory chip, or a combination thereof. Volatile memory is, for example, random-access memory (RAM). Flash memory chips are various types of machine-readable media that can store program code, such as floppy disks, hard disks, solid state disks, optical disks, and the like. The memory 111 has a power-saving function, which means that data stored in the memory 111 cannot be lost when the system is powered off and powered on again.
After the data is written to the storage system, the controller 11 needs to record the validity of the data. The validity of the data is determined whether the data is modified. If the data is the first write, the data may be recorded as valid (referred to as valid data). If the data is modified, the data before modification is recorded as invalid (referred to as invalid data). Specifically, the validity of the data may be recorded using a bitmap. For example, each "bit" of the bitmap corresponds to a logical address of data of 1KB in size, and when the "bit" is 1, it represents that the data stored in the logical address is valid, and when the "bit" is 0, it represents that the data stored in the logical address is invalid. The bitmap may be stored in the memory 111 or in the solid state disk.
It should be noted that the controller 11 belongs to a system controller, and the system controller is an independent device, which is different from a control chip in the solid state disk. In this embodiment, the control chip of the solid state disk is referred to as a flash memory controller.
The solid state disk 22 includes a flash memory controller and a plurality of flash memory chips. The flash memory controller is used for executing operations such as a write data request or a read data request sent by the controller 11.
The flash memory controller includes a Flash Translation Layer (FTL). The flash translation layer is used for storing the corresponding relation between the logical address and the actual address of the data. Therefore, the flash translation layer is used for converting the logical address in the write data request or the read data request sent by the system controller into the actual address of the data in the solid state disk. The logical address of the data is allocated by the system controller, and the solid state disk provides a subset of the logical address interval of the system controller. The logical address of the data includes a start logical address indicating a position of a segment where the data is located and a length representing a size of the data. The actual address of the data may be a physical address of the data in the solid state disk, or may be an address that is visible only to the flash memory controller through virtualization based on the physical address. The virtualized real address is not visible to the system controller.
Solid state drives typically include one or more flash memory chips. Each flash memory chip includes a number of erase blocks. Solid state disks are read or written on a page basis, but an erase operation can only be on a block basis, where an erase operation is to set all bits of the block to "1". Prior to erasure, the flash memory controller needs to copy the valid data in this erase block into the blank pages of another block.
Each erase block contains a plurality of pages. When the solid state disk executes a data writing request, data is written in units of pages. For example, the controller 11 sends a write data request to the flash memory controller. The write data request includes a logical address of the data. The flash memory controller, after receiving the write data request, successively writes the data into one or more erase blocks in the order of time of receipt. The continuous writing of one or more erase blocks means that the flash memory controller searches for a blank erase block, writes data into the blank erase block until the blank erase block is filled, and when the size of the data exceeds the capacity of the erase block, the flash memory controller searches for the next blank erase block again and continues writing. The flash translation layer establishes and stores a correspondence between the logical address and the actual address of the page to which the data is written. When the controller 11 sends a read data request to the flash memory controller, requesting to read data, the read data request includes a logical address. The flash memory controller reads data according to the logical address and the corresponding relationship between the logical address and the real address, and sends the data to the controller 11.
In a storage system, a single solid state disk may fail, resulting in data loss. In this embodiment, Redundant Array of Inexpensive Disks (RAID) technology may be used to ensure the reliability of data. The following describes a mechanism for performing redundancy protection on data in a solid state disk.
First, the controller 11 performs spatial management on the solid state disk in the form of a logical block (chunk, CK). A logical block is a spatial concept, whose size is exemplified by 4MB, but is not limited to 4 MB. The logic blocks from different solid state disks may form a logic block set, and the controller 11 divides the logic blocks in the logic block set into data groups and check groups according to the set RAID type. The data group comprises at least two logic blocks for storing data, and the check group comprises at least one logic block for storing check data of the data. In this embodiment, a logical block set including a data group and a check group is referred to as a logical block group. When the controller 11 has a certain size of data in the memory 111, the data may be divided into a plurality of data units according to the set RAID type, and a check unit is obtained by calculation, and the data units and the check unit are sent to the solid state disk to be stored in the logic block group. Each logical block in the data group is used for storing a data unit, and each logical block in the check group is used for storing a check unit. After storage, the data units and corresponding check units form a stripe. The set of logical blocks includes a plurality of stripes.
Both the data units and the check units contained in the stripe may be referred to as stripe units. In this embodiment, the size of the stripe unit is 8KB, but is not limited to 8 KB. Assume that one logical block is taken out from each of the 5 solid state disks to form a logical block set, and then the controller 11 groups the logical blocks in the logical block set according to RAID type (taking RAID6 as an example), for example, logical block 1, logical block 2, and logical block 3 are data block groups, and logical block 4 and logical block 5 are check block groups. When the data stored in the memory 111 reaches 24KB (i.e., 8KB × 3), the data is divided into 3 data units each of which is 8KB, and then 2 check units each of which is also 8KB are calculated. The controller 11 sends the data units and the verification units to the solid state disk and stores the data units and the verification units in the logic block group. It will be appreciated that in accordance with the redundancy protection mechanism of RAID6, when any two data units or check units fail, the failed units may be reconstructed from the remaining data units or check units.
In this embodiment, when the memory 111 is full of data equal to one stripe size, the controller 11 sends a write data request to the solid state disk group, so that the solid state disks 22 included in the solid state disk group write data into the stripes of the logical block group. At this time, the controller 11 needs to determine whether there is an already allocated logical block group, and if there is a logical block group and the logical block group still has enough space to accommodate the data, the controller 11 can instruct the solid state hard disk group to write the data into the already allocated logical block group. Specifically, the controller 11 acquires a segment of unused logical address from the logical address interval of the allocated logical block group, and sends the logical address carried in the write data request to the solid state disk 22. After the solid state disk 22 stores the data, a mapping relationship between the logical address and the actual address for storing the data is created in the flash translation layer.
In the above example, if the controller 11 determines that there are no already allocated logical block groups in the system, or that all the already allocated logical block groups are full of data, the controller 11 needs to create a new logical block group. The creation process may be that the controller 11 determines that the remaining space of the system is enough to create a new logical block group according to its own record of the available space owned by each solid state disk. Next, the controller 11 obtains one logical block from each of the different solid state disks 22, and constructs the logical blocks into a new logical block group according to the set RAID type (refer to the above description of the logical block group). The controller 11 then assigns a segment of logical addresses to each logical block, and the set of logical addresses is the logical address of the new set of logical blocks.
In order to make enough space available in the storage system for creating the logical block group all the time, the controller 11 may monitor the available space of each solid state disk 22 in real time or periodically, so as to know the available space of the entire system. Garbage collection may be initiated when the available space (e.g., a set of empty logic blocks) of the system falls below a set space threshold. For example, the capacity of one solid state disk 22 is 128G, the total capacity of all solid state disks included in the storage system shown in fig. 1 (assuming that the storage system includes 10 solid state disks) is 1280G, and the space threshold may be set to 640G. That is, when the storage system stores data up to half of the total capacity, the remaining available space also reaches the space threshold, and the controller 11 may perform garbage collection. It is understood that 640G is only one example of a spatial threshold, and that the spatial threshold may be set to other values. In addition, garbage collection may also be triggered when the used space of the system reaches a set space threshold. In addition, in other possible embodiments, garbage collection may also be initiated when the data amount of invalid data contained in one or more stripes reaches a set threshold. Garbage recycling is different from garbage recycling inside the solid state disk. The garbage collection in the solid state disk is completed by the solid state disk.
The controller 11 may perform garbage collection in units of logical block groups. For example, the controller 11 obtains the logical address of the valid data in one logical block group according to the bitmap, and sends the logical address of the valid data to the solid state disk group, so that each solid state disk 22 can read the data according to the logical address of the valid data and send the data to the controller 11. The controller 11 allocates a new logical address to the valid data, the new logical address belongs to the new logical block group, and sends the allocated new logical address to the solid state disk group, and after the valid data is written into the new block by each solid state disk 22, the mapping relationship between the actual address and the new logical address is saved. Then, the controller 11 sends a demapping (unmap) command (or a trim command) to the solid state disk group, where the demapping command includes a logical address interval of the logical block group to be recovered, and after each solid state disk 22 receives the demapping command, the corresponding relationship between the logical address and the actual address of the logical address interval stored in the flash translation layer is deleted. The solid state disk 22 may mark the block corresponding to the real address of the valid data before the move as a block that does not contain valid data. Subsequently, the controller 11 may release the logical address interval of the logical block group to be reclaimed and the actual physical space (which may also be understood as the actual address) occupied by the logical block group.
Garbage collection is executed by taking the logic block group as a unit, and only after all valid data in the logic block group are moved to a new logic block group, can a blank logic block group be released. If garbage collection is performed at a finer granularity than the logical block groups, the purpose of performing a smaller number of data moves but freeing up the same amount of available space can be achieved.
However, garbage collection is accompanied by certain computation and data writing, occupies part of data read-write resources of the storage system, and consumes the performance of the storage system, which affects the performance of the host 10 in the storage system, such as IOPS. Taking the IOPS as an example, when the storage system has sufficient available storage space and does not perform garbage collection, the maximum IOPS of the host 10 may reach 10000, and when the storage space of the storage system is smaller than a preset threshold, the storage system starts garbage collection to increase the available storage space of the storage system, and then computation and data writing involved in the garbage collection process preempts partial read-write performance of the storage system, so that the maximum IOPS currently reached by the host 10 in the storage system may be reduced to 6000, and the higher the concurrence of garbage collection is, the lower the maximum IOPS that can be reached by the host 10 may be. When the storage system needs to write a large amount of data quickly, the storage system executes a garbage collection process to release the storage space in a large amount, so that the system performance consumption generated by the garbage collection process may seriously reduce the IOPS of the host 10. At the same time, other performance of the host 10 may also be degraded.
Therefore, the embodiment of the application provides a garbage collection method to improve the performance of a host when the host has a large amount of data writing tasks. Specifically, the storage system may determine, according to the service pressure value of the storage system in the historical time period, a future first time period in which the service pressure value of the storage system is lower than the set threshold, so that the storage system may perform garbage collection in the first time period in which the service pressure value is smaller, to increase the blank logic block groups of the storage system, and thus, the storage system may have more blank logic block groups to support the storage system to write a large amount of new data in other time periods (for example, in the time period in which the service pressure value is higher) quickly, so that a concurrence of garbage collection performed in order to increase the blank logic block groups in other time periods by the storage system may be reduced, an influence of garbage collection performed in the other time periods on performances of the host, such as IOPS, and the like, and the performance of the host when having a large amount of data writing tasks is improved.
It is noted that the storage system may be configured such that, when determining the first time period, a controller in the storage system determines the future first time period in a predictive manner using a processor integrated in the controller; alternatively, a processor (or a controller including the processor, etc.) may be separately configured in the storage system, and the first future time period is predicted by using the separately configured processor according to the traffic pressure value of the historical time period, and then is notified to the processor for controlling the garbage collection in the storage system; alternatively, the storage system may send the traffic pressure value of the storage system in the historical time period to a processor (or processing device) independent of the storage system, and the processor (or processing device) predicts a future first time period according to the received traffic pressure value, and then sends the predicted first time period to the storage system. In this embodiment, how the storage system determines the specific implementation of the first time period is not limited.
In some scenarios, the processor may be a CPU, including a CPU core or multiple CPU cores. Further, the processor may also be an Application Specific Integrated Circuit (ASIC), or be configured as one or more integrated circuits, such as: one or more microprocessors (digital signal processors, DSPs), or one or more Field Programmable Gate Arrays (FPGAs). The processor may perform the corresponding functions by running or executing software programs stored in the memory and calling data stored in the memory.
For convenience of description, an original processor in the memory system will be referred to as a first processor, and a processor separately configured in the memory system or a processor independent of the memory system will be referred to as a second processor. The first processor and the second processor may have the same physical form or different physical forms. The second processor may be a processing chip with computing power, and is configured to assume a function of predicting a first time period in which a future traffic pressure value is small, so as to reduce a computing burden of the first processor. In practical applications, the second processor may be an accelerator card, a coprocessor, a Graphics Processing Unit (GPU), a Neural Network Processor (NPU), or the like. In this embodiment, the first processor may be configured by one or more, and the second processor may be configured by one or more. However, it should be noted that the second processor is an optional component in this embodiment. Even if only one first processor is provided, the first processor can independently predict a first time period with a future service pressure value smaller than a set threshold value according to the service pressure value of the storage system in the historical time period, and perform garbage collection in the first time period. Of course, the first processor and the second processor may cooperate to complete the above process. For example, the first processor may be configured to collect the service pressure value of the storage system in the historical time period, and send the service pressure value in the historical time period to the second processor; the second processor predicts a first time period with a smaller future service pressure value of the storage system according to the received service pressure value and informs the first processor of the first time period; the first processor performs garbage collection on the storage system in the first time period informed by the second processor. Illustratively, the first processor and the second processor may be in communication with each other via a communication bus.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, various non-limiting embodiments accompanying the present application examples are described below with reference to the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 3, fig. 3 is a schematic flow chart illustrating a garbage collection method in an embodiment of the present application, where the method may be applied to a storage system, and the method may specifically be executed by a processor in the storage system or a controller integrating the processor. The garbage recycling method in this embodiment may specifically include:
s301: the storage system determines a first time period according to the service pressure value of the storage system in the historical time period, wherein the service pressure value of the storage system in the first time period is lower than a set threshold, and the service pressure value of the storage system in the historical time period is obtained according to any one or more of the IOPS, the IO size, the data read-write proportion, the deduplication compression proportion and the throughput of the storage system in the first time period.
In general, the traffic pressure (e.g., the pressure of writing data) of a storage system does not always keep a steady state, but exhibits intermittent high and low changes. As shown in fig. 4, the horizontal axis represents time, the vertical axis represents traffic pressure values, and the larger the traffic pressure value, the higher the traffic pressure of the storage system is indicated. At T0To T1The amount of data written by the storage system may be small (or even no data may be written to the storage system) during the time period of (T) the storage system is at0～T1Traffic pressure during the period is small; however, at T1To T2The amount of data written by the storage system may be large, and the storage system is at T1～T2During which the traffic pressure is large (e.g., the storage system may perform data synchronization or backup during the period of time, etc.). Due to T1～T2During the period, more data are newly written into the storage system, more available storage space of the storage system is needed, if the storage system rapidly writes a large amount of data and simultaneously increases the available storage space of the storage system by concurrently executing garbage collection, the storage system is in T1～T2The garbage collection process performed in the meantime may seriously affect the performance of writing data by the host in the storage system, such as seriously reducing the rate of writing data by the host.
Therefore, in the embodiment of the present application, before a time period with a higher service pressure comes, the storage system may perform garbage collection first, so that when the storage system enters the time period with the higher service pressure, the storage system may have more available storage space to support writing of a large amount of data, and reduce the concurrent amount of garbage collection performed in the time period, thereby improving the service performance of the host in the time period.
In specific implementation, the storage system may collect the service pressure value of the storage system in the historical time period, and the service pressure value in the historical time period may indicate the service pressure condition of the storage system in different past time periods, so that the service pressure of the storage system in the future time period may be determined according to the service pressure value of the storage system in the historical time period, and the first time period with lower service pressure may be determined. For example, assuming that the traffic pressure value of the storage system in the historical time period indicates that the traffic pressure of the storage system is lower in the time period of 00:00 to 06:00 each day and higher in the time period of 06:00 to 24:00 each day in the past month, the storage system may predict that the traffic pressure of the storage system is lower in the time period of 00:00 to 06:00 tomorrow and higher in the time period of 06:00 to 24:00 tomorrow.
In a specific implementation, the storage system may predict the traffic pressure value of the storage system in different future time periods according to the traffic pressure value of the historical time period, so that the time period in which the traffic pressure value is smaller than the set threshold may be used as the first time period in step S301. The set threshold for determining the first time period may be a preset fixed value or a dynamically adjusted threshold. For example, when the set threshold is not fixed, the storage system may predict a future change of the traffic pressure value of the storage system, so that the ending time of the first time period and the starting time of the second time period may be determined according to a difference value of the traffic pressure values of different periods. The determined ending time of the first time period may be the starting time of the second time period, or the ending time of the first time period may be earlier than the starting time of the second time period, and the traffic pressure value of the storage device in the first time period is not greater than the traffic pressure value of the storage device in the second time period. For example, assume T in FIG. 40If the time represents the current time, the storage system determines that the service pressure value of the storage system is at T through prediction1When the time suddenly increases, the storage systemThe current time T can be adjusted0To T1The time interval between the moments is taken as a first time interval, and T is taken as1The time and the time subsequent thereto constituting a time period (e.g. T)1To T2) As the second period of time.
In this embodiment, the first time period may refer to one time period, for example, a time period which is closest to the current time and in which the traffic pressure value is smaller than the set threshold, or may refer to multiple time periods, that is, multiple discontinuous time periods in which the traffic pressure value is smaller than the set threshold may all be taken as the first time period in step S301.
It should be noted that, for convenience of description, the foregoing embodiment describes a specific implementation process of determining the first time period by the storage system, by taking the example that the storage system predicts the future first time period according to the traffic pressure value of the historical time period. It should be understood that, in other possible embodiments, the process of predicting the first time period according to the traffic pressure value of the historical time period may also be implemented by another device independent from the storage system, and the other device sends the predicted first time period to the storage system, which is not limited in this embodiment.
As some examples, the traffic pressure value of the storage system in the historical time period may be obtained according to any one or more of an IOPS, an input/output (IO) size, a data read-write ratio, a deduplication compression ratio (a ratio of deleted duplicate data to data compressed by a compression algorithm), and a throughput (an amount of data transmitted per second by the storage system) of the storage system in the first time period. For example, the service pressure value may specifically be an IOPS, an IO size, a data read-write ratio, a deduplication compression ratio, or throughput, and the larger the IOPS, the larger the IO size, the smaller the read-write ratio, the smaller the deduplication compression ratio, or the larger the throughput, the larger the service pressure value; or, the service pressure value may be a value for evaluating the service pressure of the storage system, which is determined based on any one or more of the IOPS, the IO size, the data read-write ratio, the deduplication compression ratio, and the throughput, and particularly, when the service pressure value is determined based on multiple parameters, the service pressure value may be calculated by performing weighted summation (or weighted averaging) on the multiple parameters. In addition, the traffic pressure value of a certain time period may be a value, and the value may be used to characterize the average traffic pressure level of the storage system in the time period; alternatively, the traffic pressure value may be a plurality of values, and may be used to characterize the traffic pressure change of the storage system in the time period. It should be noted that the above examples are only used as some examples for determining the service pressure value and the specific expression form of the service pressure value, in practical applications, other possible implementations may also be adopted, the service pressure value may be determined based on any one or more parameters of the IOPS, the IO size, the data read-write ratio, the deduplication compression ratio, and the throughput, and the service pressure value may also be determined based on other expression forms, which is not limited in this embodiment.
S302: the storage system performs garbage collection in the first time period.
Because the traffic pressure of the storage system is low in the first time period, and the performance requirements such as IOPS for the host are low, even if the storage system performs garbage collection in the first time period, the performance of the host in the first time period is enough to meet the front-end traffic demand. For example, assuming that the maximum IOPS of the storage device can reach 10000, and in a first time period, the front-end service only requires that the IOPS of the host reaches 1000, in general, even if the storage system performs garbage collection in the first time period and even executes a large number of concurrent garbage collection processes, the maximum IOPS that the host can reach in the first time period is not reduced below 1000, for example, only reduced to 3000 at most, and therefore, the IOPS of the host can always meet the service requirement of the front-end in the first time period.
Therefore, the storage system can perform garbage collection in the first time period, and release the blank logic block group, so as to increase the available storage space of the storage system. Therefore, when the storage system is in other time periods, for example, in a second time period with a larger predicted service pressure, because the storage system has more available storage space, the storage system can support the storage system to write a large amount of data quickly, and the concurrent requirement of garbage collection of the storage system in the second time period is also reduced, so that the influence of the garbage collection on the performance of the host, such as the IOPS (input/output system) and the like, can be reduced or even avoided in the second time period.
As an example, the storage system may perform garbage collection at a preset garbage collection rate during the first period of time. Of course, the rate of garbage collection in the first time period by the storage system enables the performance requirements of the front-end service in the first time period to be still met by the performance of the host such as IOPS, IO size, read-write ratio, deduplication compression ratio, throughput and the like.
In yet another possible example, the garbage collection rate of the storage system in the first time period may also be determined according to a traffic pressure value of the storage system in the second time period. Specifically, the storage system may determine, according to the traffic pressure value of the storage system in the historical time period, the traffic pressure value of the storage system in the second time period, and then determine, according to the traffic pressure value of the storage system in the second time period, the garbage collection rate of the storage system in the first time period, so that the storage system may perform garbage collection on the storage system based on the determined garbage collection rate. For example, when the service pressure value of the storage system in the second time period is larger, or the duration of the first time period is shorter, the storage system may perform garbage collection on the storage system at a larger garbage collection rate; and when the difference between the service pressure value of the storage system in the first time period and the service pressure value of the storage system in the second time period does not exceed the preset pressure threshold, or the duration of the first time period is longer, the storage system can perform garbage collection on the storage system at a smaller garbage collection rate. Of course, the above example is only used as an exemplary illustration for determining the garbage collection rate, and the garbage collection rate of the storage system in the first time period may also be determined in other manners, which is not limited in this embodiment.
In addition to determining the rate at which garbage collection occurs during the first time period, the storage system may also determine which data (hereinafter referred to as target data) in the storage system to garbage collect. In an exemplary embodiment of determining the target data, the storage system may predict the failure time of each valid data in the logical block group, so that the data with the later failure time may be used as the target data for garbage collection. Specifically, the storage system may use data whose expiration time is later than a preset time in the second time period as the target data. The preset time may be the starting time of the second time period, or a time that is a certain time away from the starting time, for example, the preset time may be a time five minutes after the starting time of the second time period. Therefore, in the first time period, the storage system can not perform garbage collection on the data which is invalid before the preset time, and can perform garbage collection on the data which is invalid after the preset time, so that partial data of the storage system which is subjected to garbage collection in the first time period can be reduced as far as possible, the data is invalid when entering the second time period, and invalid write amplification is reduced.
On this basis, the storage system may further determine the garbage collection order of the logic block group according to the failure time of the target data, and of course, the logic block group includes the target data whose failure time is later than the preset time. When garbage collection is performed, garbage collection can be performed on the logic block groups according to a sequence of garbage levels from low to high, specifically, garbage collection can be preferentially performed on the logic block groups with a low garbage level, for example, when a storage system performs garbage collection on the logic block group a and the logic block group B, since only one logic block on the logic block group a stores valid data, a plurality of logic blocks on the logic block group B store valid data, the storage system can preferentially perform garbage collection on the logic block group a, and when a garbage collection process needs to be continuously performed, the storage system can continue to perform garbage collection on the valid data on the logic block group B. Therefore, the storage system can release a blank logic block group based on smaller garbage collection cost, so that the efficiency of releasing the blank logic block group is improved, and the data volume of garbage collection can be reduced to a certain extent, thereby reducing the abrasion to the storage medium. For the logic blocks with the same or similar garbage level (for example, the logic blocks with the valid data in the logic blocks have similar occupation ratios), the garbage collection sequence can be determined according to the failure time of the valid data. For example, when the storage system performs garbage collection on a logic block group C and a logic block group D having similar garbage levels, the target data to be collected includes first data located on the logic block group C and second data located on the logic block group D, and since the failure time of the first data on the logic block group C is later than the failure time of the second data on the logic block group D, the storage system may preferentially perform garbage collection on the first data, that is, preferentially perform garbage collection on the logic block group C, and when the garbage collection process needs to be continuously performed, the storage system may continuously perform garbage collection on valid data on the logic block group D. It should be understood that the above garbage collection order of the storage system for the logic block group is only an exemplary illustration, and is not used to limit the order in which the storage system collects valid data on the logic block group to the above example, and this embodiment does not limit this.
For example, the preset time may be set based on a service pressure value of the storage system in the second time period. For example, when the service pressure value of the storage system in the second time period is smaller, the storage system recovers the data that is not invalidated after the starting time of the second time period in the first time period, so that the available storage space of the storage system can already meet the data writing requirement of the storage system in the second time period, and at this time, the preset time may be the starting time of the second time period. When the amateur pressure value of the storage system in the second time period is larger, the storage system may need to release more blank logic chunks, and at this time, the preset time may be later than the start time of the second time period, so that the storage system may not perform garbage collection when invalid data occurs between the start time of the second time period and the preset time. For example, the larger the service pressure value of the storage system in the second time period is, the larger the time interval between the preset time and the starting time of the second time period is, that is, the later the preset time is, the later the failure time of the target data for garbage collection is correspondingly.
Optionally, when the storage system writes data, the blank logic block group may be preferentially used to store newly written data, and when all blank logic block groups in the storage system are full of data, data may be stored continuously based on the logic block group containing valid data that is about to fail, where the valid data that is about to fail refers to data that is not to fail at the current time but fails before the preset time is reached. Because the valid data in the partial logic block group is about to fail, even if the partial valid data is subjected to garbage collection, the situation that the partial valid data is just subjected to garbage collection and fails can occur, and invalid write amplification is caused, so that the logic block group containing the valid data which is about to fail can be free from garbage collection, and after all blank logic block groups are full of data, the newly written data can be continuously stored by using the logic block group. Of course, when storing data based on a logical block group containing partial valid data, the logical block group may be erased first to release a blank logical block group, and then the data is stored based on the blank logical block group.
The storage system may predict the life cycle of the data on each logic block based on the deep learning network, and may determine the failure time of the data on each logic block. As an example, the storage system may determine how often (or how often) it has accessed the stored data over a period of time in the past, the modified time interval, and other characteristics, and then input these characteristics into a pre-completion training deep learning network, such as a Long Short-Term Memory (LSTM) network, and output by the deep learning network a lifecycle of the data on each logical block to determine the time of failure of the data based on the lifecycle of the data and the time it was written to the storage system.
In yet other possible examples, the storage system may predict the life cycle of the data currently stored on a logical block based on the life cycle of historical data on the logical block, thereby determining the time of failure of the data on each logical block. Specifically, for any logic block, the storage system may obtain a time interval at which each piece of data stored in the logic block in the past is modified, that is, a life cycle of each piece of data stored in the logic block, so as to calculate the life cycle of the data currently stored in the logic block by a weighted averaging method. The weighted values corresponding to the data historically stored in the logic block may be the same or different, and this embodiment does not limit this. In this way, the storage system can predict the life cycle of valid data currently stored on each logical block for the data on the logical block, so as to determine the failure time of the data based on the life cycle of the data and the time when the data is written into the storage system.
The foregoing mainly describes a related process of performing garbage collection in a storage system in a first time period, and certainly, in a part of application scenarios, the front-end service may not require that the performance of the host in a second time period reaches the highest performance that the host can reach, so the method may further include:
s303: and the storage system performs garbage collection in a second time period, wherein the garbage collection rate of the storage system in the second time period is smaller than that of the storage system in the first time period.
In a further possible embodiment, the storage system may also perform garbage collection at a garbage collection rate in the second period of time, and of course, the garbage collection rate of the storage system in the second period of time may be smaller than the garbage collection rate of the storage system in the first period of time. Because the garbage collection rate of the storage system in the second time period is low, the influence of the garbage collection process of the storage system in the second time period on the performance of the host in the storage system is low, and therefore the performance of the host in the second time period can still meet the requirement of the front-end service. Meanwhile, the storage system performs garbage collection in the second time period, and releases the blank logic block group while writing data, so that the storage system can have more available storage space for a long time in the second time period.
It should be noted that, when the storage system performs garbage collection at a low rate in the second time period, the storage system may start performing garbage collection at the low rate immediately after entering the second time period, may start performing garbage collection at the low rate after entering the second time period, or may start performing garbage collection when the available storage space of the storage system in the second time period decreases to a certain threshold, or the like. In this embodiment, the timing when the storage system starts garbage collection in the second time period is not limited.
The garbage recycling device and method of the present application are described above with reference to fig. 1 to 4, and the device of the present application is described below with reference to the accompanying drawings.
The embodiment of the application also provides a garbage recycling device. Referring to fig. 5, a schematic structural diagram of a garbage recycling apparatus in an embodiment of the present application is shown, where the apparatus 500 shown in fig. 5 may be applied to a storage system, and the apparatus 500 may include:
a determining module 501, configured to determine a first time period according to a service pressure value of the storage system in a historical time period, where the service pressure value of the storage system in the first time period is lower than a set threshold, and the service pressure value of the storage system in the historical time period is obtained according to any one or more of number of times of performing read/write operations per second IOPS, input/output IO size, data read/write ratio, deduplication compression ratio, and throughput;
a garbage collection module 502, configured to perform garbage collection within the first time period.
In a possible implementation manner, the determining module 501 is specifically configured to determine, according to the traffic pressure value of the storage system in the historical time period, an ending time of the first time period, where the ending time of the first time period is not later than a starting time of a second time period, and the set threshold is determined based on the traffic pressure value of the storage system in the second time period.
In a possible implementation, the determining module 501 is further configured to:
determining the service pressure value of the storage system in the second time period according to the service pressure value of the storage system in the historical time period;
and determining the garbage recovery rate of the storage system in the first time period according to the service pressure value of the storage system in the second time period.
In a possible implementation manner, the garbage collection module 502 is further configured to perform garbage collection in the second time period, and a garbage collection rate of the storage system in the second time period is smaller than a garbage collection rate of the storage system in the first time period.
In a possible implementation, the determining module 501 is further configured to:
and determining target data for garbage collection of the storage system, wherein the target data comprises data of which the predicted failure time is later than a preset time in the second time period, and the preset time is any time in the second time period.
In a possible implementation manner, the target data includes first data and second data, the failure time of the first data is later than that of the second data, and the storage system preferentially performs garbage collection on the first data.
The apparatus of an embodiment of the present application may correspond to performing the method described in an embodiment of the present application. Also, the above and other operations and/or functions of the respective modules in the garbage collection apparatus 500 are respectively directed to implement the corresponding flow of the respective method in fig. 3. The functions of the above modules can be referred to the description of the method embodiment shown in fig. 4. The function of each module in the garbage collection apparatus 500 can be executed by the processor 112 in the controller shown in fig. 2 calling the program in the memory 111.
Embodiments of the present application also provide a computer-readable medium having stored therein instructions, which when executed on a computer, cause the computer to perform the method of the above aspects.
Embodiments of the present application also provide a computer program product, which when run on a computer, causes the computer to perform the method of the above aspects.
It should be noted that the above-described embodiments are merely illustrative, wherein the modules described as separate parts may or may not be physically separate, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiments of the apparatus provided in the present application, the connection relationship between the modules indicates that there is a communication connection therebetween, and may be implemented as one or more communication buses or signal lines.
The terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely descriptive of the various embodiments of the application and how objects of the same nature can be distinguished.
As will be appreciated by one of ordinary skill in the art, the aforementioned computer-readable storage media include: a U-disk, a removable hard disk, a magnetic disk, an optical disk, a RAM, an SSD, or a non-volatile memory (non-volatile memory), among various non-transitory machine-readable media that may store program code.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same.
Claims (12)
1. A garbage recycling method is applied to a storage system, and comprises the following steps:
determining a first time period according to the service pressure value of the storage system in the historical time period, wherein the service pressure value of the storage system in the first time period is lower than a set threshold value, and the service pressure value of the storage system in the historical time period is obtained according to any one or more of the number of times of reading and writing operations per second IOPS, the input/output IO size, the data reading and writing proportion, the deduplication compression proportion and the throughput;
and performing garbage recovery in the first time period.
2. The method of claim 1, wherein determining the first time period based on the traffic pressure values of the storage system over the historical time period comprises:
and determining the ending time of the first time period according to the service pressure value of the storage system in the historical time period, wherein the ending time of the first time period is not later than the starting time of a second time period, and the set threshold is determined based on the service pressure value of the storage system in the second time period.
3. The method of claim 2, further comprising:
determining the service pressure value of the storage system in the second time period according to the service pressure value of the storage system in the historical time period;
and determining the garbage recovery rate of the storage system in the first time period according to the service pressure value of the storage system in the second time period.
4. The method of claim 3, further comprising:
and performing garbage collection in the second time period, wherein the garbage collection rate of the storage system in the second time period is smaller than the garbage collection rate of the storage system in the first time period.
5. The method according to any one of claims 2 to 4, further comprising:
and determining target data for garbage collection of the storage system, wherein the target data comprises data of which the predicted failure time is later than a preset time in the second time period, and the preset time is any time in the second time period.
6. The method of claim 5, wherein the target data comprises first data and second data, wherein the first data has a time of failure later than a time of failure of the second data, and wherein the storage system preferentially performs garbage collection on the first data.
7. A waste recycling apparatus, the apparatus being used in a storage system, the apparatus comprising:
the determining module is used for determining a first time period according to the service pressure value of the storage system in the historical time period, wherein the service pressure value of the storage system in the first time period is lower than a set threshold value, and the service pressure value of the storage system in the historical time period is obtained according to any one or more of the number of times of read-write operation per second IOPS, the input/output IO size, the data read-write proportion, the deduplication compression proportion and the throughput;
and the garbage recycling module is used for recycling garbage in the first time period.
8. The apparatus of claim 7, wherein the determining module is configured to determine an ending time of the first time period according to a traffic pressure value of the storage system in the historical time period, the ending time of the first time period is not later than a starting time of a second time period, and the set threshold is determined based on a traffic pressure value of the storage system in the second time period.
9. The apparatus of claim 8, wherein the determining module is further configured to:
determining the service pressure value of the storage system in the second time period according to the service pressure value of the storage system in the historical time period;
and determining the garbage recovery rate of the storage system in the first time period according to the service pressure value of the storage system in the second time period.
10. The apparatus of claim 9, wherein the garbage collection module is further configured to collect garbage during the second time period, and a garbage collection rate of the storage system during the second time period is smaller than a garbage collection rate of the storage system during the first time period.
11. The apparatus of any one of claims 8 to 10, wherein the determining module is further configured to:
and determining target data for garbage collection of the storage system, wherein the target data comprises data of which the predicted failure time is later than a preset time in the second time period, and the preset time is any time in the second time period.
12. The apparatus of claim 11, wherein the target data comprises first data and second data, wherein the first data has a failure time later than a failure time of the second data, and wherein the storage system preferentially performs garbage collection on the first data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010711871.XA CN113971137A (en) | 2020-07-22 | 2020-07-22 | Garbage recovery method and device |
| PCT/CN2021/097668 WO2022017002A1 (en) | 2020-07-22 | 2021-06-01 | Garbage collection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010711871.XA CN113971137A (en) | 2020-07-22 | 2020-07-22 | Garbage recovery method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113971137A true CN113971137A (en) | 2022-01-25 |
Family
ID=79584912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010711871.XA Pending CN113971137A (en) | 2020-07-22 | 2020-07-22 | Garbage recovery method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113971137A (en) |
| WO (1) | WO2022017002A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114895846A (en) * | 2022-04-28 | 2022-08-12 | 阿里巴巴（中国）有限公司 | Data processing method, device and equipment |
| CN115269451A (en) * | 2022-09-28 | 2022-11-01 | 珠海妙存科技有限公司 | Flash memory garbage recycling method and device and readable storage medium |
| CN115904264A (en) * | 2023-03-13 | 2023-04-04 | 浪潮电子信息产业股份有限公司 | Method, device and medium for recovering garbage data of blocks in storage system |
| CN116700634A (en) * | 2023-08-08 | 2023-09-05 | 苏州浪潮智能科技有限公司 | Garbage recycling method and device for distributed storage system and distributed storage system |
| CN117806837A (en) * | 2024-02-29 | 2024-04-02 | 山东云海国创云计算装备产业创新中心有限公司 | Method, device, storage medium and system for managing hard disk tasks |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114968836A (en) * | 2022-04-19 | 2022-08-30 | 华为技术有限公司 | Garbage recycling method and electronic equipment |
| US11947452B2 (en) * | 2022-06-01 | 2024-04-02 | Micron Technology, Inc. | Controlling variation of valid data counts in garbage collection source blocks |
| CN115934579A (en) * | 2022-12-23 | 2023-04-07 | 深圳大普微电子科技有限公司 | Solid state disk garbage recycling method, accelerating device, equipment and computer medium |
| CN115826886B (en) * | 2023-02-24 | 2023-05-12 | 浪潮电子信息产业股份有限公司 | Data garbage collection method, device and system in additional write mode and storage medium |
| CN116662214B (en) * | 2023-05-06 | 2024-05-07 | 深圳市晶存科技有限公司 | Hard disk garbage recycling method, device, system and medium based on fio |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102147767B (en) * | 2011-04-26 | 2014-04-02 | 记忆科技(深圳)有限公司 | Method and system for dynamically adjusting solid state disk garbage collection parameters |
| US8799561B2 (en) * | 2012-07-27 | 2014-08-05 | International Business Machines Corporation | Valid page threshold based garbage collection for solid state drive |
| CN105630638B (en) * | 2014-10-31 | 2018-01-12 | 国际商业机器公司 | For the apparatus and method for disk array distribution caching |
| CN107506136B (en) * | 2017-08-07 | 2020-07-07 | 成都华为技术有限公司 | Garbage recycling method and device |
| CN109284233B (en) * | 2018-09-18 | 2022-02-18 | 郑州云海信息技术有限公司 | Garbage recovery method of storage system and related device |
| CN109558244A (en) * | 2018-12-03 | 2019-04-02 | 郑州云海信息技术有限公司 | Junk information recovery method, device, equipment and computer readable storage medium |
| CN110764714B (en) * | 2019-11-06 | 2021-07-27 | 深圳大普微电子科技有限公司 | Data processing method, device and equipment and readable storage medium |
-
2020
- 2020-07-22 CN CN202010711871.XA patent/CN113971137A/en active Pending
-
2021
- 2021-06-01 WO PCT/CN2021/097668 patent/WO2022017002A1/en active Application Filing
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114895846A (en) * | 2022-04-28 | 2022-08-12 | 阿里巴巴（中国）有限公司 | Data processing method, device and equipment |
| WO2023207562A1 (en) * | 2022-04-28 | 2023-11-02 | 阿里巴巴（中国）有限公司 | Data processing method and apparatus, and device |
| CN115269451A (en) * | 2022-09-28 | 2022-11-01 | 珠海妙存科技有限公司 | Flash memory garbage recycling method and device and readable storage medium |
| CN115904264A (en) * | 2023-03-13 | 2023-04-04 | 浪潮电子信息产业股份有限公司 | Method, device and medium for recovering garbage data of blocks in storage system |
| CN116700634A (en) * | 2023-08-08 | 2023-09-05 | 苏州浪潮智能科技有限公司 | Garbage recycling method and device for distributed storage system and distributed storage system |
| CN116700634B (en) * | 2023-08-08 | 2023-11-03 | 苏州浪潮智能科技有限公司 | Garbage recycling method and device for distributed storage system and distributed storage system |
| CN117806837A (en) * | 2024-02-29 | 2024-04-02 | 山东云海国创云计算装备产业创新中心有限公司 | Method, device, storage medium and system for managing hard disk tasks |
| CN117806837B (en) * | 2024-02-29 | 2024-06-07 | 山东云海国创云计算装备产业创新中心有限公司 | Method, device, storage medium and system for managing hard disk tasks |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022017002A1 (en) | 2022-01-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022017002A1 (en) | Garbage collection method and device | |
| US10884630B2 (en) | Storage system | |
| US7127557B2 (en) | RAID apparatus and logical device expansion method thereof | |
| CN109358809B (en) | RAID data storage system and method | |
| EP2646919B1 (en) | Transaction log recovery | |
| US20150212752A1 (en) | Storage system redundant array of solid state disk array | |
| KR20200113047A (en) | Operation method of open-channel storage device | |
| CN105469829A (en) | Storage device health diagnosis | |
| US11321229B2 (en) | System controller and system garbage collection method | |
| CN103838676B (en) | Data-storage system, date storage method and PCM bridges | |
| US20190243758A1 (en) | Storage control device and storage control method | |
| US20240231645A1 (en) | Storage device, data storage method, and storage system | |
| US20230138586A1 (en) | Storage device and method of operating the same | |
| CN110688323B (en) | System controller and system garbage recycling method | |
| JP2019086974A (en) | Storage system and control method of the same | |
| US11416403B2 (en) | Method and apparatus for performing pipeline-based accessing management in storage server with aid of caching metadata with hardware pipeline module during processing object write command | |
| US20140372672A1 (en) | System and method for providing improved system performance by moving pinned data to open nand flash interface working group modules while the system is in a running state | |
| CN117075810A (en) | ZNS solid state disk-based data management method | |
| CN114008580A (en) | Hard disk control method and related equipment | |
| CN111338846B (en) | Method and device for recovering L2P (L2P) table based on multiple cores | |
| CN103064762A (en) | Method and device for recovering deleted backup data | |
| WO2018055686A1 (en) | Information processing system | |
| CN108536619B (en) | Method and device for rapidly recovering FTL table | |
| JP7205090B2 (en) | Control device, control program, and control method | |
| JP2017134700A (en) | Information processing system, storage control device, storage control method and storage control program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |