CN112988612B - Data processing method, device, equipment and readable storage medium - Google Patents

Data processing method, device, equipment and readable storage medium Download PDF

Info

Publication number
CN112988612B
CN112988612B CN201911310534.3A CN201911310534A CN112988612B CN 112988612 B CN112988612 B CN 112988612B CN 201911310534 A CN201911310534 A CN 201911310534A CN 112988612 B CN112988612 B CN 112988612B
Authority
CN
China
Prior art keywords
data
target
physical
period
update period
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911310534.3A
Other languages
Chinese (zh)
Other versions
CN112988612A (en
Inventor
王岩
李法豪
李卫军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Dapu Microelectronics Co Ltd
Original Assignee
Shenzhen Dapu Microelectronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Dapu Microelectronics Co Ltd filed Critical Shenzhen Dapu Microelectronics Co Ltd
Priority to CN201911310534.3A priority Critical patent/CN112988612B/en
Priority to PCT/CN2020/115543 priority patent/WO2021120731A1/en
Publication of CN112988612A publication Critical patent/CN112988612A/en
Application granted granted Critical
Publication of CN112988612B publication Critical patent/CN112988612B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application discloses a data processing method, a device, equipment and a readable storage medium. The method disclosed by the application comprises the following steps: receiving a garbage collection instruction; determining a physical data page corresponding to the garbage collection instruction; inquiring a logical address corresponding to the physical data page; increasing the update period of the logic address to obtain a target update period; the update period represents the liveness of the data in the physical data page; calculating the difference between the target updating period and the preset period of each group; and migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference degree. The method and the device can realize separate storage of cold and hot data, reduce abrasion of the storage medium and improve service life and performance of the storage medium. Accordingly, the data processing device, the data processing equipment and the readable storage medium disclosed by the application also have the technical effects.

Description

Data processing method, device, equipment and readable storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium.
Background
In the solid state disk and Nand Flash storage medium, the space occupied by invalid data pages can be released through garbage collection, so that the recovery of available space is realized. The garbage collection operation is to erase a block, the erase block can bring about migration of effective data pages, when the effective data pages are more, the data writing quantity is increased (namely, writing amplification) and the abrasion of a storage medium is aggravated, and the service life and performance of the storage medium are reduced.
Taking a solid state disk as an example, the solid state disk comprises a plurality of physical blocks, and each physical block comprises a plurality of data pages. If the memory space is released by erasing the block, the data in the valid data page in the current physical block needs to be migrated to the other physical block in order to release the memory space of the current physical block. If the number of valid data pages in the current physical block is large, the data migration amount and the writing amount are large, so that the performance of the solid state disk is directly affected.
Therefore, how to reduce wear of the storage medium and improve the life and performance of the storage medium is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
In view of the foregoing, it is an object of the present application to provide a data processing method, apparatus, device, and readable storage medium, so as to reduce wear of the storage medium and improve the lifetime and performance of the storage medium. The specific scheme is as follows:
In a first aspect, the present application provides a data processing method, including:
receiving a garbage collection instruction;
determining a physical data page corresponding to the garbage collection instruction;
inquiring a logic address corresponding to the physical data page;
increasing the update period of the logic address to obtain a target update period; the update period represents the liveness of the data in the physical data page;
calculating the difference between the target updating period and the preset period of each group;
and migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference degree.
Preferably, said increasing the update period of the logical address, obtaining a target update period, includes:
doubling the update period to obtain the target update period.
Preferably, after migrating the data in the physical data page to the physical block corresponding to the packet corresponding to the smallest difference degree, the method further includes:
acquiring I/O data corresponding to the logical address;
determining an operation I/O time stamp corresponding to the I/O data;
judging whether the record I/O time stamp is an initial time stamp or not;
if not, updating the update period of the logic address to be the difference value between the operation I/O timestamp and the record I/O timestamp;
Updating the record I/O timestamp to the operation I/O timestamp;
calculating the difference between the update period and the preset period of each group;
and storing the I/O data to a target physical block corresponding to a target packet corresponding to the minimum difference degree.
Preferably, the method further comprises:
if the record I/O time stamp is the initial time stamp, judging whether the update period is an initial period or not;
if yes, updating the record I/O time stamp to the operation I/O time stamp, and updating the updating period to the minimum preset period;
and storing the I/O data to an object physical block corresponding to the object grouping corresponding to the minimum preset period.
Preferably, the storing the I/O data to a target physical block corresponding to a target packet corresponding to a minimum degree of difference includes:
and writing the I/O data into a target data page corresponding to the target group, wherein the target data page is a physical data page in the target physical block.
Preferably, the method further comprises:
and updating the preset period of the target packet to be the average value of the update periods of all the logic addresses corresponding to the target packet.
Preferably, the method further comprises:
Judging whether the number of the data pages corresponding to the target group is twice the number of the data pages corresponding to other groups;
if yes, closing the group with the least data page number in all groups, splitting the target group into two groups, and establishing the corresponding relation between the two groups and the target physical block.
In a second aspect, the present application provides a data processing apparatus comprising:
the acquisition module is used for acquiring garbage collection instructions;
the determining module is used for determining a physical data page corresponding to the garbage collection instruction;
the inquiry module is used for inquiring the logical address corresponding to the physical data page;
the increasing module is used for increasing the update period of the logic address to obtain a target update period; the update period represents the liveness of the data in the physical data page;
the calculating module is used for calculating the difference between the target updating period and the preset period of each group;
and the migration module is used for migrating the data in the physical data page to the physical block corresponding to the grouping corresponding to the minimum difference degree.
In a third aspect, the present application provides a data processing apparatus comprising: a storage unit and a main controller, wherein:
The storage unit comprises a physical block, wherein a physical data page in the physical block is used for storing I/O data;
the main controller is used for executing the data processing method disclosed in the foregoing.
In a fourth aspect, the present application provides a readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the previously disclosed data processing method.
According to the scheme, the application provides a data processing method, which comprises the following steps: receiving a garbage collection instruction; determining a physical data page corresponding to the garbage collection instruction; inquiring a logic address corresponding to the physical data page; increasing the update period of the logic address to obtain a target update period; the update period represents the liveness of the data in the physical data page; calculating the difference between the target updating period and the preset period of each group; and migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference degree.
In the application, the logic address corresponds to an update period, the update period represents the activity of data in a physical data page corresponding to the current logic address, and the shorter the update period is, the more active the data is; the longer the update period, the less active the data. If garbage collection operation is executed on the logical address (namely, data in a physical data page corresponding to the logical address is erased), increasing the update period of the logical address to obtain a target update period; calculating the difference between the target updating period and the preset period of each group; and migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference degree. Ideally, garbage collection operation only migrates a small amount of data, so when garbage collection operation is executed, the update period of the logical address is increased, and the activity of the currently migrated data can be reduced, so that the currently migrated data and other data (data with similar activity to the currently migrated data) are stored in the same physical block, and separate storage of cold and hot data is realized. In this way, if the memory space is released by erasing the block, the effective data page remained in the physical block is greatly reduced, so that the data writing amount and the abrasion of the memory medium can be reduced, and the service life and the performance of the memory medium can be improved.
After the separate storage of the cold and hot data is realized, if the garbage collection operation is executed again, the data volume to be migrated is greatly reduced, so that the execution efficiency of the garbage collection operation can be improved.
Accordingly, the data processing device, the device and the readable storage medium provided by the application also have the technical effects.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings may be obtained according to the provided drawings without inventive effort to a person skilled in the art.
FIG. 1 is a flow chart of a first data processing method disclosed in the present application;
FIG. 2 is a flow chart of a second data processing method disclosed in the present application;
FIG. 3 is a flow chart of a third data processing method disclosed in the present application;
FIG. 4 is a schematic diagram of a data processing apparatus disclosed herein;
fig. 5 is a schematic diagram of a data processing apparatus disclosed in the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
At present, when the storage space is released by erasing the block in the prior art, the data migration quantity and the writing quantity are relatively large, so that the abrasion of the storage medium is increased, and the service life and the performance of the storage medium are reduced. Therefore, the application provides a data processing scheme which can reduce the abrasion of the storage medium and improve the service life and the performance of the storage medium.
Referring to fig. 1, an embodiment of the present application discloses a first data processing method, including:
s101, receiving a garbage collection instruction.
S102, determining a physical data page corresponding to the garbage collection instruction.
S103, inquiring the logical address corresponding to the physical data page.
Specifically, the garbage collection instruction carries a physical address of a physical data page, so that the physical address can be obtained from the garbage collection instruction, and then the logical address is determined according to a mapping table of the physical address and the logical address.
S104, increasing the update period of the logic address to obtain a target update period.
Wherein the update period represents the liveness of the data in the physical data page. The shorter the update period, the more active the data; the longer the update period, the less active the data. Active data may be considered hot data and inactive data may be considered cold data.
S105, calculating the difference between the target updating period and the preset period of each group.
S106, migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference degree.
In a preferred embodiment, increasing the update period of the logical address to obtain the target update period includes: doubling the update period to obtain the target update period. For example: the original update period is 3, and after the update period is doubled, the obtained target update period is 3+3=6. After the target updating period is obtained, the difference degree between the target updating period and the preset period of each group can be calculated, and the data in the physical data page is migrated to the physical block corresponding to the group corresponding to the minimum difference degree, so that the separate storage of cold and hot data can be realized.
If an effective physical data page exists in a certain physical block, the effective physical data page needs to be migrated when garbage collection is performed; if one physical block is garbage-recovered, the number of valid physical data pages to be migrated is large, which indicates inaccurate grouping. Thus, once garbage is recovered, the valid physical data page needs to be moved, indicating that the grouping of the valid physical data page is inaccurate, requiring punishment to readjust the grouping. Therefore, in this embodiment, the valid physical data page to be migrated is determined by the garbage collection instruction, then the logical address corresponding to the valid physical data page is queried by the physical address of the valid physical data page, and finally the penalty adjustment packet is performed by increasing the update period of the logical address.
In this embodiment, the logical address corresponds to not only the update period, but also the record I/O timestamp. The record I/O timestamp may be used to determine the update period.
It should be noted that, in this embodiment, a plurality of packets are preset, each packet corresponds to a preset period, and each packet has a corresponding relationship with a physical block. If the current storage medium is a solid state disk, the number of the packets is not greater than the parallelism of the solid state disk, and the parallelism refers to the maximum number of physical blocks that can be written at the same time. The preset period corresponding to each packet represents the activity of the written data, so that if the difference between the update period of the logical address and the preset period of the packet is smaller, the activity of the I/O data to be written is similar to that of the written data.
For example: the parallelism of the solid state disk is 3, and at most 3 groups can be preset: a first packet, a second packet, and a third packet. Assume that the current preset period of the first packet is 8ms, the current preset period of the second packet is 16ms, and the current preset period of the third packet is 32ms. After the solid state disk is initialized, the first preset period of the grouping can be manually determined according to experience or the rule of the service supported by the current solid state disk. Assuming that the updated update period corresponding to the logical address is 16ms, the difference between the update period and the preset period of the second packet is the smallest, so that the I/O data corresponding to the logical address is stored in the physical block corresponding to the second packet.
After the separate storage of the cold and hot data is realized, if the garbage collection operation is executed again, the data volume to be migrated is greatly reduced, so that the execution efficiency of the garbage collection operation can be improved.
It can be seen that in the embodiment of the present application, the logical address corresponds to an update period. If garbage collection operation is performed on the logical address (namely, data in a physical data page corresponding to the logical address is erased), the update period of the logical address is increased, so that the activity of the currently migrated data is reduced, and the currently migrated data and other data (data with similar activity to the currently migrated data) are stored in the same physical block, so that separate storage of cold and hot data is realized. In this way, if the memory space is released by erasing the block, the effective data page remained in the physical block is greatly reduced, so that the data writing amount and the abrasion of the memory medium can be reduced, and the service life and the performance of the memory medium can be improved.
Based on the above embodiment, it should be noted that, if the I/O data of the logical address corresponding to the garbage collection operation described in the above embodiment is obtained after the garbage collection operation is performed, the I/O data may be processed according to the method provided in the present embodiment.
Referring to fig. 2, the embodiment of the application discloses a second data processing method, which includes:
s201, acquiring I/O data corresponding to a logic address of the garbage collection operation, and determining an operation I/O time stamp corresponding to the I/O data.
The I/O data may be data corresponding to a host write operation, or may be data corresponding to a garbage collection operation (i.e., data in a valid data page that needs to be migrated).
S202, judging whether a record I/O time stamp of a logic address is an initial time stamp; if yes, then execute S207; if not, S203 is performed.
S203, updating the update period of the logic address to be the difference value between the operation I/O time stamp and the record I/O time stamp.
S204, updating the record I/O time stamp to be the operation I/O time stamp.
S205, respectively calculating the difference between the update period and the preset period of each group.
And when the difference between the update period and the preset period is calculated, the updated update period is used. For example: if the update period is updated from a to B in step S203, the difference between B and the preset period of each packet is calculated in step S205, so as to obtain a plurality of differences.
S206, storing the I/O data to a target physical block corresponding to a target packet corresponding to the minimum difference degree.
In a preferred embodiment, storing the I/O data to a target physical block corresponding to a target packet corresponding to a minimum degree of difference includes: and writing the I/O data into a target data page corresponding to the target group, wherein the target data page is a physical data page in a target physical block.
Specifically, storing the I/O data to a target physical block corresponding to a target packet corresponding to a minimum degree of difference includes: determining a target group corresponding to the minimum difference degree; the current logical address is marked so that the current logical address corresponds to the target packet, and the I/O data is written into a target data page corresponding to the target packet. The corresponding relation between the logic address and the grouping changes along with the change of the updating period and the preset period, so that the corresponding relation between the logic address and the grouping changes in real time.
It should be noted that, the preset period corresponding to the packet is updated according to the update of the update period of the corresponding logical address. For example: the current preset period of a certain group is 8, and the difference between the update period of the logical address corresponding to the I/O data and the preset period 8 is the smallest, so that after the current I/O data is written into the physical block corresponding to the group, the preset period of the current group can be updated according to the update period of the current logical address. In a preferred embodiment, the preset period may be updated to be an average value of update periods of all logical addresses corresponding to the current packet.
In this embodiment, the preset period of the target packet may be updated to be an average value of update periods of all logical addresses corresponding to the target packet.
In a preferred embodiment, after storing the I/O data in the target physical block corresponding to the target packet corresponding to the minimum preset period, the method further includes a packet splitting step. The packet splitting step specifically includes: judging whether the number of the data pages corresponding to the target group is twice the number of the data pages corresponding to other groups; if yes, closing the packets with the least data pages in all the packets, splitting the target packet into two packets, and establishing the corresponding relation between the two packets and the target physical block.
In a preferred embodiment, before storing the I/O data to the target physical block corresponding to the target packet corresponding to the minimum preset period, the method further includes a packet splitting step. The packet splitting step specifically includes: judging whether the number of the data pages corresponding to the target group is twice the number of the data pages corresponding to other groups; if yes, closing the packets with the least data pages in all the packets, splitting the target packet into two packets, and establishing the corresponding relation between the two packets and the target physical block.
That is, when the number of data pages corresponding to the target packet is large, it is split into 2 packets. If the number of the current groups is equal to the parallelism of the solid state disk, in order to avoid that the number of the groups is greater than the parallelism of the solid state disk, the group with the least number of data pages in all the groups is closed. From the above, it can be seen that: when the packet splitting condition is satisfied, the splitting step of the packet may be performed before or after the I/O data is stored in the physical block.
According to the core idea of the splitting step described above, the packet splitting step may comprise:
judging whether the number of the data pages corresponding to the target group is N times of the number of the data pages corresponding to other groups; n is a real number and not less than 2; if yes, ordering all the packets from small to large according to the number of the corresponding data pages, obtaining a packet sequence, and deleting the first N-1 packets in the packet sequence; splitting the target packet into N-1 sub-packets, and establishing a corresponding relation between the N-1 sub-packets and the target physical block.
S207, judging whether the update period is an initial period; if yes, executing S208; if not, the update period is reserved and S204 is performed.
S208, updating the record I/O time stamp to be the operation I/O time stamp, and updating the updating period to be the minimum preset period.
S209, storing the I/O data to an object physical block corresponding to an object group corresponding to the minimum preset period.
It should be noted that, if the update period of the current logical address and the record I/O timestamp are both initial values, which indicates that the logical address has never been written with data, in this case, it cannot be determined exactly which preset period has the smallest difference between the current update period and the current update period, so that the I/O data is directly stored in the physical block of the object corresponding to the object packet corresponding to the smallest preset period. The update period of the minimum preset period is shortest, the possibility of garbage collection is greatest in a short time, and whether the grouping is accurate or not can be exposed as early as possible by enabling the I/O data to be garbage collected in advance, if not, grouping adjustment is performed in time (namely, the update period of the logic address is updated, so that the data is stored to a physical block which is closer to the heat of the data when the data is written on the logic address next time). The minimum preset period is the minimum value in all preset periods at present. Since the preset period of each packet varies in real time, the minimum preset period also varies in real time.
Wherein the object packet and the target packet may be the same packet or two different packets. The object physical block and the target physical block may be the same physical block or two different physical blocks.
In a preferred embodiment, storing the I/O data to an object physical block corresponding to an object group corresponding to a minimum preset period includes: and writing the I/O data into an object data page corresponding to the object group, wherein the object data page is a physical data page in the object physical block.
In this embodiment, the preset period of the object group may be updated to be an average value of update periods of all logical addresses corresponding to the object group.
In a preferred embodiment, after storing the I/O data in the object physical block corresponding to the object packet corresponding to the minimum preset period, the method further includes a packet splitting step. The packet splitting step specifically includes: judging whether the number of the data pages corresponding to the object groups is twice the number of the data pages corresponding to other groups; if yes, closing the group with the least data page number in all groups, splitting the object group into two groups, and establishing the corresponding relation between the two groups and the object physical block.
In a preferred embodiment, before storing the I/O data to the object physical block corresponding to the object packet corresponding to the minimum preset period, the method further includes a packet splitting step. The packet splitting step specifically includes: judging whether the number of the data pages corresponding to the object groups is twice the number of the data pages corresponding to other groups; if yes, closing the group with the least data page number in all groups, splitting the object group into two groups, and establishing the corresponding relation between the two groups and the object physical block.
That is, when the number of data pages corresponding to the object group is large, it is split into 2 groups. If the number of the current groups is equal to the parallelism of the solid state disk, in order to avoid that the number of the groups is greater than the parallelism of the solid state disk, the group with the least number of data pages in all the groups is closed. From the above, it can be seen that: when the packet splitting condition is satisfied, the splitting step of the packet may be performed before or after the I/O data is stored in the physical block.
According to the core idea of the splitting step described above, the packet splitting step may comprise:
judging whether the number of the data pages corresponding to the object groups is N times of the number of the data pages corresponding to other groups; n is a real number and not less than 2; if yes, ordering all the packets from small to large according to the number of the corresponding data pages, obtaining a packet sequence, and deleting the first N-1 packets in the packet sequence; splitting the object group into N-1 sub-groups, and establishing a corresponding relation between the N-1 sub-groups and the object physical block.
In a preferred embodiment, before determining whether the record I/O timestamp of the logical address is the initial timestamp, further comprising: the address range to which the logical address belongs is determined, and the record I/O time stamp and the update period of the address range are determined as the record I/O time stamp and the update period of the logical address.
It should be noted that, the number of logical addresses is large, if each logical address corresponds to one record I/O timestamp and one update period, the record I/O timestamp and the update period to be maintained will be very large, which is unfavorable for implementation of the scheme and increases the calculation overhead. For this reason, the logical addresses may be segmented, and after segmentation, a plurality of logical addresses belonging to the same address segment (i.e. address range) will correspond to the same record I/O timestamp and the same update period, thereby reducing the computation overhead.
It can be seen that if the logical address of the garbage collection operation has I/O data to be written, then determining whether the recorded I/O timestamp of the logical address is an initial timestamp; if not, the recorded I/O time stamp and the update period of the logical address are updated, the difference between the update period and the preset period of each group is further determined, and the current I/O data is stored to a target physical block corresponding to a target group corresponding to the minimum difference, so that the data with similar activity are stored in the same physical block, and the separate storage of cold and hot data is realized. In this way, if the memory space is released by erasing the block, the effective data page remained in the physical block is greatly reduced, so that the data writing amount and the abrasion of the memory medium can be reduced, and the service life and the performance of the memory medium can be improved.
Based on the data processing method shown in fig. 2, if a Trim operation instruction is received, a third data processing method shown in fig. 3 may be implemented, including:
s301, if a Trim operation instruction is received, determining a logic address according to the Trim operation instruction.
Specifically, the Trim operation instruction carries a logical address, so that the logical address can be obtained from the Trim operation instruction to determine the logical address.
S302, updating the update period of the logic address to be the difference value between the instruction time stamp of the Trim operation instruction and the record I/O time stamp of the logic address.
Wherein the update period indicates the liveness of the data stored by the logical address. The shorter the update period, the more active the data; the longer the update period, the less active the data.
S303, updating the record I/O time stamp of the logic address to be an initial time stamp.
S304, if the I/O data corresponding to the logical address is obtained, the I/O data is stored according to the update period and the record I/O time stamp.
In this embodiment, the logical address corresponds to a record I/O timestamp and an update period. The record I/O timestamp may be used to determine the update period.
After the record I/O time stamp and the update period of the logical address corresponding to the Trim operation are updated, the I/O data on the logical address can be stored according to the update period and the record I/O time stamp. The shorter the update period, the more active the data; the longer the update period, the less active the data. Active data may be considered hot data and inactive data may be considered cold data. Therefore, the data can be stored according to the update period and the record I/O time stamp, and the cold and hot data can be stored separately.
After the cold and hot data are stored separately, if Trim operation needs to be executed, invalid data to be processed is also reduced, so that execution efficiency of Trim operation can be improved.
It can be seen that in the present embodiment, the logical address corresponds to a record I/O timestamp and an update period. If the Trim operation instruction is received, determining a logic address according to the Trim operation instruction; updating the update period of the logic address to be the difference value between the instruction time stamp of the Trim operation instruction and the record I/O time stamp of the logic address; updating the record I/O time stamp of the logic address to be an initial time stamp; and if the I/O data corresponding to the logical address is acquired, storing the I/O data according to the update period and the record I/O time stamp. In this way, if the memory space is released by erasing the block, the effective data page remained in the physical block is greatly reduced, so that the data writing amount and the abrasion of the memory medium can be reduced, and the service life and the performance of the memory medium can be improved.
With reference to the method flows shown in fig. 1, fig. 2, and fig. 3, the following schemes may be implemented, and the specific implementation process includes: an initialization section and an update section.
1. The initialization section includes: initialize LBA (Logical Block Address, logical address) lifecycle table and initialize packets.
(1) Initializing the LBA lifecycle table includes:
and establishing an LBA life cycle table in the solid state disk, wherein the LBA life cycle table comprises fields shown in table 1.
TABLE 1
Line number (r) Recording I/O time stamps Update period
1 1.244542359 2.35969464
In table 1, a row number indicates a certain address range including a plurality of logical addresses; the record I/O time stamp and the update period are corresponding to the current address range. In the present embodiment, the operation I/O timestamp of any one logical address closest to the current time is recorded as the record I/O timestamp. Of course, the average of the operation I/O timestamps of all logical addresses included in the current line number may also be recorded as the record I/O timestamp.
The initial time stamp corresponding to each line number is-1, and the initial period is 0. Since multiple logical addresses correspond to a row, a write operation may occur at any one logical address, and the record I/O time stamp and update period for that row may change.
The time interval between two write operations on the same logical address is called the update period of the current logical address, and is represented by the difference between the I/O time stamps of the two operations, and the unit is generally subtle. Of course, the time interval of the last write operation of a different logical address of the same row number is also referred to as the update period of the current logical address, and is represented by the difference of the I/O time stamps of two operations of two different logical addresses, which is generally subtle in unit. The shorter the update period, the more active the manipulated data (i.e., the hotter the data); the longer the update period, the less active the data being manipulated (i.e., the cooler the data).
The record I/O timestamp may be calculated by using a floating point number as a unit, or may be calculated by using an integer through a custom timer (for example, a number of clock cycles is one timing cycle). If floating point numbers are utilized, 64 bits are required to record the I/O timestamp, if integers are utilized, 32 bits are required to record the I/O timestamp.
(2) The initialization packet specifically includes:
8 groups are built in the solid state disk, numbering is carried out from 0 to 7, a preset period corresponding to each group is set, and physical blocks corresponding to each group are set. Wherein different packets correspond to different physical blocks. 8 is the parallelism of the current solid state disk. See table 2 for each packet and predetermined period.
TABLE 2
Packet numbering 0 1 2 3 4 5 6 7
Preset period/ms 8 16 32 64 128 256 512 1024
2. The updating section includes: updating LBA life cycle table and updating preset period.
(1) Updating the LBA lifecycle table specifically includes:
if a write operation (which may be a host write operation or a migration write operation to a valid data page during garbage collection operation) occurs at any one logical address, determining an operation I/O timestamp of the current write operation; judging whether the record I/O time stamp of the logic address is an initial time stamp or not; if not, updating the update period of the logic address into a difference value between the operation I/O time stamp and the record I/O time stamp; updating the record I/O timestamp to an operation I/O timestamp; if the recorded I/O time stamp is an initial time stamp, judging whether the update period is an initial period or not; if yes, the record I/O time stamp is updated to be the operation I/O time stamp, and the update period is updated to be the minimum preset period. If the update period is not the initial period, the update period is reserved and the record I/O timestamp is updated to the operation I/O timestamp. After the I/O time stamp and the update period are recorded and updated, the data corresponding to the current writing operation can be written in according to the corresponding flow, so that the cold and hot data can be stored separately. The specific process can be seen in fig. 2, and the description of this embodiment is omitted.
If a Trim operation instruction is received, determining a logic address according to the Trim operation instruction; updating the update period of the logic address to be the difference value between the instruction time stamp of the Trim operation instruction and the record I/O time stamp of the logic address; updating the record I/O time stamp of the logic address to be an initial time stamp; and if the I/O data corresponding to the logical address is acquired later, storing the I/O data according to the update period and the recorded I/O time stamp, thereby realizing the separate storage of the cold and hot data.
If garbage collection operation is executed on the logical address (namely, data in a physical data page corresponding to the logical address is erased), increasing the update period of the logical address to obtain a target update period; calculating the difference between the target updating period and the preset period of each group; and migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference, so that the data with similar liveness are stored in the same physical block, and the separate storage of cold and hot data is realized.
(2) The updating of the preset period specifically includes:
and updating the preset period of the packet corresponding to the current logical address according to the current updating period corresponding to the current logical address. Namely: and updating the preset period of the current packet to be the average value of the updating periods of all the logical addresses corresponding to the current packet.
Therefore, the scheme provided by the implementation stores cold and hot data by the grouping thought, so that the write amplification can be reduced, the service life of the solid state disk is prolonged, the cost is reduced, and the performance of the solid state disk is improved.
A data processing apparatus according to an embodiment of the present application is described below, and a data processing apparatus described below and a data processing method described above may be referred to each other.
Referring to fig. 4, an embodiment of the present application discloses a data processing apparatus, including:
an acquisition module 401, configured to acquire a garbage collection instruction;
a determining module 402, configured to determine a physical data page corresponding to the garbage collection instruction;
a query module 403, configured to query a logical address corresponding to the physical data page;
an increasing module 404, configured to increase an update period of the logical address to obtain a target update period; the update period represents the liveness of the data in the physical data page;
a calculating module 405, configured to calculate a degree of difference between the target update period and a preset period of each packet;
and a migration module 406, configured to migrate the data in the physical data page to a physical block corresponding to the packet corresponding to the smallest difference degree.
Preferably, the augmentation module is specifically configured to:
doubling the update period to obtain the target update period.
Preferably, after migrating the data in the physical data page to the physical block corresponding to the packet corresponding to the smallest difference degree, the method further includes:
the I/O data acquisition module is used for acquiring the I/O data corresponding to the logical address;
an operation I/O timestamp determining module, configured to determine an operation I/O timestamp corresponding to the I/O data;
the judging module is used for judging whether the record I/O time stamp of the logic address is an initial time stamp or not;
the first updating module is used for updating the updating period of the logic address into the difference value between the operation I/O timestamp and the recording I/O timestamp if the recording I/O timestamp is not the initial timestamp;
a second updating module for updating the record I/O time stamp to an operation I/O time stamp;
the difference degree calculation module is used for calculating the difference degree between the update period and the preset period of each group respectively;
and the storage module is used for storing the I/O data to a target physical block corresponding to a target packet corresponding to the minimum difference degree.
In a preferred embodiment, the method further comprises:
the updating period judging module is used for judging whether the updating period is an initial period or not if the recorded I/O time stamp is the initial time stamp;
The third updating module is used for updating the record I/O time stamp into the operation I/O time stamp and updating the updating period into the minimum preset period if the updating period is the initial period;
and storing the I/O data to an object physical block corresponding to the object grouping corresponding to the minimum preset period.
In a preferred embodiment, the method further comprises:
and the execution module is used for reserving the update period if the update period is not the initial period and executing the steps in the second update module, the difference calculation module and the storage module.
In a preferred embodiment, the memory module is specifically configured to:
and writing the I/O data into a target data page corresponding to the target group, wherein the target data page is a physical data page in a target physical block.
In a preferred embodiment, the method further comprises:
and a fourth updating module, configured to update the preset period of the target packet to an average value of update periods of all logical addresses corresponding to the target packet.
In a preferred embodiment, the method further comprises:
the grouping splitting judging module is used for judging whether the number of the data pages corresponding to the target grouping is twice the number of the data pages corresponding to other groupings;
And the splitting module is used for closing the packets with the least data pages in all the packets if the number of the data pages corresponding to the target packet is twice the number of the data pages corresponding to other packets, splitting the target packet into two packets and establishing the corresponding relation between the two packets and the target physical block.
In a preferred embodiment, the method further comprises:
and the address range determining module is used for determining the address range to which the logical address belongs and determining the record I/O time stamp and the update period of the address range as the record I/O time stamp and the update period of the logical address.
The more specific working process of each module and unit in this embodiment may refer to the corresponding content disclosed in the foregoing embodiment, and will not be described herein.
Therefore, the embodiment provides a data processing device, which can reduce write amplification, thereby prolonging the service life of the solid state disk, reducing the cost and improving the performance of the solid state disk.
The following describes a data processing apparatus according to an embodiment of the present application, and the data processing apparatus described below and the data processing method and apparatus described above may be referred to each other.
Referring to fig. 5, an embodiment of the present application discloses a data processing apparatus, including: a storage unit 501 and a main controller 502, wherein:
the storage unit comprises a physical block, wherein a physical data page in the physical block is used for storing I/O data;
the main controller is used for executing the data processing method disclosed in any of the foregoing embodiments.
The following describes a readable storage medium provided in the embodiments of the present application, and the readable storage medium described below and the data processing method, apparatus and device described above may be referred to with each other.
A readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the data processing method disclosed in the foregoing embodiments. For specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, and no further description is given here.
Reference to "first," "second," "third," "fourth," etc. (if present) herein is used to distinguish similar objects from each other and does not necessarily describe a particular order or sequence. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, or apparatus.
It should be noted that the description herein of "first," "second," etc. is for descriptive purposes only and is not to be construed as indicating or implying a relative importance or implying an indication of the number of technical features being indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of readable storage medium known in the art.
The principles and embodiments of the present application are described herein with specific examples, the above examples being provided only to assist in understanding the methods of the present application and their core ideas; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims (10)

1. A method of data processing, comprising:
presetting a plurality of groups, and setting a preset period and a physical block corresponding to each group; different groups correspond to different physical blocks; the number of the groups is not more than the parallelism of the solid state disk, and the parallelism is as follows: the maximum number of physical blocks that can be written simultaneously;
receiving a garbage collection instruction;
determining a physical data page corresponding to the garbage collection instruction;
inquiring a logic address corresponding to the physical data page;
increasing the update period of the logic address to obtain a target update period; the update period represents the liveness of the data in the physical data page;
calculating the difference between the target updating period and the preset period of each group;
and migrating the data in the physical data page to a physical block corresponding to the grouping corresponding to the minimum difference degree.
2. The method of claim 1, wherein said increasing the update period of the logical address to obtain a target update period comprises:
doubling the update period to obtain the target update period.
3. The data processing method according to claim 1 or 2, wherein after the migrating the data in the physical data page to the physical block corresponding to the packet corresponding to the smallest difference degree, further comprising:
acquiring I/O data corresponding to the logical address;
determining an operation I/O time stamp corresponding to the I/O data;
judging whether the record I/O time stamp is an initial time stamp or not;
if not, updating the update period of the logic address to be the difference value between the operation I/O timestamp and the record I/O timestamp;
updating the record I/O timestamp to the operation I/O timestamp;
calculating the difference between the update period and the preset period of each group;
and storing the I/O data to a target physical block corresponding to a target packet corresponding to the minimum difference degree.
4. A data processing method according to claim 3, further comprising:
if the record I/O time stamp is the initial time stamp, judging whether the update period is an initial period or not;
If yes, updating the record I/O time stamp to the operation I/O time stamp, and updating the updating period to the minimum preset period;
and storing the I/O data to an object physical block corresponding to the object grouping corresponding to the minimum preset period.
5. The data processing method according to claim 3, wherein storing the I/O data to a target physical block corresponding to a target packet corresponding to a minimum degree of difference, comprises:
and writing the I/O data into a target data page corresponding to the target group, wherein the target data page is a physical data page in the target physical block.
6. The data processing method according to claim 5, further comprising:
and updating the preset period of the target packet to be the average value of the update periods of all the logic addresses corresponding to the target packet.
7. The data processing method according to claim 5, further comprising:
judging whether the number of the data pages corresponding to the target group is twice the number of the data pages corresponding to other groups;
if yes, closing the group with the least data page number in all groups, splitting the target group into two groups, and establishing the corresponding relation between the two groups and the target physical block.
8. A data processing device, characterized in that a plurality of groups are preset, and a preset period and a physical block corresponding to each group are set; different groups correspond to different physical blocks; the number of the groups is not more than the parallelism of the solid state disk, and the parallelism is as follows: the maximum number of physical blocks that can be written simultaneously; comprising the following steps:
the acquisition module is used for acquiring garbage collection instructions;
the determining module is used for determining a physical data page corresponding to the garbage collection instruction;
the inquiry module is used for inquiring the logical address corresponding to the physical data page;
the increasing module is used for increasing the update period of the logic address to obtain a target update period; the update period represents the liveness of the data in the physical data page;
the calculating module is used for calculating the difference between the target updating period and the preset period of each group;
and the migration module is used for migrating the data in the physical data page to the physical block corresponding to the grouping corresponding to the minimum difference degree.
9. A data processing apparatus, comprising: a storage unit and a main controller, wherein:
the storage unit comprises a physical block, wherein a physical data page in the physical block is used for storing I/O data;
The main controller is configured to perform the data processing method according to any one of claims 1 to 7.
10. A readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the data processing method according to any one of claims 1 to 7.
CN201911310534.3A 2019-12-18 2019-12-18 Data processing method, device, equipment and readable storage medium Active CN112988612B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201911310534.3A CN112988612B (en) 2019-12-18 2019-12-18 Data processing method, device, equipment and readable storage medium
PCT/CN2020/115543 WO2021120731A1 (en) 2019-12-18 2020-09-16 Data storage method and assembly, and data processing method and assembly

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911310534.3A CN112988612B (en) 2019-12-18 2019-12-18 Data processing method, device, equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN112988612A CN112988612A (en) 2021-06-18
CN112988612B true CN112988612B (en) 2024-03-19

Family

ID=76343904

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911310534.3A Active CN112988612B (en) 2019-12-18 2019-12-18 Data processing method, device, equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN112988612B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108646977A (en) * 2018-03-07 2018-10-12 深圳忆联信息***有限公司 A kind of method and rubbish recovering method of the cold and hot data judgements of SSD

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9183134B2 (en) * 2010-04-22 2015-11-10 Seagate Technology Llc Data segregation in a storage device
US10296452B2 (en) * 2015-05-11 2019-05-21 SK Hynix Inc. Data separation by delaying hot block garbage collection
JP6414853B2 (en) * 2015-12-14 2018-10-31 東芝メモリ株式会社 Memory system and control method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108646977A (en) * 2018-03-07 2018-10-12 深圳忆联信息***有限公司 A kind of method and rubbish recovering method of the cold and hot data judgements of SSD

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Fei W,et al.FastGC: Accelerate Garbage Collection via an Efficient Copyback-based Data Migration in SSDs.《ACM》.2018,正文第1-6页. *
全程优化的固态硬盘垃圾回收方法;方才华;刘景宁;童薇;高阳;雷霞;蒋瑜;;计算机应用(05);正文第1257-1262页 *
基于逻辑区间冷热分离的NAND闪存垃圾回收算法;覃仁谅;宁芊;严华;;微电子学与计算机(05);正文第58-63页 *

Also Published As

Publication number Publication date
CN112988612A (en) 2021-06-18

Similar Documents

Publication Publication Date Title
US10209902B1 (en) Method and apparatus for selecting a memory block for writing data, based on a predicted frequency of updating the data
Jimenez et al. Wear Unleveling: Improving {NAND} Flash Lifetime by Balancing Page Endurance
EP2758882B1 (en) Adaptive mapping of logical addresses to memory devices in solid state drives
US10310766B2 (en) Memory system and data relocating method
TW201413451A (en) Method of data collection in a non-volatile memory
CN109471594B (en) M L C flash memory read-write method
CN111414138B (en) Wear balancing method and device for solid state disk
WO2011072538A1 (en) Method and apparatus for nand flash erase leveling
JP6167646B2 (en) Information processing apparatus, control circuit, control program, and control method
CN110389712B (en) Data writing method and device, solid state disk and computer readable storage medium
CN102981971A (en) Quick-response phase change memory wear-leveling method
CN115756312A (en) Data access system, data access method, and storage medium
US20140281132A1 (en) Method and system for ram cache coalescing
CN102981972A (en) Wear-leveling method for phase change memory
CN108563586B (en) Method for separating garbage recovery data and user data in solid-state disk
CN112988612B (en) Data processing method, device, equipment and readable storage medium
CN111078136B (en) Method for preventing flash life reduction caused by BMS dynamic data storage
CN112988040B (en) Data storage method, device and equipment and readable storage medium
CN112988042B (en) Data storage method, device and equipment and readable storage medium
US11620058B2 (en) Temperature-adjusted power-on data retention time tracking for solid state drives
Chen et al. Refinery swap: An efficient swap mechanism for hybrid DRAM–NVM systems
Khanbadr et al. A novel method for victim block selection for NAND flash-based solid state drives based on scoring
WO2014185038A1 (en) Semiconductor storage device and control method thereof
WO2021120731A1 (en) Data storage method and assembly, and data processing method and assembly
CN107436726B (en) Data writing method, storage controller, computing device and storage device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant