WO2024103596A1 - Metadata restoration method and apparatus, and electronic device and storage medium - Google Patents

Metadata restoration method and apparatus, and electronic device and storage medium Download PDF

Info

Publication number
WO2024103596A1
WO2024103596A1 PCT/CN2023/084812 CN2023084812W WO2024103596A1 WO 2024103596 A1 WO2024103596 A1 WO 2024103596A1 CN 2023084812 W CN2023084812 W CN 2023084812W WO 2024103596 A1 WO2024103596 A1 WO 2024103596A1
Authority
WO
WIPO (PCT)
Prior art keywords
entry
address table
damaged
secondary address
data
Prior art date
Application number
PCT/CN2023/084812
Other languages
French (fr)
Chinese (zh)
Inventor
刘海伟
公维锋
Original Assignee
浪潮电子信息产业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浪潮电子信息产业股份有限公司 filed Critical 浪潮电子信息产业股份有限公司
Publication of WO2024103596A1 publication Critical patent/WO2024103596A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing

Definitions

  • the present application relates to the field of virtual storage, and in particular to a metadata repair method, device, electronic device and computer-readable storage medium.
  • Virtual disk is an important part of virtual machine equipment, and qcow2 is a common format for virtual disk.
  • the first-level address table (L1Table) and the second-level address table (L2Table) are important metadata required for virtual disk address management, where the first-level address table is used to store the address of the second-level address table in the virtual disk, and the second-level address table is used to store the address of the user data cluster (Data Cluster) in the virtual disk.
  • Data Cluster user data cluster
  • the virtual disk can query user data in the order of L1 to L2.
  • the virtual disk in order to improve the efficiency of finding data, the virtual disk often uses the cached second-level address table in the memory to record the user data cluster that has been accessed.
  • the cached data in the second-level address table may be disordered, which may lead to the problem that the second-level address table cannot be flushed to the virtual disk and the data content is converted into random numbers.
  • the second-level address table cannot be used for addressing, which causes the problem of data loss of the virtual disk.
  • the present application provides a metadata repair method, comprising:
  • each entry in the secondary address table is used to record the address of each user data in the virtual disk;
  • a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged table entry is repaired using the reverse secondary address table.
  • the method before determining that there is a damaged entry in the secondary address table of the virtual disk, the method further includes:
  • obtaining a secondary address table from the virtual disk includes:
  • the primary address table is obtained from the virtual disk, and the corresponding secondary address table is searched in the virtual disk according to each table item of the primary address table; the table items of the primary address table are used to record the starting address of each secondary address table in the virtual disk.
  • the method further includes:
  • querying the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk includes:
  • the backup first-level address table is obtained from the backup disk, and the backup table entry is searched in the backup disk using the index and the backup first-level address table.
  • the method further includes:
  • the next entry to be detected is determined to be a damaged entry.
  • the method before determining that there is a damaged entry in the secondary address table of the virtual disk, the method further includes:
  • the damaged table entry is repaired using the backup table entry, including:
  • the backup entry is rewritten back to the corresponding position of the damaged entry in the secondary address table of the virtual disk to complete the repair of the damaged entry.
  • it also includes:
  • a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, including:
  • the data to be verified is matched and verified using preset features, and when it is determined that the data to be verified passes the matching verification, the target address of the data to be verified in the virtual disk is written into the reverse secondary address table, so that the target address is used as an entry in the reverse secondary address table.
  • the specified size is the size of user data pointed to by the entry of the secondary address table.
  • matching verification is performed on the data to be verified using preset features, including:
  • the damaged entry is repaired by using the reverse secondary address table, including:
  • the entry content of the normal entry is smaller than the correct entry content corresponding to the damaged entry;
  • the content of the target entry is the same as that of the normal entry.
  • the entry content of the latter entry is greater than the entry content of the target entry.
  • the damaged entry is repaired by using the reverse secondary address table, including:
  • the method further includes:
  • an offset value is added to the target index value within a preset offset interval to move the position of the repaired entry in the secondary address table; offset The value is an integer multiple of the preset data cluster size.
  • the offset value is adjusted within a preset offset interval, the adjusted offset value is added to the target index value of the repaired table entry, and the step of restarting the virtual machine to which the virtual disk belongs is entered.
  • a backup table entry query module is used to query the backup table entry corresponding to the damaged table entry in the backup disk corresponding to the virtual disk when it is determined that there is a damaged table entry in the secondary address table of the virtual disk; each table entry in the secondary address table is used to record the address of each user data in the virtual disk;
  • a first repair module configured to repair a damaged entry using the backup entry when a backup entry is found
  • the present application also provides an electronic device, comprising:
  • One or more processors are used to implement the above metadata repair method when executing computer-readable instructions.
  • the present application also provides one or more non-volatile computer-readable storage media storing computer-readable instructions.
  • the computer-readable instructions are executed by one or more processors, the one or more processors execute the metadata repair method as described above.
  • FIG1 is a flowchart of a metadata repair method provided by one or more embodiments of the present application.
  • FIG2 is a schematic diagram of a first type of secondary address table entry repair provided by one or more embodiments of the present application
  • FIG3 is a schematic diagram of a second type of secondary address table entry repair provided by one or more embodiments of the present application.
  • FIG4 is a schematic diagram of a third type of secondary address table entry repair provided by one or more embodiments of the present application.
  • FIG5 is a structural block diagram of a metadata repair device provided by one or more embodiments of the present application.
  • FIG6 is a structural block diagram of another metadata repair device provided by one or more embodiments of the present application.
  • FIG8 is a block diagram of a non-volatile computer-readable storage medium provided by one or more embodiments of the present application.
  • the first-level address table (L1 Table) and the second-level address table (L2 Table) are important metadata required for address management of the virtual disk, wherein the second-level address table is used to store the address of the user data cluster in the virtual disk.
  • the second-level address table is used to store the address of the user data cluster in the virtual disk.
  • data disorder may occur in the second-level address table cache, which may lead to problems such as the data in the second-level address table cannot be flushed to the virtual disk and the data content is converted into random numbers.
  • the second-level address table cannot be used for addressing, which causes the problem of data loss of the virtual disk.
  • the present application may provide a metadata repair method, which may use backup data and a reverse second-level address table constructed by scanning all user data addresses in the virtual disk to repair damaged entries in the second-level address table of the virtual disk, so as to repair the lost user data as much as possible.
  • a metadata repair method which may use backup data and a reverse second-level address table constructed by scanning all user data addresses in the virtual disk to repair damaged entries in the second-level address table of the virtual disk, so as to repair the lost user data as much as possible.
  • Figure 1 is a metadata repair method provided in an embodiment of the present application.
  • a flow chart of a method the method may include:
  • each entry in the secondary address table is used to record the address of each user data in the virtual disk.
  • each entry in the secondary address table is used to store the address of the corresponding user data in the virtual disk, and the virtual disk relies on each entry in the secondary address table to address the user data.
  • a corresponding backup disk can be first introduced for the virtual disk, and the backup disk can be used to back up the user data, the primary address table, the secondary address table, etc. stored therein. Then, when it is found that there is a damaged entry in the secondary address table of the virtual disk, the corresponding backup entry in the backup disk can be used to repair the damaged entry to achieve the effect of restoring the user data.
  • the backup disk can be physically isolated from the production storage pool where the virtual disk is located, so that when the production storage pool encounters a mechanical or power failure, the backup storage pool will not be affected, and the user data security can be protected to the greatest extent.
  • the backup disk will be used to restore the damaged entry in the secondary address table of the virtual disk, but the user data corresponding to the entry will not be repaired.
  • the user data itself may not be damaged, but only the corresponding entry in the secondary address table is abnormal; at the same time, affected by the backup cycle, the user data in the backup disk may not be exactly the same as the user data in the virtual disk, and then if the backup disk is used to repair the user data corresponding to the damaged entry, it is easy to cause user data damage, which brings inconvenience to the normal use of the user.
  • the method may further include: obtaining the secondary address table from the virtual disk, and setting each entry in the secondary address table as a to-be-detected item in turn. determining whether the corresponding user data is found using the entry to be detected; and in response to the use of the entry to be detected not being able to find the corresponding user data, determining that the entry to be detected is a damaged entry.
  • Step 11 Obtain the secondary address table from the virtual disk, and set each entry in the secondary address table as an entry to be detected in turn;
  • Step 12 Determine whether the data content of the entry to be detected is divisible by the preset data cluster size; if so, proceed to step 13; if not, proceed to step 14;
  • Step 13 Execute step 12 for the next entry to be detected
  • the data content of any item to be detected can be divided by the preset data cluster size; in response to the data content of any item to be detected being divisible by the preset data cluster size, it is determined whether the data content of the next item to be detected of any item to be detected can be divided by the preset data cluster size; and in response to the data content of the next item to be detected not being divisible by the preset data cluster size, it is determined that the next item to be detected is a damaged item.
  • Step 14 Determine that the entry to be detected is a damaged entry.
  • any to-be-detected entry is a damaged entry.
  • each secondary address table there may be multiple secondary address tables in the virtual disk, and the data clusters occupied by each secondary address table may not be adjacent, so it is not convenient to search for the secondary address table by scanning the virtual disk.
  • each entry of the primary address table stores the starting address of each secondary address table in the virtual disk, the embodiment of the present application can use the primary address table to query the secondary address table.
  • obtaining the secondary address table from the virtual disk includes:
  • Step 21 Obtain the primary address table from the virtual disk, and search for the corresponding secondary address table in the virtual disk according to each entry in the primary address table; the entries in the primary address table are used to record the starting address of each secondary address table in the virtual disk.
  • the primary address table in order to facilitate query and management, in addition to recording the initial address of the secondary address table, also stores the index of each table item in the secondary address table.
  • the primary address table in the backup disk and the primary address table in the virtual disk are the same in terms of data content and table structure, after finding the damaged table item in the secondary address table, the corresponding index of the table item in the primary address table of the virtual disk can also be determined, and then the corresponding backup table item in the backup disk can be found by using the index and the primary address table of the backup disk.
  • Step 31 Record the index corresponding to the damaged entry in the primary address table
  • querying the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk may include:
  • Step 32 Obtain the backup first-level address table from the backup disk, and use the index and the backup first-level address table to search for the backup table entry in the backup disk.
  • the data in the backup disk may be backed up and updated regularly using the data in the virtual disk.
  • the method may further include:
  • Step 41 Regularly back up and update the data in the backup disk using the data in the virtual disk.
  • the backup entry corresponding to the damaged entry can be found in the backup disk, the backup entry can be rewritten back to the corresponding position of the damaged entry in the virtual disk secondary address table to complete the repair of the damaged entry.
  • a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged entry is repaired using the reverse secondary address table.
  • the backup disk may not store the backup table entries corresponding to the damaged table entries.
  • the backup data cannot be used to repair the virtual disk's secondary address table.
  • the embodiment of the present application can provide another way to repair the secondary address table.
  • a reverse secondary address table (Maybe_L2_Table) can be constructed by scanning all user data addresses in the virtual disk, and the reverse secondary address table can be used to repair the virtual disk's secondary address table.
  • the virtual disk also records metadata such as the primary address table, the secondary address table, and the reference count table (Refcount Table).
  • This batch of metadata has significant data features, and the data in the virtual disk can be screened based on these data features, filtering out the metadata content and retaining possible user data, and using the addresses of these possible user data to construct a reverse secondary address table.
  • both the reverse secondary address table and the secondary address table of the virtual disk store the user data address, that is, there is a similarity between the two tables, the reverse secondary table can be searched for the reverse entry closest to the loss entry based on the similarity between the two tables, and then the loss entry can be repaired using the reverse entry.
  • the specific construction process of the reverse secondary address table and the repair process of the damaged entry using the table can also include other preferred steps, which can be specifically referred to the description in the subsequent embodiments.
  • the present application when determining that there is a damaged entry in the secondary address table of the virtual disk, the present application will give priority to The backup table items corresponding to the damaged items are queried in the backup disk corresponding to the virtual disk, and the damaged items are repaired preferentially using the more accurate backup data to improve the success rate of recovering user data; subsequently, if the relevant backup items cannot be queried from the backup disk, the present application can also construct a reverse secondary address table by scanning all user data addresses in the virtual disk, and use the reverse secondary address table to repair the damaged items. This is because the reverse secondary address table can contain all user data addresses, and there are still normal items containing the correct user data addresses in the secondary address table, and then there is a similarity between the secondary address table and the reverse secondary table.
  • the present application can use the reverse secondary address table to repair the damaged items and avoid user data loss as much as possible.
  • the present application can use the backup data and the above-mentioned reverse secondary address table to repair the damaged items in the secondary address table of the virtual disk to repair the lost user data as much as possible.
  • the present application provides a metadata repair method, comprising: when determining that there is a damaged table entry in the secondary address table of a virtual disk, querying the backup table entry corresponding to the damaged table entry in the backup disk corresponding to the virtual disk; each table entry in the secondary address table is used to record the address of each user data in the virtual disk; when the backup table entry is found, the damaged table entry is repaired by using the backup table entry; when the backup table entry is not found, a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged table entry is repaired by using the reverse secondary address table.
  • the present application determines that there is a damaged entry in the secondary address table of the virtual disk, it will first query the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk, and use the more accurate backup data to repair the damaged entry first, so as to improve the success rate of recovering user data; then, if the relevant backup entry cannot be queried from the backup disk, the present application can also construct a reverse secondary address table by scanning all user data addresses in the virtual disk, and use the reverse secondary address table to repair the damaged entry. This is because the reverse secondary address table can contain all user data addresses, and there are still normal entries containing correct user data addresses in the secondary address table, and then there is a similarity between the secondary address table and the reverse secondary table.
  • the present application can use the reverse secondary address table to repair the damaged entry and avoid user data loss as much as possible.
  • the present application can use backup data and the above-mentioned reverse secondary address table to repair the damaged entry in the secondary address table of the virtual disk to repair the lost user data as much as possible.
  • the reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, including:
  • Disk data This is because this part of the data is mainly the file header of the virtual disk file (qcow2 file) and does not contain valid user data, so it does not need to be scanned.
  • the embodiment of the present application does not limit the specific preset size, for example, it can be 3 data clusters.
  • the implementation of the present application will scan the remaining disk data in order from front to back. Specifically, the embodiment of the present application sets the data of a specified size as the data to be verified in order from front to back.
  • the specified size is the user data size that a secondary address table entry can point to, for example, it can be 64 bits (bit). In other words, the embodiment of the present application will extract 64 bits of data from the remaining disk data in sequence for detection.
  • S201 Perform matching verification on the data to be verified using preset features, and when it is determined that the data to be verified passes the matching verification, write the target address of the data to be verified in the virtual disk into the reverse secondary address table, so as to use the target address as an entry in the reverse secondary address table.
  • the embodiment of the present application will specifically use the relevant features of the metadata to match and verify the data to be verified.
  • the keywords of the primary address table and the reference count table can be used for verification, where the keyword of the primary address table is 0X8000*0000, where * represents a wildcard, that is, skipping the fields starting with 8000 and suffixed with 0000; and the keyword of the reference count table is 0X0001000100010001. If it is determined that the data to be verified does not contain the above keywords, it can be roughly determined that the data to be verified belongs to user data, and then it can be determined that the data to be verified has passed the verification, and its address in the virtual disk can be written into the reverse secondary address table.
  • using preset features to perform matching verification on the data to be verified may include:
  • Step 51 When it is determined that the header and the tail of the data to be verified do not contain the keyword of the primary address table at the same time, and the keyword of the data to be verified is different from the keyword of the reference count table, it is determined that the data to be verified passes the matching verification.
  • FIG. 2 is a schematic diagram of the first secondary address table entry repair provided by an embodiment of the present application, wherein the left side is the reverse secondary address table, the right side is the secondary address table of the virtual disk, and the entry marked by mark 13 (the entry content is 0x111111111) is a damaged entry.
  • the embodiment of the present application can search for the previous normal entry of the damaged entry in the original secondary address table (i.e., the entry marked by mark 14, the entry content is 0x33c8e0000), the normal entry is adjacent to the damaged entry, and its entry content is less than the correct entry content corresponding to the damaged entry. Then, the target entry (i.e., the entry marked with mark 12, the content of which is exactly the same as the normal entry) corresponding to the inverse secondary address table can be searched. The damaged entry is repaired by using the next entry of the target entry (i.e., the entry marked by marker 11, whose content is 0x33cb80000), wherein the next entry of the target entry refers to the entry that is adjacent to the target entry and whose content is greater than the target entry.
  • the target entry i.e., the entry marked with mark 12, the content of which is exactly the same as the normal entry
  • the damaged entry is repaired by using the next entry of the target entry (i.e., the entry marked by marker 11, whose content is 0
  • repairing the damaged entry by inverting the secondary address table may include:
  • Step 61 Find the previous normal entry corresponding to the damaged entry in the secondary address table
  • Step 62 Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use the next entry of the target entry to repair the damaged entry.
  • Figure 3 is a schematic diagram of the second type of secondary address table entry repair provided by the embodiment of the present application, wherein the side is the reverse secondary address table, the right side is the secondary address table of the virtual disk, and the 4 consecutive entries marked by mark 23 (the entry contents are 0x111111111, 0x222222222, 0x333333333, 0x444444444) are all damaged entries, and these 4 damaged entries can form a damaged entry group.
  • the embodiment of the present application can search for the previous normal entry of the damaged entry group in the original secondary address table (i.e., the entry marked by mark 24, the entry content is 0x33c8e0000), the normal entry is adjacent to the damaged entry group, and its entry content is less than the smallest correct entry content in the damaged entry group.
  • a target entry corresponding to the normal entry and with exactly the same content as the normal entry can be searched in the reverse secondary address table (i.e., the entry marked by marker 22, whose content is 0x33c8e0000), and the damaged entry group can be repaired using the four entries after the target entry (i.e., the four consecutive entries pointed to by marker 21, whose contents are 0x33cb80000, 0x33cb80000, 0x33ce70000, and 0x33d200000, respectively).
  • repairing the damaged entry by inverting the secondary address table may include:
  • Step 71 determining a damaged entry group consisting of consecutive damaged entries in the secondary address table, and searching for a previous normal entry corresponding to the damaged entry group;
  • Step 71 Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use a specified number of entries after the target entry to repair the damaged entry group; the specified number is the number of damaged entries included in the damaged entry group.
  • the present application can construct a reverse secondary address table by scanning all user data addresses in the virtual disk, and repair damaged table entries based on the similarity between the table and the original secondary address table. It can further repair damaged table entries that cannot be repaired by backup data, thereby further improving the effectiveness of user data recovery.
  • the reverse secondary address table is to repair the damaged table entries using possible user data addresses, but it is inevitable that some user data addresses will be missed when constructing the reverse secondary address table, which will cause the virtual disk secondary address table to have missing user addresses, making it difficult to recover user data.
  • FIG. 4 is a schematic diagram of the third type of secondary address table entry repair provided by the embodiment of the present application, wherein the left side represents the correct secondary address table, and the right side represents the wrong secondary address table.
  • the embodiment of the present application can also adjust the position of the table entries that have been repaired but the corresponding user data has not been restored, so as to restore the user data as much as possible.
  • the damaged table entries are repaired by using the reverse secondary address table, it can also include:
  • S301 Mark the repaired damaged entry as a repaired entry, and determine a target index value of the repaired entry in the secondary address table.
  • all entries included in entry group 32 in Figure 4 can be marked as repaired entries. It should be particularly noted that the present embodiment further requires determining the target index value of the repaired entry in the secondary address table, that is, determining the current position of the repaired entry in the secondary address table.
  • step S302 restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry is restored; if so, determining that the repair is successful; if not, proceeding to step S303.
  • the user data corresponding to the entry marked by marker 32 cannot be read due to the missing data content of the entry marked by marker 31 before the entry marked by marker 32.
  • the embodiment of the present application performs additional processing on the entry marked by marker 32 that has been repaired but cannot restore the user data.
  • an offset value is added to the target index value within a preset offset interval to move the position of the repaired table entry in the secondary address table; the offset value is an integer multiple of the preset data cluster size.
  • the embodiment of the present application can continue to increase the offset value for the target index value of these repaired entries to try to move these repaired entries to other locations and determine whether they can recover the user data corresponding to other locations.
  • the offset value should be an integer multiple of the preset data cluster size; at the same time, if the target index value of the repaired entry (or the target index value of the repaired entry group) is increased by the offset value, it has shifted to the position of the normal entry, then the repaired entry will be overwritten by the normal entry, and the offset value will not have a repair effect. Therefore, in order to avoid using invalid offset values, the embodiment of the present application can be in the preset offset area.
  • the offset value is increased within the interval. Obviously, the value of the preset offset interval is related to the normal table entry adjacent to the repaired table entry. Specifically, the preset offset interval is determined as follows:
  • the method before adding the offset value to the target index value within the preset offset interval, the method further includes:
  • Step 81 determining a repaired entry group consisting of continuous repaired entries, and determining a previous normal entry and a next normal entry corresponding to the repaired entry group in the secondary address table;
  • Step 82 Determine in the secondary address table a first index value corresponding to a previous normal entry and a second index value corresponding to a next normal entry.
  • first and second are only used to distinguish the index values of the previous normal table entry and the next normal table entry, and do not mean that these two normal table entries have multiple entries in the secondary address table.
  • Step 83 Determine a preset offset interval corresponding to the repaired entry group using the first index value, the second index value and the preset data cluster size.
  • the upper limit of the preset offset interval should be the interval between the index values of the above two normal table entries minus the preset data cluster size.
  • adding an offset value to the target index value within a preset offset interval may include:
  • Step 84 Add an offset value to the target index value within a preset offset interval corresponding to the repaired entry group to which the repaired entry belongs.
  • the offset value can be further verified whether the user data is restored. If it is still not restored, the offset value can be continuously modified until the restoration is completed or it is determined that the restoration has failed.
  • the method further includes:
  • step S304 restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry with the added offset value is restored; if so, determining that the repair is successful; if not, proceeding to step S305.
  • the offset value is adjusted within a preset offset interval, the adjusted offset value is added to the target index value of the repaired table entry, and the step of restarting the virtual machine to which the virtual disk belongs is entered.
  • the embodiments of the present application may also adjust the positions of the table entries that have been repaired but the corresponding user data has not been restored, so as to restore the user data as much as possible.
  • FIG5 is an embodiment of the present application.
  • a structural block diagram of a metadata repair device is provided, and the modules included in the device and their uses are introduced in detail below.
  • Disk bad block check module used to check the L2 table (L2Table) of the damaged disk. It traverses and checks the L2 table items (the contents of the L2 table) according to the L1->L2 addressing rules corresponding to the qcow2 virtual disk, and finds the L2 table items that do not meet the expected rules. The rule is whether the L2 table item can be divided by the data cluster size (Cluster_size). If it cannot be divided, it means that the L2 table item is damaged. The disk bad block check module will obtain the damaged L2 table item and its index in L1;
  • Virtual disk backup module used to back up the virtual disk to the backup storage pool to recover data when an environmental anomaly occurs.
  • two backup strategies can be provided: scheduled backup and immediate backup.
  • the virtual disk will be backed up to the backup storage pool according to the backup strategy.
  • the backup disk contains the L1 index and L2 table entries found by the disk bad block check module, the L2 table entries in the backup disk can be used to restore the damaged qcow2 metadata.
  • the real data corresponding to the L2 table entry is not restored to avoid previous data overwriting the current data (once the qcow2 disk metadata is established, the corresponding relationship will not change).
  • Disk offset reverse inference module used to explore all possible user data clusters (Data Cluster). It does not rely on the L1->L2 addressing rules of the qcow2 virtual disk, but according to the size of each data cluster offset and the qcow2 metadata characteristics (first skip three table headers of the same size as the data cluster, then skip the field with the content of 0X0001000100010001 (most likely the reference count table), and finally skip the content of 0X8000*0000 [* represents a wildcard, that is, skip the field with the beginning of 8000 and the suffix of 0000]), and screen out the addresses of data blocks that may be user data clusters one by one (possible L2 table entries); then obtain all possible L2 table entries, and use them to construct the reverse secondary address table (Maybe_L2_Tables);
  • Bad block precision repair module used to repair qcow2 disk metadata. It takes the L1 index output by the disk bad block check module as input, and first searches the backup disk for the corresponding L2 table entry. If it exists, the L2 table entry is written into the L2 table entry corresponding to the L1 index of the current damaged disk. If the backup disk does not find the corresponding L2 table entry, the inverse secondary address table obtained by the disk offset inverse module is written into the L2 table entry corresponding to the L1 index of the current damaged disk.
  • Repair adjustment module used to adjust the offset value (Offset) of the L2 table entry written from the reverse secondary address table in the L2 table.
  • Offset the offset value of the L2 table entry written from the reverse secondary address table in the L2 table.
  • the metadata repair device, electronic device, and computer-readable storage medium provided in the embodiments of the present application are described below. Introduction: The metadata repair device, electronic device, and computer-readable storage medium described below can correspond to the data repair method described above.
  • FIG. 6 is a structural block diagram of another metadata repair device provided in an embodiment of the present application.
  • the device may include:
  • the backup entry query module 601 is used to query the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk when it is determined that there is a damaged entry in the secondary address table of the virtual disk; each entry in the secondary address table is used to record the address of each user data in the virtual disk;
  • a first repair module 602 configured to repair a damaged entry using the backup entry when a backup entry is found
  • the second repair module 603 is used to construct a reverse secondary address table by scanning the addresses of all user data in the virtual disk when no backup entry is found, and to repair the damaged entry using the reverse secondary address table.
  • the device may further include:
  • a setting module used for obtaining the secondary address table from the virtual disk before determining that there is a damaged table entry in the secondary address table of the virtual disk, and setting each table entry in the secondary address table as a table entry to be detected in sequence;
  • the damaged entry judging module is used to judge whether the data content of the entry to be checked can be divided by the preset data cluster size; if not, the entry to be checked is determined to be a damaged entry.
  • the setting module may include:
  • the search submodule is used to obtain the primary address table from the virtual disk and search the corresponding secondary address table in the virtual disk according to each entry of the primary address table; the entries of the primary address table are used to record the starting address of each secondary address table in the virtual disk.
  • the device may further include:
  • a recording module used for recording the index corresponding to the damaged table entry in the primary address table after determining that the table entry to be detected is a damaged table entry
  • the backup entry query module 601 may include:
  • the backup table entry query submodule is used to obtain the backup first-level address table from the backup disk, and use the index and the backup first-level address table to search for the backup table entry in the backup disk.
  • the device may further include:
  • the backup module is used to regularly back up and update the data in the backup disk using the data in the virtual disk.
  • the second repair module 603 includes:
  • the scanning setting submodule is used to skip the disk data of a preset size in the virtual disk header, and set the data of a specified size in the remaining disk data in order from front to back as the data to be verified;
  • the reverse secondary address table construction submodule is used to match and verify the data to be verified using preset features, and when it is determined that the data to be verified passes the matching verification, the target address of the data to be verified in the virtual disk is written into the reverse secondary address table, so as to use the target address as a table entry of the reverse secondary address table.
  • the reverse secondary address table construction submodule may include:
  • the verification unit is used to determine that the data to be verified passes the matching verification when it is determined that the header and the tail of the data to be verified do not contain the keyword of the primary address table at the same time, and the keywords of the data to be verified and the reference count table are different.
  • the second repair module 603 includes:
  • a first search submodule is used to search for a previous normal table entry corresponding to the damaged table entry in the secondary address table
  • the first repair submodule is used to determine the target table entry corresponding to the normal table entry in the reverse secondary address table, and repair the damaged table entry using the table entry following the target table entry.
  • the damaged entry is repaired by using the reverse secondary address table, including:
  • a second search submodule is used to determine a damaged table entry group consisting of consecutive damaged table entries in the secondary address table, and to search for a previous normal table entry corresponding to the damaged table entry group;
  • the second repair submodule is used to determine the target entry corresponding to the normal entry in the reverse secondary address table, and use a specified number of entries after the target entry to repair the damaged entry group; the specified number is the number of damaged entries contained in the damaged entry group.
  • the device may further include:
  • a marking module used for marking the repaired damaged table entry as a repaired table entry after repairing the damaged table entry by using the reverse secondary address table, and determining a target index value of the repaired table entry in the secondary address table;
  • a first recovery verification module is used to restart the virtual machine to which the virtual disk belongs, and determine in the virtual machine whether the user data corresponding to the repaired table entry is restored;
  • the offset value increasing module is used to increase the offset value for the target index value within the preset offset interval to move the position of the repaired table entry in the secondary address table if not; the offset value is an integer multiple of the preset data cluster size.
  • the device may further include:
  • An entry determination module used to determine a repaired entry group consisting of consecutive repaired entries before adding an offset value to a target index value within a preset offset interval, and determine a previous normal entry and a next normal entry corresponding to the repaired entry group in the secondary address table;
  • An index value determination module used to determine a first index value corresponding to a previous normal table entry and a second index value corresponding to a next normal table entry in the secondary address table;
  • a preset offset interval setting module is used to determine the offset interval using the first index value, the second index value and the preset data cluster size.
  • the offset value increasing module is specifically used for:
  • An offset value is added to the target index value within a preset offset interval corresponding to the repaired entry group to which the repaired entry belongs.
  • the device may further include:
  • the second recovery verification module is used to restart the virtual machine to which the virtual disk belongs after adding an offset value to the target index value within a preset offset interval, and determine in the virtual machine whether the user data corresponding to the repaired table entry with the added offset value is restored; if not, adjust the offset value within the preset offset interval, add the adjusted offset value to the original table entry content of the repaired table entry, and enter the step of restarting the virtual machine to which the virtual disk belongs.
  • FIG. 7 is a structural block diagram of an electronic device provided in an embodiment of the present application.
  • the embodiment of the present application further provides an electronic device, including:
  • Memory 701 used to store computer-readable instructions
  • One or more processors 702 are configured to implement the steps of the metadata repair method described above when executing computer-readable instructions.
  • the embodiments of the electronic device part and the embodiments of the metadata repair method part correspond to each other, the embodiments of the electronic device part refer to the description of the embodiments of the metadata repair method part, which will not be repeated here.
  • Figure 8 is a structural block diagram of one or more non-volatile computer-readable storage media storing computer-readable instructions provided in an embodiment of the present application.
  • the embodiment of the present application also provides one or more non-volatile computer-readable storage media 810 storing computer-readable instructions, and one or more non-volatile computer-readable storage media 810 storing computer-readable instructions store computer-readable instructions 811.
  • the steps of the metadata repair method of any of the above embodiments are implemented.
  • the steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two.
  • the software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present application relates to the field of virtual storage. Provided are a metadata restoration method and apparatus, and an electronic device and a storage medium. The method comprises: when determining that there is a damaged table entry in a second-level address table of a virtual disk, querying, from a backup disk corresponding to the virtual disk, a backup table entry corresponding to the damaged table entry, wherein each table entry of the second-level address table is used for recording the address of each piece of user data in the virtual disk; when the backup table entry is found, restoring the damaged table entry by using the backup table entry; and when the backup table entry is not found, constructing a reverse second-level address table by means of scanning the addresses of all the user data in the virtual disk, and restoring the damaged table entry by using the reverse second-level address table. The damaged table entry in the second-level address table of the virtual disk can be restored by using backup data and the reverse second-level address table, which is constructed by means of scanning the addresses of all the user data in the virtual disk, so as to restore lost user data as much as possible.

Description

一种元数据修复方法、装置、电子设备及存储介质Metadata repair method, device, electronic device and storage medium
相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS
本申请要求于2022年11月15日提交中国专利局,申请号为202211420802.9,申请名称为“一种元数据修复方法、装置、电子设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority to a Chinese patent application filed with the China Patent Office on November 15, 2022, with application number 202211420802.9 and application name “A metadata repair method, device, electronic device and storage medium”, all contents of which are incorporated by reference in this application.
技术领域Technical Field
本申请涉及虚拟存储领域,特别涉及一种元数据修复方法、装置、电子设备及计算机可读存储介质。The present application relates to the field of virtual storage, and in particular to a metadata repair method, device, electronic device and computer-readable storage medium.
背景技术Background technique
虚拟磁盘是虚拟机设备中的重要组成部分,其中qcow2是虚拟磁盘常见的格式。相关技术中,一级地址表(L1Table)和二级地址表(L2Table)是虚拟磁盘进行地址管理所需的重要元数据,其中一级地址表用于存放二级地址表在虚拟磁盘中的地址,而二级地址表用于存放用户数据簇(Data Cluster)在虚拟磁盘中的地址。这样,虚拟磁盘便可依照L1至L2的顺序进行用户数据查询。此外,虚拟磁盘在寻址过程中,为了提高查找数据的效率,经常使用内存中已缓存的二级地址表记录已经访问过的用户数据簇。然而,当***断电、文件***异常时,可能导致二级地址表缓存数据紊乱,进而导致二级地址表无法下刷到虚拟磁盘、数据内容转变为随机数等问题。此时,该二级地址表便无法用于寻址,进而就造成了虚拟磁盘的数据丢失问题。Virtual disk is an important part of virtual machine equipment, and qcow2 is a common format for virtual disk. In related technologies, the first-level address table (L1Table) and the second-level address table (L2Table) are important metadata required for virtual disk address management, where the first-level address table is used to store the address of the second-level address table in the virtual disk, and the second-level address table is used to store the address of the user data cluster (Data Cluster) in the virtual disk. In this way, the virtual disk can query user data in the order of L1 to L2. In addition, in the process of addressing, in order to improve the efficiency of finding data, the virtual disk often uses the cached second-level address table in the memory to record the user data cluster that has been accessed. However, when the system is powered off or the file system is abnormal, the cached data in the second-level address table may be disordered, which may lead to the problem that the second-level address table cannot be flushed to the virtual disk and the data content is converted into random numbers. At this time, the second-level address table cannot be used for addressing, which causes the problem of data loss of the virtual disk.
因此,如何修复损坏的二级数据表是本领域技术人员亟需解决的技术问题。Therefore, how to repair the damaged secondary data table is a technical problem that technical personnel in this field need to solve urgently.
发明内容Summary of the invention
本申请提供一种元数据修复方法,包括:The present application provides a metadata repair method, comprising:
在确定虚拟磁盘的二级地址表中存在损坏表项时,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项;二级地址表的各个表项用于记录虚拟磁盘中各个用户数据的地址;When it is determined that there is a damaged entry in the secondary address table of the virtual disk, a backup entry corresponding to the damaged entry is queried in the backup disk corresponding to the virtual disk; each entry in the secondary address table is used to record the address of each user data in the virtual disk;
当查找到备份表项时,利用备份表项对损坏表项进行修复;和 When a backup entry is found, the damaged entry is repaired using the backup entry; and
当未查找到备份表项时,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复。When the backup table entry is not found, a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged table entry is repaired using the reverse secondary address table.
可选地,在确定虚拟磁盘的二级地址表中存在损坏表项之前,还包括:Optionally, before determining that there is a damaged entry in the secondary address table of the virtual disk, the method further includes:
从虚拟磁盘中获取二级地址表,并将二级地址表中的每个表项依次设置为待检测表项;Obtain a secondary address table from the virtual disk, and set each entry in the secondary address table as an entry to be detected in turn;
判断任一待检测表项的数据内容能否被预设数据簇大小整除;和Determine whether the data content of any to-be-detected entry is divisible by a preset data cluster size; and
响应于任一待检测表项的数据内容不能被预设数据簇大小整除,确定任一待检测表项为损坏表项。In response to the data content of any to-be-detected entry not being divisible by the preset data cluster size, it is determined that any to-be-detected entry is a damaged entry.
可选地,从虚拟磁盘中获取二级地址表,包括:Optionally, obtaining a secondary address table from the virtual disk includes:
从虚拟磁盘中获取一级地址表,并根据一级地址表的各个表项在虚拟磁盘中查找对应的二级地址表;一级地址表的表项用于记录各二级地址表在虚拟磁盘中的起始地址。The primary address table is obtained from the virtual disk, and the corresponding secondary address table is searched in the virtual disk according to each table item of the primary address table; the table items of the primary address table are used to record the starting address of each secondary address table in the virtual disk.
可选地,在确定待检测表项为损坏表项之后,还包括:Optionally, after determining that the entry to be detected is a damaged entry, the method further includes:
记录损坏表项在一级地址表中对应的索引;Record the corresponding index of the damaged entry in the primary address table;
相应的,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项,包括:Accordingly, querying the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk includes:
从备份磁盘中获取备份一级地址表,并利用索引及备份一级地址表在备份磁盘中查找备份表项。The backup first-level address table is obtained from the backup disk, and the backup table entry is searched in the backup disk using the index and the backup first-level address table.
可选地,将二级地址表中的每个表项依次设置为待检测表项之后,还包括:Optionally, after each entry in the secondary address table is sequentially set as an entry to be detected, the method further includes:
响应于任一待检测表项的数据内容能被预设数据簇大小整除,判断任一待检测表项的下一项待检测表项的数据内容能否被预设数据簇大小整除;和In response to the data content of any to-be-detected entry being divisible by a preset data cluster size, determining whether the data content of a next to-be-detected entry of any to-be-detected entry is divisible by the preset data cluster size; and
响应于下一待检测表项的数据内容不能被预设数据簇大小整除,确定下一待检测表项为损坏表项。In response to the data content of the next entry to be detected not being divisible by the preset data cluster size, the next entry to be detected is determined to be a damaged entry.
可选地,在确定虚拟磁盘的二级地址表中存在损坏表项之前,还包括:Optionally, before determining that there is a damaged entry in the secondary address table of the virtual disk, the method further includes:
从虚拟磁盘中获取所述二级地址表,并将二级地址表中的每个表项依次设置为待检测表项;Obtain the secondary address table from the virtual disk, and set each entry in the secondary address table as an entry to be detected in sequence;
判断利用待检测表项是否查找到对应的用户数据;和Determine whether the corresponding user data is found using the table item to be detected; and
响应于利用待检测表项不能查找到对应的用户数据,确定待检测表项为损坏表项。In response to the failure to find the corresponding user data using the entry to be detected, it is determined that the entry to be detected is a damaged entry.
可选地,利用备份表项对损坏表项进行修复,包括:Optionally, the damaged table entry is repaired using the backup table entry, including:
将备份表项重新写回至损坏表项在虚拟磁盘的二级地址表中的对应位置,以完成对损坏表项的修复。The backup entry is rewritten back to the corresponding position of the damaged entry in the secondary address table of the virtual disk to complete the repair of the damaged entry.
可选地,还包括: Optionally, it also includes:
定期利用虚拟磁盘中的数据对备份磁盘中的数据进行备份更新。The data in the backup disk is backed up and updated periodically using the data in the virtual disk.
可选地,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,包括:Optionally, a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, including:
跳过虚拟磁盘头部预设大小的磁盘数据,并在剩余的磁盘数据中,依照从前到后的顺序将指定大小的数据依次设置为待验证数据;和Skipping disk data of a preset size in the virtual disk header, and setting data of a specified size in the remaining disk data as data to be verified in order from front to back; and
利用预设特征对待验证数据进行匹配验证,并在确定待验证数据通过匹配验证时,将待验证数据在虚拟磁盘中的目标地址写入反推二级地址表,以将目标地址作为反推二级地址表的表项。The data to be verified is matched and verified using preset features, and when it is determined that the data to be verified passes the matching verification, the target address of the data to be verified in the virtual disk is written into the reverse secondary address table, so that the target address is used as an entry in the reverse secondary address table.
可选地,指定大小为二级地址表的表项指向的用户数据的大小。Optionally, the specified size is the size of user data pointed to by the entry of the secondary address table.
可选地,利用预设特征对待验证数据进行匹配验证,包括:Optionally, matching verification is performed on the data to be verified using preset features, including:
在确定待验证数据的首部和尾部并不同时包含一级地址表的关键字,以及待验证数据与引用计数表的关键字不同时,确定待验证数据通过匹配验证。When it is determined that the header and the tail of the data to be verified do not simultaneously contain the keyword of the primary address table, and the keywords of the data to be verified and the reference count table are different, it is determined that the data to be verified passes the matching verification.
可选地,利用反推二级地址表对损坏表项进行修复,包括:Optionally, the damaged entry is repaired by using the reverse secondary address table, including:
查找损坏表项在二级地址表中对应的前一个正常表项;和Find the previous normal entry corresponding to the damaged entry in the secondary address table; and
确定正常表项在反推二级地址表中对应的目标表项,并利用目标表项的后一表项对损坏表项进行修复。Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use the next entry of the target entry to repair the damaged entry.
可选地,正常表项的表项内容小于损坏表项对应的正确表项内容;Optionally, the entry content of the normal entry is smaller than the correct entry content corresponding to the damaged entry;
目标表项的表项内容与正常表项的表项内容相同;The content of the target entry is the same as that of the normal entry.
后一表项的表项内容大于目标表项的表项内容。The entry content of the latter entry is greater than the entry content of the target entry.
可选地,利用反推二级地址表对损坏表项进行修复,包括:Optionally, the damaged entry is repaired by using the reverse secondary address table, including:
在二级地址表中确定由连续的已损坏表项构成的损坏表项组,并查找损坏表项组对应的前一个正常表项;和Determining a damaged entry group consisting of consecutive damaged entries in the secondary address table, and searching for a previous normal entry corresponding to the damaged entry group; and
确定正常表项在反推二级地址表中对应的目标表项,并利用目标表项之后指定数量的表项对损坏表项组进行修复;指定数量为损坏表项组所包含的损坏表项数量。Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use a specified number of entries after the target entry to repair the damaged entry group; the specified number is the number of damaged entries included in the damaged entry group.
可选地,在利用反推二级地址表对损坏表项进行修复之后,还包括:Optionally, after repairing the damaged entry by using the reverse secondary address table, the method further includes:
将已修复的损坏表项标记为已修复表项,并确定已修复表项在二级地址表中的目标索引值;Mark the repaired damaged table entry as a repaired table entry, and determine the target index value of the repaired table entry in the secondary address table;
重新启动虚拟磁盘所属的虚拟机,并在虚拟机中确定已修复表项对应的用户数据是否恢复;和Restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry is restored; and
响应于在所述虚拟机中确定所述已修复表项对应的用户数据未恢复,则在预设偏移区间内为目标索引值增加偏移值,以移动已修复表项在二级地址表中所处的位置;偏移 值为预设数据簇大小的整数倍。In response to determining in the virtual machine that the user data corresponding to the repaired entry has not been restored, an offset value is added to the target index value within a preset offset interval to move the position of the repaired entry in the secondary address table; offset The value is an integer multiple of the preset data cluster size.
可选地,在预设偏移区间内为目标索引值增加偏移值之前,还包括:Optionally, before adding the offset value to the target index value within the preset offset interval, the method further includes:
确定由连续的已修复表项构成的已修复表项组,并确定与已修复表项组在二级地址表中对应的前一正常表项和后一正常表项;Determine a repaired table entry group consisting of continuous repaired table entries, and determine a previous normal table entry and a next normal table entry corresponding to the repaired table entry group in the secondary address table;
在二级地址表中确定前一正常表项对应的第一索引值和后一正常表项对应的第二索引值;和Determining in the secondary address table a first index value corresponding to a previous normal entry and a second index value corresponding to a next normal entry; and
利用第一索引值、第二索引值及预设数据簇大小确定与已修复表项组对应的预设偏移区间;Determine a preset offset interval corresponding to the repaired table entry group using the first index value, the second index value and the preset data cluster size;
相应的,在预设偏移区间内为目标索引值增加偏移值,包括:Accordingly, an offset value is added to the target index value within a preset offset interval, including:
在已修复表项所属的已修复表项组对应的预设偏移区间内为目标索引值增加偏移值。An offset value is added to the target index value within a preset offset interval corresponding to the repaired entry group to which the repaired entry belongs.
可选地,在预设偏移区间内为目标索引值增加偏移值之后,还包括:Optionally, after adding the offset value to the target index value within the preset offset interval, the method further includes:
重新启动虚拟磁盘所属的虚拟机,并在虚拟机中确定增加了偏移值的已修复表项对应的用户数据是否恢复;和Restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry with the increased offset value is restored; and
响应于在虚拟机中确定增加了偏移值的已修复表项对应的用户数据未恢复,则在预设偏移区间内调整偏移值,为已修复表项的目标索引值增加调整后的偏移值,并进入重新启动虚拟磁盘所属的虚拟机的步骤。In response to determining in the virtual machine that the user data corresponding to the repaired table entry with the added offset value has not been restored, the offset value is adjusted within a preset offset interval, the adjusted offset value is added to the target index value of the repaired table entry, and the step of restarting the virtual machine to which the virtual disk belongs is entered.
本申请还提供一种元数据修复装置,包括:The present application also provides a metadata repair device, comprising:
备份表项查询模块,用于在确定虚拟磁盘的二级地址表中存在损坏表项时,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项;二级地址表的各个表项用于记录虚拟磁盘中各个用户数据的地址;A backup table entry query module is used to query the backup table entry corresponding to the damaged table entry in the backup disk corresponding to the virtual disk when it is determined that there is a damaged table entry in the secondary address table of the virtual disk; each table entry in the secondary address table is used to record the address of each user data in the virtual disk;
第一修复模块,用于当查找到备份表项时,利用备份表项对损坏表项进行修复;和A first repair module, configured to repair a damaged entry using the backup entry when a backup entry is found; and
第二修复模块,用于当未查找到备份表项时,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复。The second repair module is used to construct a reverse secondary address table by scanning the addresses of all user data in the virtual disk when no backup table entry is found, and to repair the damaged table entry using the reverse secondary address table.
本申请还提供一种电子设备,包括:The present application also provides an electronic device, comprising:
存储器,用于存储计算机可读指令;a memory for storing computer readable instructions;
一个或多个处理器,用于执行计算机可读指令时实现如上的元数据修复方法。One or more processors are used to implement the above metadata repair method when executing computer-readable instructions.
本申请还提供一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行如上的元数据修复方法。 The present application also provides one or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the metadata repair method as described above.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
图1为本申请一个或多个实施例所提供的一种元数据修复方法的流程图;FIG1 is a flowchart of a metadata repair method provided by one or more embodiments of the present application;
图2为本申请一个或多个实施例所提供的第一种二级地址表表项修复的示意图;FIG2 is a schematic diagram of a first type of secondary address table entry repair provided by one or more embodiments of the present application;
图3为本申请一个或多个实施例所提供的第二种二级地址表表项修复的示意图;FIG3 is a schematic diagram of a second type of secondary address table entry repair provided by one or more embodiments of the present application;
图4为本申请一个或多个实施例所提供的第三种二级地址表表项修复的示意图;FIG4 is a schematic diagram of a third type of secondary address table entry repair provided by one or more embodiments of the present application;
图5为本申请一个或多个实施例所提供的一种元数据修复装置的结构框图;FIG5 is a structural block diagram of a metadata repair device provided by one or more embodiments of the present application;
图6为本申请一个或多个实施例所提供的另一种元数据修复装置的结构框图;FIG6 is a structural block diagram of another metadata repair device provided by one or more embodiments of the present application;
图7为本申请一个或多个实施例所提供的一种电子设备的结构框图;FIG7 is a structural block diagram of an electronic device provided by one or more embodiments of the present application;
图8为本申请一个或多个实施例所提供的一种非易失性计算机可读存储介质的结构框图。FIG8 is a block diagram of a non-volatile computer-readable storage medium provided by one or more embodiments of the present application.
具体实施方式Detailed ways
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
相关技术中,一级地址表(L1 Table)和二级地址表(L2 Table)是虚拟磁盘进行地址管理所需的重要元数据,其中二级地址表用于存放用户数据簇在虚拟磁盘中的地址。然而,当***断电、文件***异常时,二级地址表缓存可能出现数据紊乱的情况,进而导致二级地址表的数据无法下刷到虚拟磁盘、数据内容转变为随机数等问题。此时,该二级地址表便无法用于寻址,进而就造成了虚拟磁盘的数据丢失问题。有鉴于此,本申请可提供一种元数据修复方法,可采用备份数据和通过扫描虚拟磁盘中所有用户数据地址的方式构建的反推二级地址表对虚拟磁盘的二级地址表中的损坏表项进行修复,以尽可能修复丢失的用户数据。请参考图1,图1为本申请实施例所提供的一种元数据修复方 法的流程图,该方法可以包括:In the related art, the first-level address table (L1 Table) and the second-level address table (L2 Table) are important metadata required for address management of the virtual disk, wherein the second-level address table is used to store the address of the user data cluster in the virtual disk. However, when the system is powered off or the file system is abnormal, data disorder may occur in the second-level address table cache, which may lead to problems such as the data in the second-level address table cannot be flushed to the virtual disk and the data content is converted into random numbers. At this time, the second-level address table cannot be used for addressing, which causes the problem of data loss of the virtual disk. In view of this, the present application may provide a metadata repair method, which may use backup data and a reverse second-level address table constructed by scanning all user data addresses in the virtual disk to repair damaged entries in the second-level address table of the virtual disk, so as to repair the lost user data as much as possible. Please refer to Figure 1, which is a metadata repair method provided in an embodiment of the present application. A flow chart of a method, the method may include:
S101、在确定虚拟磁盘的二级地址表中存在损坏表项时,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项;二级地址表的各个表项用于记录虚拟磁盘中各个用户数据的地址。S101. When it is determined that there is a damaged entry in the secondary address table of the virtual disk, query the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk; each entry in the secondary address table is used to record the address of each user data in the virtual disk.
首先应当指出的是,二级地址表中的每个表项均用于保存对应的用户数据在虚拟磁盘中的地址,而虚拟磁盘正是依靠二级地址表中的各个表项进行用户数据寻址。这意味着假若在二级地址表中发现了损坏的表项,则该表项中保存的用户数据地址便无效,虚拟磁盘将无法利用该损坏表项查找到对应的用户数据,进而便意味着该用户数据丢失。因此,假若能够对二级地址表中损坏的表项进行修复,恢复二级地址表与用户数据之间的对应关系,便意味着找回了丢失的用户数据。First of all, it should be pointed out that each entry in the secondary address table is used to store the address of the corresponding user data in the virtual disk, and the virtual disk relies on each entry in the secondary address table to address the user data. This means that if a damaged entry is found in the secondary address table, the user data address stored in the entry will be invalid, and the virtual disk will not be able to use the damaged entry to find the corresponding user data, which means that the user data is lost. Therefore, if the damaged entry in the secondary address table can be repaired and the corresponding relationship between the secondary address table and the user data can be restored, it means that the lost user data has been recovered.
本申请实施例首先可为虚拟磁盘引入对应的备份磁盘,并利用该备份磁盘对其保存的用户数据、一级地址表、二级地址表等进行备份。进而,在发现虚拟磁盘的二级地址表中存在损坏表项时,便可利用备份磁盘中对应的备份表项对损坏表项进行修复,以达到恢复用户数据的效果。在实际应用中,备份磁盘可从物理上与虚拟磁盘所在的生产存储池相隔离,这样在生产存储池遭遇机械或电力故障时,备份存储池将不受影响,尽可最大程度地保护用户数据安全。当然,需说明的是,在本申请实施例中,仅会使用备份磁盘恢复虚拟磁盘二级地址表中存在故障的损坏表项,但并不会修复该表项对应的用户数据。这是由于用户数据本身可能并未存在损坏,仅是其在二级地址表中对应的表项存在异常;同时,受备份周期的影响,备份磁盘中的用户数据可能与虚拟磁盘中的用户数据不完全相同,进而若采用备份磁盘对损坏表项对应的用户数据也进行修复,则容易导致用户数据损坏,给用户的正常使用带来不便。In the embodiment of the present application, a corresponding backup disk can be first introduced for the virtual disk, and the backup disk can be used to back up the user data, the primary address table, the secondary address table, etc. stored therein. Then, when it is found that there is a damaged entry in the secondary address table of the virtual disk, the corresponding backup entry in the backup disk can be used to repair the damaged entry to achieve the effect of restoring the user data. In practical applications, the backup disk can be physically isolated from the production storage pool where the virtual disk is located, so that when the production storage pool encounters a mechanical or power failure, the backup storage pool will not be affected, and the user data security can be protected to the greatest extent. Of course, it should be noted that in the embodiment of the present application, only the backup disk will be used to restore the damaged entry in the secondary address table of the virtual disk, but the user data corresponding to the entry will not be repaired. This is because the user data itself may not be damaged, but only the corresponding entry in the secondary address table is abnormal; at the same time, affected by the backup cycle, the user data in the backup disk may not be exactly the same as the user data in the virtual disk, and then if the backup disk is used to repair the user data corresponding to the damaged entry, it is easy to cause user data damage, which brings inconvenience to the normal use of the user.
进一步,需要说明的是,本申请实施例并不限定如何确定上述损坏表项的检测方法,如可根据能否利用该表项查找到对应的用户数据来确定该表项是否损坏,也可以根据该表项能否被预设的数据簇大小整除来确定该表项是否损坏,其中数据簇(Cluster)是虚拟磁盘中的最小存储单位,每个用户数据可采用对应的用户数据簇(Data Cluster)保存,进而每个用户数据的地址应当为数据簇大小的整数倍。考虑到采用预设数据簇大小进行检测效率较高,因此在本申请实施例中将采用这种方法对二级地址表中的各个表项进行检测。Further, it should be noted that the embodiments of the present application do not limit the detection method of determining the above-mentioned damaged table entries. For example, whether the table entry is damaged can be determined based on whether the corresponding user data can be found using the table entry, or whether the table entry is damaged can be determined based on whether the table entry can be divided by a preset data cluster size, wherein a data cluster is the smallest storage unit in a virtual disk, and each user data can be stored using a corresponding user data cluster, and the address of each user data should be an integer multiple of the data cluster size. Considering that the detection efficiency using a preset data cluster size is high, this method will be used in the embodiments of the present application to detect each table entry in the secondary address table.
在一种可能的情况中,在确定虚拟磁盘的二级地址表中存在损坏表项之前,还可以包括:从虚拟磁盘中获取二级地址表,并将二级地址表中的每个表项依次设置为待检测 表项;判断利用待检测表项是否查找到对应的用户数据;和响应于利用待检测表项不能查找到对应的用户数据,确定待检测表项为损坏表项。In a possible case, before determining that there is a damaged entry in the secondary address table of the virtual disk, the method may further include: obtaining the secondary address table from the virtual disk, and setting each entry in the secondary address table as a to-be-detected item in turn. determining whether the corresponding user data is found using the entry to be detected; and in response to the use of the entry to be detected not being able to find the corresponding user data, determining that the entry to be detected is a damaged entry.
在一种可能的情况中,在确定虚拟磁盘的二级地址表中存在损坏表项之前,还可以包括:In a possible case, before determining that there is a damaged entry in the secondary address table of the virtual disk, the following may also be included:
步骤11:从虚拟磁盘中获取二级地址表,并将二级地址表中的每个表项依次设置为待检测表项;Step 11: Obtain the secondary address table from the virtual disk, and set each entry in the secondary address table as an entry to be detected in turn;
步骤12:判断待检测表项的数据内容能否被预设数据簇大小整除;若是,则进入步骤13;若否,则进入步骤14;Step 12: Determine whether the data content of the entry to be detected is divisible by the preset data cluster size; if so, proceed to step 13; if not, proceed to step 14;
步骤13:对下一个待检测表项执行步骤12;Step 13: Execute step 12 for the next entry to be detected;
具体地,判断任一待检测表项的数据内容能否被预设数据簇大小整除,响应于任一待检测表项的数据内容能被预设数据簇大小整除,判断任一待检测表项的下一项待检测表项的数据内容能否被预设数据簇大小整除;和响应于下一待检测表项的数据内容不能被预设数据簇大小整除,确定下一待检测表项为损坏表项。Specifically, it is determined whether the data content of any item to be detected can be divided by the preset data cluster size; in response to the data content of any item to be detected being divisible by the preset data cluster size, it is determined whether the data content of the next item to be detected of any item to be detected can be divided by the preset data cluster size; and in response to the data content of the next item to be detected not being divisible by the preset data cluster size, it is determined that the next item to be detected is a damaged item.
步骤14:确定待检测表项为损坏表项。Step 14: Determine that the entry to be detected is a damaged entry.
具体地,响应于任一待检测表项的数据内容不能被预设数据簇大小整除,确定任一待检测表项为损坏表项。Specifically, in response to the data content of any to-be-detected entry not being divisible by the preset data cluster size, it is determined that any to-be-detected entry is a damaged entry.
需要说明的是,在虚拟磁盘中可能设置有多个二级地址表,且每个二级地址表所占据的数据簇可能并不相邻,进而通过扫描虚拟磁盘的方式查找二级地址表并不方便。考虑到一级地址表的各个表项保存有各二级地址表在虚拟磁盘中的起始地址,因此本申请实施例可利用一级地址表进行二级地址表查询。It should be noted that there may be multiple secondary address tables in the virtual disk, and the data clusters occupied by each secondary address table may not be adjacent, so it is not convenient to search for the secondary address table by scanning the virtual disk. Considering that each entry of the primary address table stores the starting address of each secondary address table in the virtual disk, the embodiment of the present application can use the primary address table to query the secondary address table.
在一种可能的情况中,从虚拟磁盘中获取二级地址表,包括:In one possible case, obtaining the secondary address table from the virtual disk includes:
步骤21:从虚拟磁盘中获取一级地址表,并根据一级地址表的各个表项在虚拟磁盘中查找对应的二级地址表;一级地址表的表项用于记录各二级地址表在虚拟磁盘中的起始地址。Step 21: Obtain the primary address table from the virtual disk, and search for the corresponding secondary address table in the virtual disk according to each entry in the primary address table; the entries in the primary address table are used to record the starting address of each secondary address table in the virtual disk.
进一步,还需指出的是,为便于查询和管理,除了记录二级地址表的初始地址之外,一级地址表中还保存有二级地址表中各个表项的索引。同时,考虑到备份磁盘中的一级地址表与虚拟磁盘中的一级地址表在数据内容及表结构方面均是相同的,因此在查找到二级地址表中的损坏表项之后,还可确定该表项在虚拟磁盘的一级地址表中对应的索引,进而利用该索引及备份磁盘一级地址表在备份磁盘中查找对应的备份表项。Furthermore, it should be pointed out that, in order to facilitate query and management, in addition to recording the initial address of the secondary address table, the primary address table also stores the index of each table item in the secondary address table. At the same time, considering that the primary address table in the backup disk and the primary address table in the virtual disk are the same in terms of data content and table structure, after finding the damaged table item in the secondary address table, the corresponding index of the table item in the primary address table of the virtual disk can also be determined, and then the corresponding backup table item in the backup disk can be found by using the index and the primary address table of the backup disk.
在一种可能的情况中,在确定待检测表项为损坏表项之后,还可以包括: In a possible case, after determining that the entry to be detected is a damaged entry, the following steps may also be included:
步骤31:记录损坏表项在一级地址表中对应的索引;Step 31: Record the index corresponding to the damaged entry in the primary address table;
相应的,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项,可以包括:Accordingly, querying the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk may include:
步骤32:从备份磁盘中获取备份一级地址表,并利用索引及备份一级地址表在备份磁盘中查找备份表项。Step 32: Obtain the backup first-level address table from the backup disk, and use the index and the backup first-level address table to search for the backup table entry in the backup disk.
进一步,为确保备份磁盘中的数据可靠有效,还可定期利用虚拟磁盘中的数据对备份磁盘中的数据进行备份更新。Furthermore, in order to ensure that the data in the backup disk is reliable and effective, the data in the backup disk may be backed up and updated regularly using the data in the virtual disk.
在一种可能的情况中,本方法还可以包括:In one possible case, the method may further include:
步骤41:定期利用虚拟磁盘中的数据对备份磁盘中的数据进行备份更新。Step 41: Regularly back up and update the data in the backup disk using the data in the virtual disk.
需要说明的是,本申请实施例并不限定具体的备份更新周期,可根据实际应用需求进行设定。It should be noted that the embodiment of the present application does not limit the specific backup update cycle and can be set according to actual application requirements.
S102、当查找到备份表项时,利用备份表项对损坏表项进行修复。S102: When a backup entry is found, the damaged entry is repaired using the backup entry.
假若能够在备份磁盘中查找到与损坏表项对应的备份表项,则可将备份表项重新写回至损坏表项在虚拟磁盘二级地址表中的对应位置,以完成对损坏表项的修复。If the backup entry corresponding to the damaged entry can be found in the backup disk, the backup entry can be rewritten back to the corresponding position of the damaged entry in the virtual disk secondary address table to complete the repair of the damaged entry.
S103、当未查找到备份表项时,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复。S103: When no backup entry is found, a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged entry is repaired using the reverse secondary address table.
进一步,受备份周期等因素的影响,备份磁盘中可能并未保存损坏表项对应的备份表项,此时将无法利用备份数据完成对虚拟磁盘二级地址表的修复。也这是如此,本申请实施例可提供另一种修复二级地址表的方式,可通过扫描虚拟磁盘中所有用户数据地址的方式构建反推二级地址表(Maybe_L2_Table),并利用这一反推二级地址表对虚拟磁盘的二级地址表进行修复。简单来讲,虚拟磁盘中除了记录有用户数据之外,还记录有类似于一级地址表、二级地址表、引用计数表(Refcount Table)等元数据,而这批元数据具有显著的数据特征,进而可根据这些数据特征对虚拟磁盘中的数据进行筛选,筛除元数据内容并保留可能的用户数据,并利用这些可能的用户数据的地址构造反推二级地址表。随后,由于反推二级地址表与虚拟磁盘的二级地址表均保存有用户数据地址,即两者存在相似性,因此可根据两表之间相似性在反推二级表中查找与损失表项最为接近的反推表项,进而便可利用反推表项对损失表项进行修复。当然,反推二级地址表具体的构建过程,以及利用该表对损坏表项的修复过程还可包含其他优选的步骤,具体可参考后续实施例中的描述。Furthermore, affected by factors such as the backup cycle, the backup disk may not store the backup table entries corresponding to the damaged table entries. In this case, the backup data cannot be used to repair the virtual disk's secondary address table. This is also the case. The embodiment of the present application can provide another way to repair the secondary address table. A reverse secondary address table (Maybe_L2_Table) can be constructed by scanning all user data addresses in the virtual disk, and the reverse secondary address table can be used to repair the virtual disk's secondary address table. Simply put, in addition to recording user data, the virtual disk also records metadata such as the primary address table, the secondary address table, and the reference count table (Refcount Table). This batch of metadata has significant data features, and the data in the virtual disk can be screened based on these data features, filtering out the metadata content and retaining possible user data, and using the addresses of these possible user data to construct a reverse secondary address table. Subsequently, since both the reverse secondary address table and the secondary address table of the virtual disk store the user data address, that is, there is a similarity between the two tables, the reverse secondary table can be searched for the reverse entry closest to the loss entry based on the similarity between the two tables, and then the loss entry can be repaired using the reverse entry. Of course, the specific construction process of the reverse secondary address table and the repair process of the damaged entry using the table can also include other preferred steps, which can be specifically referred to the description in the subsequent embodiments.
基于上述实施例,本申请在确定虚拟磁盘的二级地址表中存在损坏表项时,会优先 在与虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项,并利用较为准确的备份数据优先修复损坏表项,以提升恢复用户数据的成功率;随后,假若无法从备份磁盘中查询到相关备份表项,则本申请还可通过扫描虚拟磁盘中所有用户数据地址的方式构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复,这是由于反推二级地址表可包含所有用户数据地址,且二级地址表中仍存在包含正确用户数据地址的正常表项,进而二级地址表与反推二级表之间存在相似性,本申请便可基于这一相似性,利用反推二级地址表对损坏表项进行修复,并尽可能地避免用户数据丢失。简单来讲,本申请可采用备份数据和上述反推二级地址表对虚拟磁盘的二级地址表中的损坏表项进行修复,以尽可能修复丢失的用户数据。Based on the above embodiment, when determining that there is a damaged entry in the secondary address table of the virtual disk, the present application will give priority to The backup table items corresponding to the damaged items are queried in the backup disk corresponding to the virtual disk, and the damaged items are repaired preferentially using the more accurate backup data to improve the success rate of recovering user data; subsequently, if the relevant backup items cannot be queried from the backup disk, the present application can also construct a reverse secondary address table by scanning all user data addresses in the virtual disk, and use the reverse secondary address table to repair the damaged items. This is because the reverse secondary address table can contain all user data addresses, and there are still normal items containing the correct user data addresses in the secondary address table, and then there is a similarity between the secondary address table and the reverse secondary table. Based on this similarity, the present application can use the reverse secondary address table to repair the damaged items and avoid user data loss as much as possible. Simply put, the present application can use the backup data and the above-mentioned reverse secondary address table to repair the damaged items in the secondary address table of the virtual disk to repair the lost user data as much as possible.
本申请提供一种元数据修复方法,包括:在确定虚拟磁盘的二级地址表中存在损坏表项时,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项;二级地址表的各个表项用于记录虚拟磁盘中各个用户数据的地址;当查找到备份表项时,利用备份表项对损坏表项进行修复;当未查找到备份表项时,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复。The present application provides a metadata repair method, comprising: when determining that there is a damaged table entry in the secondary address table of a virtual disk, querying the backup table entry corresponding to the damaged table entry in the backup disk corresponding to the virtual disk; each table entry in the secondary address table is used to record the address of each user data in the virtual disk; when the backup table entry is found, the damaged table entry is repaired by using the backup table entry; when the backup table entry is not found, a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged table entry is repaired by using the reverse secondary address table.
可见,本申请在确定虚拟磁盘的二级地址表中存在损坏表项时,会优先在与虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项,并利用较为准确的备份数据优先修复损坏表项,以提升恢复用户数据的成功率;随后,假若无法从备份磁盘中查询到相关备份表项,则本申请还可通过扫描虚拟磁盘中所有用户数据地址的方式构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复,这是由于反推二级地址表可包含所有用户数据地址,且二级地址表中仍存在包含正确用户数据地址的正常表项,进而二级地址表与反推二级表之间存在相似性,本申请便可基于这一相似性,利用反推二级地址表对损坏表项进行修复,并尽可能地避免用户数据丢失。简单来讲,本申请可采用备份数据和上述反推二级地址表对虚拟磁盘的二级地址表中的损坏表项进行修复,以尽可能修复丢失的用户数据。It can be seen that when the present application determines that there is a damaged entry in the secondary address table of the virtual disk, it will first query the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk, and use the more accurate backup data to repair the damaged entry first, so as to improve the success rate of recovering user data; then, if the relevant backup entry cannot be queried from the backup disk, the present application can also construct a reverse secondary address table by scanning all user data addresses in the virtual disk, and use the reverse secondary address table to repair the damaged entry. This is because the reverse secondary address table can contain all user data addresses, and there are still normal entries containing correct user data addresses in the secondary address table, and then there is a similarity between the secondary address table and the reverse secondary table. Based on this similarity, the present application can use the reverse secondary address table to repair the damaged entry and avoid user data loss as much as possible. Simply put, the present application can use backup data and the above-mentioned reverse secondary address table to repair the damaged entry in the secondary address table of the virtual disk to repair the lost user data as much as possible.
基于上述实施例,下面对反推二级地址表的构建方式及利用该表对二级地址表中的损坏表项进行修复的过程进行详细介绍。在一种可能的情况中,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,包括:Based on the above embodiment, the following describes in detail the construction method of the reverse secondary address table and the process of using the table to repair damaged entries in the secondary address table. In one possible case, the reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, including:
S201、跳过虚拟磁盘头部预设大小的磁盘数据,并在剩余的磁盘数据中,依照从前到后的顺序将指定大小的数据依次设置为待验证数据。S201 , skipping disk data of a preset size in the virtual disk header, and setting data of a specified size in the remaining disk data as data to be verified in sequence from front to back.
在进行虚拟磁盘扫描时,为了提升扫描效率,可首先跳过虚拟磁盘头部预设大小的 磁盘数据。这是由于此部分数据主要为虚拟磁盘文件(qcow2文件)的文件头,并不包含有效的用户数据,因此可无需扫描。本申请实施例并不限定具体的预设大小,例如可以为3个数据簇大小。在跳过文件头后,本申请实施将对剩余的磁盘数据按照从前到后的顺序依次进行扫描。具体的,本申请实施例依照从前到后的顺序将指定大小的数据依次设置为待验证数据,该指定大小是一个二级地址表表项可指向的用户数据大小,例如可以为64比特(bit)。换句话说,本申请实施例会从剩余的磁盘数据中依次提取64比特的数据进行检测。When scanning a virtual disk, you can first skip the virtual disk header with a preset size to improve scanning efficiency. Disk data. This is because this part of the data is mainly the file header of the virtual disk file (qcow2 file) and does not contain valid user data, so it does not need to be scanned. The embodiment of the present application does not limit the specific preset size, for example, it can be 3 data clusters. After skipping the file header, the implementation of the present application will scan the remaining disk data in order from front to back. Specifically, the embodiment of the present application sets the data of a specified size as the data to be verified in order from front to back. The specified size is the user data size that a secondary address table entry can point to, for example, it can be 64 bits (bit). In other words, the embodiment of the present application will extract 64 bits of data from the remaining disk data in sequence for detection.
S201、利用预设特征对待验证数据进行匹配验证,并在确定待验证数据通过匹配验证时,将待验证数据在虚拟磁盘中的目标地址写入反推二级地址表,以将目标地址作为反推二级地址表的表项。S201. Perform matching verification on the data to be verified using preset features, and when it is determined that the data to be verified passes the matching verification, write the target address of the data to be verified in the virtual disk into the reverse secondary address table, so as to use the target address as an entry in the reverse secondary address table.
本申请实施例具体会采用元数据的相关特征对待验证数据进行匹配验证。具体的,可采用一级地址表的关键字和引用计数表的关键字进行验证,其中一级地址表的关键字为0X8000*0000,其中*代表通配符,即跳过开头为8000,0000为后缀的字段;而引用计数表的关键字为0X0001000100010001。假若确定待验证数据中不包含上述关键字,则大致可判定待验证数据属于用户数据,进而便可判定待验证数据通过验证,并可将其在虚拟磁盘中的地址写入反推二级地址表。The embodiment of the present application will specifically use the relevant features of the metadata to match and verify the data to be verified. Specifically, the keywords of the primary address table and the reference count table can be used for verification, where the keyword of the primary address table is 0X8000*0000, where * represents a wildcard, that is, skipping the fields starting with 8000 and suffixed with 0000; and the keyword of the reference count table is 0X0001000100010001. If it is determined that the data to be verified does not contain the above keywords, it can be roughly determined that the data to be verified belongs to user data, and then it can be determined that the data to be verified has passed the verification, and its address in the virtual disk can be written into the reverse secondary address table.
在一种可能的情况中,利用预设特征对待验证数据进行匹配验证,可以包括:In a possible case, using preset features to perform matching verification on the data to be verified may include:
步骤51:在确定待验证数据的首部和尾部并不同时包含一级地址表的关键字,以及待验证数据与引用计数表的关键字不同时,确定待验证数据通过匹配验证。Step 51: When it is determined that the header and the tail of the data to be verified do not contain the keyword of the primary address table at the same time, and the keyword of the data to be verified is different from the keyword of the reference count table, it is determined that the data to be verified passes the matching verification.
值得指出的是,由于本申请实施例对虚拟磁盘的扫描是依照从前至后的顺序进行的,因此反推二级地址表中保存的地址也是依次递增的。It is worth pointing out that, since the scanning of the virtual disk in the embodiment of the present application is performed in a sequence from front to back, the addresses stored in the reverse secondary address table are also incremented in sequence.
下面介绍利用反推二级地址表修复损坏表项的具体步骤。如上所述,反推二级地址表与虚拟磁盘中的二级地址表存在相似性,因此可基于这种相似性在反推二级地址表中查找与损坏表项最接近的反推表项,并利用反推表项对损坏表项进行修复。为便于理解,请参考图2,图2为本申请实施例所提供的第一种二级地址表表项修复的示意图,其中左侧为反推二级地址表,右侧为虚拟磁盘的二级地址表,标记13所标出的表项(表项内容为0x111111111)为损坏表项。本申请实施例可在原二级地址表中查找损坏表项的前一个正常表项(即标记14所标出的表项,表项内容为0x33c8e0000),该正常表项与损坏表项紧邻,且其表项内容小于损坏表项对应的正确表项内容。进而,可在反推二级地址表中查找与正常表项对应的、内容完全相同的目标表项(即标记12所标出的表项,表项内 容为0x33c8e0000),并利用目标表项的后一表项(即标记11所标出的表项,表项内容为0x33cb80000)对损坏表项进行修复,其中目标表项的后一表项是指与目标表项紧邻的、表项内容大于目标表项的表项。The specific steps of repairing the damaged entry using the reverse secondary address table are described below. As described above, there is similarity between the reverse secondary address table and the secondary address table in the virtual disk, so based on this similarity, the reverse entry closest to the damaged entry can be found in the reverse secondary address table, and the damaged entry can be repaired using the reverse entry. For ease of understanding, please refer to Figure 2, which is a schematic diagram of the first secondary address table entry repair provided by an embodiment of the present application, wherein the left side is the reverse secondary address table, the right side is the secondary address table of the virtual disk, and the entry marked by mark 13 (the entry content is 0x111111111) is a damaged entry. The embodiment of the present application can search for the previous normal entry of the damaged entry in the original secondary address table (i.e., the entry marked by mark 14, the entry content is 0x33c8e0000), the normal entry is adjacent to the damaged entry, and its entry content is less than the correct entry content corresponding to the damaged entry. Then, the target entry (i.e., the entry marked with mark 12, the content of which is exactly the same as the normal entry) corresponding to the inverse secondary address table can be searched. The damaged entry is repaired by using the next entry of the target entry (i.e., the entry marked by marker 11, whose content is 0x33cb80000), wherein the next entry of the target entry refers to the entry that is adjacent to the target entry and whose content is greater than the target entry.
在一种可能的情况中,利用反推二级地址表对损坏表项进行修复,可以包括:In one possible case, repairing the damaged entry by inverting the secondary address table may include:
步骤61:查找损坏表项在二级地址表中对应的前一个正常表项;Step 61: Find the previous normal entry corresponding to the damaged entry in the secondary address table;
步骤62:确定正常表项在反推二级地址表中对应的目标表项,并利用目标表项的后一表项对损坏表项进行修复。Step 62: Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use the next entry of the target entry to repair the damaged entry.
进一步,假若二级地址表中存在若干个相邻的损坏表项,则也可对它们进行批量修复。为便于理解,请参考图3,图3为本申请实施例所提供的第二种二级地址表表项修复的示意图,其中侧为反推二级地址表,右侧为虚拟磁盘的二级地址表,标记23所标出的4个连续的表项(表项内容分别为0x111111111、0x222222222、0x333333333、0x444444444)均为损坏表项,这4个损坏表项可组成一个损坏表项组。本申请实施例可在原二级地址表中该查找损坏表项组的前一个正常表项(即标记24所标出的表项,表项内容为0x33c8e0000),该正常表项与损坏表项组紧邻,且其表项内容小于损坏表项组中最小的正确表项内容。进而,可在反推二级地址表中查找与正常表项对应的、内容完全相同的目标表项(即标记22所标出的表项,表项内容为0x33c8e0000),并利用目标表项后的4个表项(即标记21所指向的4个连续的表项,表项内容依次为0x33cb80000、0x33cb80000、0x33ce70000及0x33d200000)对损坏表项组进行修复。Further, if there are several adjacent damaged entries in the secondary address table, they can also be repaired in batches. For ease of understanding, please refer to Figure 3, which is a schematic diagram of the second type of secondary address table entry repair provided by the embodiment of the present application, wherein the side is the reverse secondary address table, the right side is the secondary address table of the virtual disk, and the 4 consecutive entries marked by mark 23 (the entry contents are 0x111111111, 0x222222222, 0x333333333, 0x444444444) are all damaged entries, and these 4 damaged entries can form a damaged entry group. The embodiment of the present application can search for the previous normal entry of the damaged entry group in the original secondary address table (i.e., the entry marked by mark 24, the entry content is 0x33c8e0000), the normal entry is adjacent to the damaged entry group, and its entry content is less than the smallest correct entry content in the damaged entry group. Furthermore, a target entry corresponding to the normal entry and with exactly the same content as the normal entry can be searched in the reverse secondary address table (i.e., the entry marked by marker 22, whose content is 0x33c8e0000), and the damaged entry group can be repaired using the four entries after the target entry (i.e., the four consecutive entries pointed to by marker 21, whose contents are 0x33cb80000, 0x33cb80000, 0x33ce70000, and 0x33d200000, respectively).
在一种可能的情况中,利用反推二级地址表对损坏表项进行修复,可以包括:In one possible case, repairing the damaged entry by inverting the secondary address table may include:
步骤71:在二级地址表中确定由连续的已损坏表项构成的损坏表项组,并查找损坏表项组对应的前一个正常表项;Step 71: determining a damaged entry group consisting of consecutive damaged entries in the secondary address table, and searching for a previous normal entry corresponding to the damaged entry group;
步骤71:确定正常表项在反推二级地址表中对应的目标表项,并利用目标表项之后指定数量的表项对损坏表项组进行修复;指定数量为损坏表项组所包含的损坏表项数量。Step 71: Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use a specified number of entries after the target entry to repair the damaged entry group; the specified number is the number of damaged entries included in the damaged entry group.
基于上述实施例,本申请可通过扫描虚拟磁盘中所有用户数据地址的方式构造反推二级地址表,并基于该表与原二级地址表间的相似性,对损坏表项进行修复,可进一步对备份数据无法修复的损坏表项进行修复,进一步提升了用户数据恢复的有效程度。Based on the above embodiments, the present application can construct a reverse secondary address table by scanning all user data addresses in the virtual disk, and repair damaged table entries based on the similarity between the table and the original secondary address table. It can further repair damaged table entries that cannot be repaired by backup data, thereby further improving the effectiveness of user data recovery.
基于上述实施例,反推二级地址表是利用可能的用户数据地址对损坏表项进行修复,但在构建反推二级地址表时难免会遗漏部分用户数据地址,进而会导致虚拟磁盘二级地址表出现用户地址缺漏的情况,进而导致用户数据难以恢复。例如,请参考图4,图 4为本申请实施例所提供的第三种二级地址表表项修复的示意图,其中左侧表示正确的二级地址表,而右侧为错误的二级地址表。在对该表进行恢复得到右侧所示的二级地址表时,尽管由标记32所标出的四个表项所包含的表项内容与正确的二级地址表中由标记31所标出的两个表项之后的4个表项完全一致,但由于缺漏的标记31所标出的两个表项的内容,不仅导致标记32所标出的四个表项对应的四个地址完全无效,同时还额外导致标记32所标出的四个表项后面的两个表项无法恢复,进而造成6个用户数据丢失。为此,本申请实施例还可对已修复但并未恢复对应用户数据的表项进行位置调整,以尽可能恢复用户数据。在一种可能的情况中,在利用反推二级地址表对损坏表项进行修复之后,还可以包括:Based on the above embodiment, the reverse secondary address table is to repair the damaged table entries using possible user data addresses, but it is inevitable that some user data addresses will be missed when constructing the reverse secondary address table, which will cause the virtual disk secondary address table to have missing user addresses, making it difficult to recover user data. For example, please refer to FIG. 4 is a schematic diagram of the third type of secondary address table entry repair provided by the embodiment of the present application, wherein the left side represents the correct secondary address table, and the right side represents the wrong secondary address table. When the table is restored to obtain the secondary address table shown on the right, although the table entry content contained in the four entries marked by mark 32 is completely consistent with the four entries after the two entries marked by mark 31 in the correct secondary address table, due to the missing content of the two entries marked by mark 31, not only the four addresses corresponding to the four entries marked by mark 32 are completely invalid, but also the two entries after the four entries marked by mark 32 cannot be restored, thereby causing 6 user data to be lost. For this reason, the embodiment of the present application can also adjust the position of the table entries that have been repaired but the corresponding user data has not been restored, so as to restore the user data as much as possible. In a possible situation, after the damaged table entries are repaired by using the reverse secondary address table, it can also include:
S301、将已修复的损坏表项标记为已修复表项,并确定已修复表项在二级地址表中的目标索引值。S301: Mark the repaired damaged entry as a repaired entry, and determine a target index value of the repaired entry in the secondary address table.
具体的,图4中表项组32所包含的表项均可被标记为已修复表项。应当特别指出的是,本申请实施例还需额外确定已修复表项在二级地址表中的目标索引值,即确定已修复表项当前在二级地址表中所处的位置。Specifically, all entries included in entry group 32 in Figure 4 can be marked as repaired entries. It should be particularly noted that the present embodiment further requires determining the target index value of the repaired entry in the secondary address table, that is, determining the current position of the repaired entry in the secondary address table.
S302、重新启动虚拟磁盘所属的虚拟机,并在虚拟机中确定已修复表项对应的用户数据是否恢复;若是,则判定修复成功;若否,则进入步骤S303。S302, restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry is restored; if so, determining that the repair is successful; if not, proceeding to step S303.
具体的,由于标记32所标出的表项前缺漏的标记31所标出的表项的数据内容,进而导致标记32所标出的表项对应的用户数据均无法读取。本申请实施例正是对如同标记32标出的这类已修复但无法恢复用户数据的表项进行额外处理。Specifically, the user data corresponding to the entry marked by marker 32 cannot be read due to the missing data content of the entry marked by marker 31 before the entry marked by marker 32. The embodiment of the present application performs additional processing on the entry marked by marker 32 that has been repaired but cannot restore the user data.
S303、在预设偏移区间内为目标索引值增加偏移值,以移动已修复表项在二级地址表中所处的位置;偏移值为预设数据簇大小的整数倍。S303: adding an offset value to the target index value within a preset offset interval to move the position of the repaired entry in the secondary address table; the offset value is an integer multiple of a preset data cluster size.
具体地,响应于在虚拟机中确定已修复表项对应的用户数据未恢复,在预设偏移区间内为目标索引值增加偏移值,以移动已修复表项在二级地址表中所处的位置;偏移值为预设数据簇大小的整数倍。Specifically, in response to determining in the virtual machine that the user data corresponding to the repaired table entry has not been restored, an offset value is added to the target index value within a preset offset interval to move the position of the repaired table entry in the secondary address table; the offset value is an integer multiple of the preset data cluster size.
本申请实施例可为这些已修复表项的目标索引值继续增加偏移值,以尝试将这些已修复表项移动至其他位置,并确定其是否能对其他位置对应的用户数据进行恢复。应当指出的是,由于数据簇为虚拟磁盘中的最小存储单位,因此偏移值应当为预设数据簇大小的整数倍;同时,假若为已修复表项的目标索引值(或已修复表项组的目标索引值)增加偏移值后,其已偏移至正常表项所处的位置,则已修复表项将被正常表项覆盖,该偏移值起不到修复效果,因此为避免使用无效的偏移值,本申请实施例可在预设偏移区 间内增加偏移值。显然,预设偏移区间的数值与已修复表项相邻的正常表项有关。具体的,预设偏移区间的确定方式如下:The embodiment of the present application can continue to increase the offset value for the target index value of these repaired entries to try to move these repaired entries to other locations and determine whether they can recover the user data corresponding to other locations. It should be noted that since the data cluster is the smallest storage unit in the virtual disk, the offset value should be an integer multiple of the preset data cluster size; at the same time, if the target index value of the repaired entry (or the target index value of the repaired entry group) is increased by the offset value, it has shifted to the position of the normal entry, then the repaired entry will be overwritten by the normal entry, and the offset value will not have a repair effect. Therefore, in order to avoid using invalid offset values, the embodiment of the present application can be in the preset offset area. The offset value is increased within the interval. Obviously, the value of the preset offset interval is related to the normal table entry adjacent to the repaired table entry. Specifically, the preset offset interval is determined as follows:
在一种可能的情况中,在预设偏移区间内为目标索引值增加偏移值之前,还包括:In a possible case, before adding the offset value to the target index value within the preset offset interval, the method further includes:
步骤81:确定由连续的已修复表项构成的已修复表项组,并确定与已修复表项组在二级地址表中对应的前一正常表项和后一正常表项;Step 81: determining a repaired entry group consisting of continuous repaired entries, and determining a previous normal entry and a next normal entry corresponding to the repaired entry group in the secondary address table;
步骤82:在二级地址表中确定前一正常表项对应的第一索引值和后一正常表项对应的第二索引值。Step 82: Determine in the secondary address table a first index value corresponding to a previous normal entry and a second index value corresponding to a next normal entry.
应当指出的是,“第一”和“第二”仅用于区分前一正常表项和后一正常表项的索引值,并不代表这两个正常表项在二级地址表中带有多个表项。It should be noted that "first" and "second" are only used to distinguish the index values of the previous normal table entry and the next normal table entry, and do not mean that these two normal table entries have multiple entries in the secondary address table.
步骤83:利用第一索引值、第二索引值及预设数据簇大小确定与已修复表项组对应的预设偏移区间。Step 83: Determine a preset offset interval corresponding to the repaired entry group using the first index value, the second index value and the preset data cluster size.
具体的,预设偏移区间的上限值应当为上述两个正常表项的索引值间隔减去预设数据簇大小。Specifically, the upper limit of the preset offset interval should be the interval between the index values of the above two normal table entries minus the preset data cluster size.
相应的,在预设偏移区间内为目标索引值增加偏移值,可以包括:Accordingly, adding an offset value to the target index value within a preset offset interval may include:
步骤84:在已修复表项所属的已修复表项组对应的预设偏移区间内为目标索引值增加偏移值。Step 84: Add an offset value to the target index value within a preset offset interval corresponding to the repaired entry group to which the repaired entry belongs.
进一步,在完成偏移值添加之后,可进一步验证用户数据是否恢复,假若仍未恢复则可继续修改偏移值,直至完成恢复或判定恢复失败。Furthermore, after the offset value is added, it can be further verified whether the user data is restored. If it is still not restored, the offset value can be continuously modified until the restoration is completed or it is determined that the restoration has failed.
在一种可能的情况中,在预设偏移区间内为目标索引值增加偏移值值之后,还包括:In one possible case, after adding the offset value to the target index value within the preset offset interval, the method further includes:
S304、重新启动虚拟磁盘所属的虚拟机,并在虚拟机中确定增加了偏移值的已修复表项对应的用户数据是否恢复;若是,则判定修复成功;若否,则进入步骤S305。S304, restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry with the added offset value is restored; if so, determining that the repair is successful; if not, proceeding to step S305.
S305、若否,则在预设偏移区间内调整偏移值,为已修复表项的目标索引值增加调整后的偏移值,并进入重新启动虚拟磁盘所属的虚拟机的步骤。S305: If not, then adjust the offset value within the preset offset interval, add the adjusted offset value to the target index value of the repaired entry, and proceed to the step of restarting the virtual machine to which the virtual disk belongs.
具体地,响应于在虚拟机中确定增加了偏移值的已修复表项对应的用户数据未恢复,在预设偏移区间内调整偏移值,为已修复表项的目标索引值增加调整后的偏移值,并进入重新启动虚拟磁盘所属的虚拟机的步骤。Specifically, in response to determining in the virtual machine that the user data corresponding to the repaired table entry with the added offset value has not been restored, the offset value is adjusted within a preset offset interval, the adjusted offset value is added to the target index value of the repaired table entry, and the step of restarting the virtual machine to which the virtual disk belongs is entered.
基于上述实施例,本申请实施例还可对已修复但并未恢复对应用户数据的表项进行位置调整,以尽可能恢复用户数据。Based on the above embodiments, the embodiments of the present application may also adjust the positions of the table entries that have been repaired but the corresponding user data has not been restored, so as to restore the user data as much as possible.
下面基于具体的示意图介绍上述元数据修复方法。请参考图5,图5为本申请实施例 所提供的一种元数据修复装置的结构框图,下面对该装置所包含的模块及用途进行详细介绍。The following is an introduction to the metadata repair method based on a specific schematic diagram. Please refer to FIG5, which is an embodiment of the present application. A structural block diagram of a metadata repair device is provided, and the modules included in the device and their uses are introduced in detail below.
(1)磁盘坏块检查模块:用于检查损坏磁盘的L2表(L2Table)。它按照qcow2虚拟磁盘对应的L1->L2寻址规则,遍历检查L2表项(L2表的内容),并找出不符合预期规则的L2表项。规则为该L2表项是否能被数据簇大小(Cluster_size)整除,如果不能被整除,说明该L2表项已经损坏。磁盘坏块检查模块将得到损坏的L2表项及其在L1的索引;(1) Disk bad block check module: used to check the L2 table (L2Table) of the damaged disk. It traverses and checks the L2 table items (the contents of the L2 table) according to the L1->L2 addressing rules corresponding to the qcow2 virtual disk, and finds the L2 table items that do not meet the expected rules. The rule is whether the L2 table item can be divided by the data cluster size (Cluster_size). If it cannot be divided, it means that the L2 table item is damaged. The disk bad block check module will obtain the damaged L2 table item and its index in L1;
(2)虚拟磁盘备份模块:用于把虚拟磁盘备份到备份存储池中,以在发生环境异常时进行数据恢复。通常在虚拟化平台中,可以提供定时备份和立即备份两种备份策略。在备份虚拟磁盘时,不管qcow2的元数据还是真实数据,都会按照备份策略,把虚拟磁盘备份到备份存储池中。如果备份磁盘中存在磁盘坏块检查模块找出的L1索引及L2表项,则可以使用备份磁盘中的L2表项恢复损坏的qcow2元数据。在恢复元数据时,并不恢复L2表项对应的真实数据,以避免之前的数据覆盖当前数据(qcow2磁盘元数据一旦建立,其对应关系不会改变)。(2) Virtual disk backup module: used to back up the virtual disk to the backup storage pool to recover data when an environmental anomaly occurs. Usually, in a virtualization platform, two backup strategies can be provided: scheduled backup and immediate backup. When backing up the virtual disk, regardless of the qcow2 metadata or the real data, the virtual disk will be backed up to the backup storage pool according to the backup strategy. If the backup disk contains the L1 index and L2 table entries found by the disk bad block check module, the L2 table entries in the backup disk can be used to restore the damaged qcow2 metadata. When restoring the metadata, the real data corresponding to the L2 table entry is not restored to avoid previous data overwriting the current data (once the qcow2 disk metadata is established, the corresponding relationship will not change).
(3)磁盘偏移反推模块:用于探索所有可能的用户数据簇(Data Cluster)。它不依赖qcow2虚拟磁盘的L1->L2寻址规则,而是按照每次偏移数据簇大小,并根据qcow2元数据特征(首先跳过三个大小均为数据簇大小的表头,其次跳过内容为0X0001000100010001的字段(很可能为引用计数表),最后跳过内容为0X8000*0000[*代表通配符,即跳过开头为8000,0000为后缀的字段]),逐个筛选出可能为用户数据簇的数据块的地址(可能的L2表项);进而得到所有可能的L2表项,并利用其构造反推二级地址表(Maybe_L2_Tables);(3) Disk offset reverse inference module: used to explore all possible user data clusters (Data Cluster). It does not rely on the L1->L2 addressing rules of the qcow2 virtual disk, but according to the size of each data cluster offset and the qcow2 metadata characteristics (first skip three table headers of the same size as the data cluster, then skip the field with the content of 0X0001000100010001 (most likely the reference count table), and finally skip the content of 0X8000*0000 [* represents a wildcard, that is, skip the field with the beginning of 8000 and the suffix of 0000]), and screen out the addresses of data blocks that may be user data clusters one by one (possible L2 table entries); then obtain all possible L2 table entries, and use them to construct the reverse secondary address table (Maybe_L2_Tables);
(4)坏块精准修复模块:用于修复qcow2磁盘元数据。它以磁盘坏块检查模块输出的L1索引为输入,优先从备份磁盘中查找是否存在对应的L2表项,如果存在,则把L2表项写入当前损坏磁盘的L1索引所对应的L2表项;如果备份磁盘未找到对应的L2表项,则把磁盘偏移反推模块得到的反推二级地址表,写入当前损坏磁盘的L1索引所对应的L2表项。(4) Bad block precision repair module: used to repair qcow2 disk metadata. It takes the L1 index output by the disk bad block check module as input, and first searches the backup disk for the corresponding L2 table entry. If it exists, the L2 table entry is written into the L2 table entry corresponding to the L1 index of the current damaged disk. If the backup disk does not find the corresponding L2 table entry, the inverse secondary address table obtained by the disk offset inverse module is written into the L2 table entry corresponding to the L1 index of the current damaged disk.
(5)修复调整模块:用于调整从反推二级地址表写入的L2表项在L2表中的偏移值(Offset)。当通过坏块精准修复模块修复虚拟磁盘后,还是发现某些数据不能恢复时,可通过该模块调整从反推二级地址表写入的L2表项在L2表中的偏移值,以尽可能修复所有数据块。(5) Repair adjustment module: used to adjust the offset value (Offset) of the L2 table entry written from the reverse secondary address table in the L2 table. When some data is still found to be unrecoverable after the virtual disk is repaired by the bad block precise repair module, the module can be used to adjust the offset value of the L2 table entry written from the reverse secondary address table in the L2 table to repair all data blocks as much as possible.
下面对本申请实施例提供的元数据修复装置、电子设备及计算机可读存储介质进行 介绍,下文描述的元数据修复装置、电子设备及计算机可读存储介质与上文描述的数据修复方法可相互对应参照。The metadata repair device, electronic device, and computer-readable storage medium provided in the embodiments of the present application are described below. Introduction: The metadata repair device, electronic device, and computer-readable storage medium described below can correspond to the data repair method described above.
请参考图6,图6为本申请实施例所提供的另一种元数据修复装置的结构框图,该装置可以包括:Please refer to FIG. 6 , which is a structural block diagram of another metadata repair device provided in an embodiment of the present application. The device may include:
备份表项查询模块601,用于在确定虚拟磁盘的二级地址表中存在损坏表项时,在虚拟磁盘对应的备份磁盘中查询与损坏表项对应的备份表项;二级地址表的各个表项用于记录虚拟磁盘中各个用户数据的地址;The backup entry query module 601 is used to query the backup entry corresponding to the damaged entry in the backup disk corresponding to the virtual disk when it is determined that there is a damaged entry in the secondary address table of the virtual disk; each entry in the secondary address table is used to record the address of each user data in the virtual disk;
第一修复模块602,用于当查找到备份表项时,利用备份表项对损坏表项进行修复;A first repair module 602, configured to repair a damaged entry using the backup entry when a backup entry is found;
第二修复模块603,用于当未查找到备份表项时,通过扫描虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用反推二级地址表对损坏表项进行修复。The second repair module 603 is used to construct a reverse secondary address table by scanning the addresses of all user data in the virtual disk when no backup entry is found, and to repair the damaged entry using the reverse secondary address table.
可选地,该装置还可以包括:Optionally, the device may further include:
设置模块,用于在确定虚拟磁盘的二级地址表中存在损坏表项之前,从虚拟磁盘中获取二级地址表,并将二级地址表中的每个表项依次设置为待检测表项;A setting module, used for obtaining the secondary address table from the virtual disk before determining that there is a damaged table entry in the secondary address table of the virtual disk, and setting each table entry in the secondary address table as a table entry to be detected in sequence;
损坏表项判断模块,用于判断待检测表项的数据内容能否被预设数据簇大小整除;若否,则确定待检测表项为损坏表项。The damaged entry judging module is used to judge whether the data content of the entry to be checked can be divided by the preset data cluster size; if not, the entry to be checked is determined to be a damaged entry.
可选地,设置模块,可以包括:Optionally, the setting module may include:
查找子模块,用于从虚拟磁盘中获取一级地址表,并根据一级地址表的各个表项在虚拟磁盘中查找对应的二级地址表;一级地址表的表项用于记录各二级地址表在虚拟磁盘中的起始地址。The search submodule is used to obtain the primary address table from the virtual disk and search the corresponding secondary address table in the virtual disk according to each entry of the primary address table; the entries of the primary address table are used to record the starting address of each secondary address table in the virtual disk.
可选地,该装置还可以包括:Optionally, the device may further include:
记录模块,用于在确定待检测表项为损坏表项之后,记录损坏表项在一级地址表中对应的索引;A recording module, used for recording the index corresponding to the damaged table entry in the primary address table after determining that the table entry to be detected is a damaged table entry;
相应的,备份表项查询模块601,可以包括:Accordingly, the backup entry query module 601 may include:
备份表项查询子模块,用于从备份磁盘中获取备份一级地址表,并利用索引及备份一级地址表在备份磁盘中查找备份表项。The backup table entry query submodule is used to obtain the backup first-level address table from the backup disk, and use the index and the backup first-level address table to search for the backup table entry in the backup disk.
可选地,该装置还可以包括:Optionally, the device may further include:
备份模块,用于定期利用虚拟磁盘中的数据对备份磁盘中的数据进行备份更新。The backup module is used to regularly back up and update the data in the backup disk using the data in the virtual disk.
可选地,第二修复模块603,包括:Optionally, the second repair module 603 includes:
扫描设置子模块,用于跳过虚拟磁盘头部预设大小的磁盘数据,并在剩余的磁盘数据中,依照从前到后的顺序将指定大小的数据依次设置为待验证数据; The scanning setting submodule is used to skip the disk data of a preset size in the virtual disk header, and set the data of a specified size in the remaining disk data in order from front to back as the data to be verified;
反推二级地址表构建子模块,用于利用预设特征对待验证数据进行匹配验证,并在确定待验证数据通过匹配验证时,将待验证数据在虚拟磁盘中的目标地址写入反推二级地址表,以将目标地址作为反推二级地址表的表项。The reverse secondary address table construction submodule is used to match and verify the data to be verified using preset features, and when it is determined that the data to be verified passes the matching verification, the target address of the data to be verified in the virtual disk is written into the reverse secondary address table, so as to use the target address as a table entry of the reverse secondary address table.
可选地,反推二级地址表构建子模块,可以包括:Optionally, the reverse secondary address table construction submodule may include:
验证单元,用于在确定待验证数据的首部和尾部并不同时包含一级地址表的关键字,以及待验证数据与引用计数表的关键字不同时,确定待验证数据通过匹配验证。The verification unit is used to determine that the data to be verified passes the matching verification when it is determined that the header and the tail of the data to be verified do not contain the keyword of the primary address table at the same time, and the keywords of the data to be verified and the reference count table are different.
可选地,第二修复模块603,包括:Optionally, the second repair module 603 includes:
第一查找子模块,用于查找损坏表项在二级地址表中对应的前一个正常表项;A first search submodule is used to search for a previous normal table entry corresponding to the damaged table entry in the secondary address table;
第一修复子模块,用于确定正常表项在反推二级地址表中对应的目标表项,并利用目标表项的后一表项对损坏表项进行修复。The first repair submodule is used to determine the target table entry corresponding to the normal table entry in the reverse secondary address table, and repair the damaged table entry using the table entry following the target table entry.
可选地,利用反推二级地址表对损坏表项进行修复,包括:Optionally, the damaged entry is repaired by using the reverse secondary address table, including:
第二查找子模块,用于在二级地址表中确定由连续的已损坏表项构成的损坏表项组,并查找损坏表项组对应的前一个正常表项;A second search submodule is used to determine a damaged table entry group consisting of consecutive damaged table entries in the secondary address table, and to search for a previous normal table entry corresponding to the damaged table entry group;
第二修复子模块,用于确定正常表项在反推二级地址表中对应的目标表项,并利用目标表项之后指定数量的表项对损坏表项组进行修复;指定数量为损坏表项组所包含的损坏表项数量。The second repair submodule is used to determine the target entry corresponding to the normal entry in the reverse secondary address table, and use a specified number of entries after the target entry to repair the damaged entry group; the specified number is the number of damaged entries contained in the damaged entry group.
可选地,该装置还可以包括:Optionally, the device may further include:
标记模块,用于在利用反推二级地址表对损坏表项进行修复之后,将已修复的损坏表项标记为已修复表项,并确定已修复表项在二级地址表中的目标索引值;A marking module, used for marking the repaired damaged table entry as a repaired table entry after repairing the damaged table entry by using the reverse secondary address table, and determining a target index value of the repaired table entry in the secondary address table;
第一恢复验证模块,用于重新启动虚拟磁盘所属的虚拟机,并在虚拟机中确定已修复表项对应的用户数据是否恢复;A first recovery verification module is used to restart the virtual machine to which the virtual disk belongs, and determine in the virtual machine whether the user data corresponding to the repaired table entry is restored;
偏移值增加模块,用于若否,则在预设偏移区间内为目标索引值增加偏移值,以移动已修复表项在二级地址表中所处的位置;偏移值为预设数据簇大小的整数倍。The offset value increasing module is used to increase the offset value for the target index value within the preset offset interval to move the position of the repaired table entry in the secondary address table if not; the offset value is an integer multiple of the preset data cluster size.
可选地,该装置还可以包括:Optionally, the device may further include:
表项确定模块,用于在预设偏移区间内为目标索引值增加偏移值之前,确定由连续的已修复表项构成的已修复表项组,并确定与已修复表项组在二级地址表中对应的前一正常表项和后一正常表项;An entry determination module, used to determine a repaired entry group consisting of consecutive repaired entries before adding an offset value to a target index value within a preset offset interval, and determine a previous normal entry and a next normal entry corresponding to the repaired entry group in the secondary address table;
索引值确定模块,用于在二级地址表中确定前一正常表项对应的第一索引值和后一正常表项对应的第二索引值;An index value determination module, used to determine a first index value corresponding to a previous normal table entry and a second index value corresponding to a next normal table entry in the secondary address table;
预设偏移区间设置模块,用于利用第一索引值、第二索引值及预设数据簇大小确定 与已修复表项组对应的预设偏移区间;A preset offset interval setting module is used to determine the offset interval using the first index value, the second index value and the preset data cluster size. A preset offset interval corresponding to the repaired entry group;
相应的,偏移值增加模块,具体用于:Accordingly, the offset value increasing module is specifically used for:
在已修复表项所属的已修复表项组对应的预设偏移区间内为目标索引值增加偏移值。An offset value is added to the target index value within a preset offset interval corresponding to the repaired entry group to which the repaired entry belongs.
可选地,该装置还可以额包括:Optionally, the device may further include:
第二恢复验证模块,用于在预设偏移区间内为目标索引值增加偏移值之后,重新启动虚拟磁盘所属的虚拟机,并在虚拟机中确定增加了偏移值的已修复表项对应的用户数据是否恢复;若否,则在预设偏移区间内调整偏移值,为已修复表项的原始表项内容增加调整后的偏移值,并进入重新启动虚拟磁盘所属的虚拟机的步骤。The second recovery verification module is used to restart the virtual machine to which the virtual disk belongs after adding an offset value to the target index value within a preset offset interval, and determine in the virtual machine whether the user data corresponding to the repaired table entry with the added offset value is restored; if not, adjust the offset value within the preset offset interval, add the adjusted offset value to the original table entry content of the repaired table entry, and enter the step of restarting the virtual machine to which the virtual disk belongs.
请参考图7,图7为本申请实施例所提供的一种电子设备的结构框图,本申请实施例还提供一种电子设备,包括:Please refer to FIG. 7 , which is a structural block diagram of an electronic device provided in an embodiment of the present application. The embodiment of the present application further provides an electronic device, including:
存储器701,用于存储计算机可读指令;Memory 701, used to store computer-readable instructions;
一个或多个处理器702,用于执行计算机可读指令时实现如上述的元数据修复方法的步骤。One or more processors 702 are configured to implement the steps of the metadata repair method described above when executing computer-readable instructions.
由于电子设备部分的实施例与元数据修复方法部分的实施例相互对应,因此电子设备部分的实施例请参见元数据修复方法部分的实施例的描述,这里不再赘述。Since the embodiments of the electronic device part and the embodiments of the metadata repair method part correspond to each other, the embodiments of the electronic device part refer to the description of the embodiments of the metadata repair method part, which will not be repeated here.
请参考图8,图8为本申请实施例所提供的一个或多个存储有计算机可读指令的非易失性计算机可读存储介质的结构框图,本申请实施例还提供一个或多个存储有计算机可读指令的非易失性计算机可读存储介质810,一个或多个存储有计算机可读指令的非易失性计算机可读存储介质810上存储有计算机可读指令811,计算机可读指令811被处理器执行时实现上述任意实施例的元数据修复方法的步骤。Please refer to Figure 8, which is a structural block diagram of one or more non-volatile computer-readable storage media storing computer-readable instructions provided in an embodiment of the present application. The embodiment of the present application also provides one or more non-volatile computer-readable storage media 810 storing computer-readable instructions, and one or more non-volatile computer-readable storage media 810 storing computer-readable instructions store computer-readable instructions 811. When the computer-readable instructions 811 are executed by the processor, the steps of the metadata repair method of any of the above embodiments are implemented.
由于计算机可读存储介质部分的实施例与元数据修复方法部分的实施例相互对应,因此存储介质部分的实施例请参见元数据修复方法部分的实施例的描述,这里不再赘述。Since the embodiments of the computer-readable storage medium part correspond to the embodiments of the metadata repair method part, please refer to the description of the embodiments of the metadata repair method part for the embodiments of the storage medium part, which will not be repeated here.
说明书中各个实施例采用递进的方式描述,每个实施例重点说明的都是与其他实施例的不同之处,各个实施例之间相同相似部分互相参见即可。对于实施例公开的装置而言,由于其与实施例公开的方法相对应,所以描述的比较简单,相关之处参见方法部分说明即可。The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
专业人员还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件 和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。Professionals may further appreciate that the units and algorithm steps of each example described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to avoid the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块,或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
以上对本申请所提供的一种元数据修复方法、装置、电子设备及计算机可读存储介质进行了详细介绍。本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想。应当指出,对于本技术领域的普通技术人员来说,在不脱离本申请原理的前提下,还可以对本申请进行若干改进和修饰,这些改进和修饰也落入本申请权利要求的保护范围内。 The above is a detailed introduction to a metadata repair method, device, electronic device and computer-readable storage medium provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims (20)

  1. 一种元数据修复方法,其特征在于,包括:A metadata repair method, characterized by comprising:
    在确定虚拟磁盘的二级地址表中存在损坏表项时,在所述虚拟磁盘对应的备份磁盘中查询与所述损坏表项对应的备份表项;所述二级地址表的各个表项用于记录所述虚拟磁盘中各个用户数据的地址;When it is determined that there is a damaged entry in the secondary address table of the virtual disk, querying a backup entry corresponding to the damaged entry in a backup disk corresponding to the virtual disk; each entry in the secondary address table is used to record the address of each user data in the virtual disk;
    当查找到所述备份表项时,利用所述备份表项对所述损坏表项进行修复;和When the backup entry is found, repairing the damaged entry using the backup entry; and
    当未查找到所述备份表项时,通过扫描所述虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用所述反推二级地址表对所述损坏表项进行修复。When the backup entry is not found, a reverse secondary address table is constructed by scanning the addresses of all user data in the virtual disk, and the damaged entry is repaired using the reverse secondary address table.
  2. 根据权利要求1所述的元数据修复方法,其特征在于,在确定虚拟磁盘的二级地址表中存在损坏表项之前,还包括:The metadata repair method according to claim 1 is characterized in that, before determining that there is a damaged entry in the secondary address table of the virtual disk, it also includes:
    从所述虚拟磁盘中获取所述二级地址表,并将所述二级地址表中的每个表项依次设置为待检测表项;Acquire the secondary address table from the virtual disk, and set each entry in the secondary address table as an entry to be detected in sequence;
    判断任一待检测表项的数据内容能否被预设数据簇大小整除;和Determine whether the data content of any to-be-detected entry is divisible by a preset data cluster size; and
    响应于所述任一待检测表项的数据内容不能被预设数据簇大小整除,确定所述任一待检测表项为损坏表项。In response to the data content of any of the to-be-detected entries not being divisible by a preset data cluster size, it is determined that the any of the to-be-detected entries is a damaged entry.
  3. 根据权利要求2所述的元数据修复方法,其特征在于,所述从所述虚拟磁盘中获取所述二级地址表,包括:The metadata repair method according to claim 2, characterized in that the obtaining the secondary address table from the virtual disk comprises:
    从所述虚拟磁盘中获取一级地址表,并根据所述一级地址表的各个表项在所述虚拟磁盘中查找对应的二级地址表;所述一级地址表的表项用于记录各所述二级地址表在所述虚拟磁盘中的起始地址。A primary address table is obtained from the virtual disk, and a corresponding secondary address table is searched in the virtual disk according to each entry of the primary address table; the entries of the primary address table are used to record the starting address of each secondary address table in the virtual disk.
  4. 根据权利要求3所述的元数据修复方法,其特征在于,在确定所述待检测表项为损坏表项之后,还包括:The metadata repair method according to claim 3 is characterized in that after determining that the table entry to be detected is a damaged table entry, it also includes:
    记录所述损坏表项在所述一级地址表中对应的索引;Record the index corresponding to the damaged entry in the primary address table;
    相应的,所述在所述虚拟磁盘对应的备份磁盘中查询与所述损坏表项对应的备份表项,包括:Correspondingly, the step of searching the backup disk corresponding to the virtual disk for a backup entry corresponding to the damaged entry includes:
    从所述备份磁盘中获取备份一级地址表,并利用所述索引及所述备份一级地址表在所述备份磁盘中查找所述备份表项。A backup first-level address table is obtained from the backup disk, and the backup table entry is searched for in the backup disk using the index and the backup first-level address table.
  5. 根据权利要求2所述的元数据修复方法,其特征在于,所述将所述二级地址表中的每个表项依次设置为待检测表项之后,还包括:The metadata repair method according to claim 2 is characterized in that after setting each entry in the secondary address table as an entry to be detected in sequence, it also includes:
    响应于任一待检测表项的数据内容能被预设数据簇大小整除,判断所述任一待检测表项的下一项待检测表项的数据内容能否被预设数据簇大小整除;和 In response to the data content of any to-be-detected entry being divisible by a preset data cluster size, determining whether the data content of a next to-be-detected entry of the to-be-detected entry is divisible by the preset data cluster size; and
    响应于所述下一待检测表项的数据内容不能被预设数据簇大小整除,确定所述下一待检测表项为损坏表项。In response to the data content of the next entry to be detected not being divisible by the preset data cluster size, it is determined that the next entry to be detected is a damaged entry.
  6. 根据权利要求1所述的元数据修复方法,其特征在于,在确定虚拟磁盘的二级地址表中存在损坏表项之前,还包括:The metadata repair method according to claim 1 is characterized in that, before determining that there is a damaged entry in the secondary address table of the virtual disk, it also includes:
    从所述虚拟磁盘中获取所述二级地址表,并将所述二级地址表中的每个表项依次设置为待检测表项;Acquire the secondary address table from the virtual disk, and set each entry in the secondary address table as an entry to be detected in sequence;
    判断利用所述待检测表项是否查找到对应的用户数据;和Determine whether corresponding user data is found using the table item to be detected; and
    响应于利用所述待检测表项不能查找到对应的用户数据,确定所述待检测表项为损坏表项。In response to the failure to find the corresponding user data using the entry to be detected, it is determined that the entry to be detected is a damaged entry.
  7. 根据权利要求1所述的元数据修复方法,其特征在于,所述利用所述备份表项对所述损坏表项进行修复,包括:The metadata repair method according to claim 1, characterized in that the repairing of the damaged entry by using the backup entry comprises:
    将所述备份表项重新写回至所述损坏表项在所述虚拟磁盘的二级地址表中的对应位置,以完成对所述损坏表项的修复。The backup entry is rewritten back to the corresponding position of the damaged entry in the secondary address table of the virtual disk to complete the repair of the damaged entry.
  8. 根据权利要求1所述的元数据修复方法,其特征在于,还包括:The metadata repair method according to claim 1, further comprising:
    定期利用所述虚拟磁盘中的数据对所述备份磁盘中的数据进行备份更新。The data in the backup disk is backed up and updated periodically using the data in the virtual disk.
  9. 根据权利要求1所述的元数据修复方法,其特征在于,所述通过扫描所述虚拟磁盘中所有用户数据的地址构建反推二级地址表,包括:The metadata repair method according to claim 1, characterized in that the step of constructing a reverse secondary address table by scanning the addresses of all user data in the virtual disk comprises:
    跳过所述虚拟磁盘头部预设大小的磁盘数据,并在剩余的磁盘数据中,依照从前到后的顺序将指定大小的数据依次设置为待验证数据;和Skipping disk data of a preset size in the virtual disk header, and sequentially setting data of a specified size as data to be verified in the remaining disk data in a sequence from front to back; and
    利用预设特征对所述待验证数据进行匹配验证,并在确定所述待验证数据通过所述匹配验证时,将所述待验证数据在所述虚拟磁盘中的目标地址写入所述反推二级地址表,以将所述目标地址作为所述反推二级地址表的表项。The data to be verified is matched and verified using preset features, and when it is determined that the data to be verified passes the matching verification, the target address of the data to be verified in the virtual disk is written into the reverse secondary address table to use the target address as an entry in the reverse secondary address table.
  10. 根据权利要求9所述的元数据修复方法,其特征在于,所述指定大小为所述二级地址表的表项指向的用户数据的大小。The metadata repair method according to claim 9 is characterized in that the specified size is the size of the user data pointed to by the table entry of the secondary address table.
  11. 根据权利要求9所述的元数据修复方法,其特征在于,所述利用预设特征对所述待验证数据进行匹配验证,包括:The metadata repair method according to claim 9, characterized in that the matching verification of the data to be verified by using preset features comprises:
    在确定所述待验证数据的首部和尾部并不同时包含一级地址表的关键字,以及所述待验证数据与引用计数表的关键字不同时,确定所述待验证数据通过所述匹配验证。When it is determined that the header and the tail of the data to be verified do not simultaneously contain the keyword of the primary address table, and the keyword of the data to be verified is different from the keyword of the reference count table, it is determined that the data to be verified passes the matching verification.
  12. 根据权利要求9所述的元数据修复方法,其特征在于,所述利用所述反推二级地址表对所述损坏表项进行修复,包括:The metadata repair method according to claim 9, characterized in that the repairing of the damaged entry by using the reverse secondary address table comprises:
    查找所述损坏表项在所述二级地址表中对应的前一个正常表项;和 Find the previous normal entry corresponding to the damaged entry in the secondary address table; and
    确定所述正常表项在所述反推二级地址表中对应的目标表项,并利用所述目标表项的后一表项对所述损坏表项进行修复。A target entry corresponding to the normal entry in the reverse secondary address table is determined, and the damaged entry is repaired using a subsequent entry of the target entry.
  13. 根据权利要求12所述的元数据修复方法,其特征在于,所述正常表项的表项内容小于所述损坏表项对应的正确表项内容;The metadata repair method according to claim 12, characterized in that the table content of the normal table entry is smaller than the correct table content corresponding to the damaged table entry;
    所述目标表项的表项内容与所述正常表项的表项内容相同;The entry content of the target entry is the same as the entry content of the normal entry;
    所述后一表项的表项内容大于所述目标表项的表项内容。The entry content of the latter entry is greater than the entry content of the target entry.
  14. 根据权利要求9所述的元数据修复方法,其特征在于,所述利用所述反推二级地址表对所述损坏表项进行修复,包括:The metadata repair method according to claim 9, characterized in that the repairing of the damaged entry by using the reverse secondary address table comprises:
    在所述二级地址表中确定由连续的已损坏表项构成的损坏表项组,并查找所述损坏表项组对应的前一个正常表项;和Determining a damaged entry group consisting of consecutive damaged entries in the secondary address table, and searching for a previous normal entry corresponding to the damaged entry group; and
    确定所述正常表项在所述反推二级地址表中对应的目标表项,并利用所述目标表项之后指定数量的表项对所述损坏表项组进行修复;所述指定数量为所述损坏表项组所包含的损坏表项数量。Determine the target entry corresponding to the normal entry in the reverse secondary address table, and use a specified number of entries after the target entry to repair the damaged entry group; the specified number is the number of damaged entries included in the damaged entry group.
  15. 根据权利要求1至14任一项所述的元数据修复方法,其特征在于,在利用所述反推二级地址表对所述损坏表项进行修复之后,还包括:The metadata repair method according to any one of claims 1 to 14, characterized in that after repairing the damaged entry using the reverse secondary address table, it also includes:
    将已修复的损坏表项标记为已修复表项,并确定所述已修复表项在所述二级地址表中的目标索引值;Marking the repaired damaged entry as a repaired entry, and determining a target index value of the repaired entry in the secondary address table;
    重新启动所述虚拟磁盘所属的虚拟机,并在所述虚拟机中确定所述已修复表项对应的用户数据是否恢复;和Restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry is restored; and
    响应于在所述虚拟机中确定所述已修复表项对应的用户数据未恢复,在预设偏移区间内为所述目标索引值增加偏移值,以移动所述已修复表项在所述二级地址表中所处的位置;所述偏移值为预设数据簇大小的整数倍。In response to determining in the virtual machine that the user data corresponding to the repaired table entry has not been restored, an offset value is added to the target index value within a preset offset interval to move the position of the repaired table entry in the secondary address table; the offset value is an integer multiple of a preset data cluster size.
  16. 根据权利要求15所述的元数据修复方法,其特征在于,在预设偏移区间内为所述目标索引值增加偏移值之前,还包括:The metadata repair method according to claim 15, characterized in that before adding the offset value to the target index value within the preset offset interval, it also includes:
    确定由连续的已修复表项构成的已修复表项组,并确定与所述已修复表项组在所述二级地址表中对应的前一正常表项和后一正常表项;Determine a repaired table entry group consisting of continuous repaired table entries, and determine a previous normal table entry and a next normal table entry corresponding to the repaired table entry group in the secondary address table;
    在所述二级地址表中确定所述前一正常表项对应的第一索引值和所述后一正常表项对应的第二索引值;和Determining, in the secondary address table, a first index value corresponding to the previous normal entry and a second index value corresponding to the next normal entry; and
    利用所述第一索引值、所述第二索引值及所述预设数据簇大小确定与所述已修复表项组对应的预设偏移区间;Determine a preset offset interval corresponding to the repaired table entry group by using the first index value, the second index value and the preset data cluster size;
    相应的,所述在预设偏移区间内为所述目标索引值增加偏移值,包括: Correspondingly, the step of adding an offset value to the target index value within a preset offset interval includes:
    在所述已修复表项所属的已修复表项组对应的预设偏移区间内为所述目标索引值增加偏移值。An offset value is added to the target index value within a preset offset interval corresponding to the repaired entry group to which the repaired entry belongs.
  17. 根据权利要求15所述的元数据修复方法,其特征在于,在预设偏移区间内为所述目标索引值增加偏移值之后,还包括:The metadata repair method according to claim 15, characterized in that after adding the offset value to the target index value within the preset offset interval, it also includes:
    重新启动所述虚拟磁盘所属的虚拟机,并在所述虚拟机中确定增加了所述偏移值的已修复表项对应的用户数据是否恢复;和Restarting the virtual machine to which the virtual disk belongs, and determining in the virtual machine whether the user data corresponding to the repaired entry to which the offset value is added is restored; and
    响应于在所述虚拟机中确定增加了所述偏移值的已修复表项对应的用户数据未恢复,在所述预设偏移区间内调整所述偏移值,为所述已修复表项的目标索引值增加调整后的偏移值,并进入所述重新启动所述虚拟磁盘所属的虚拟机的步骤。In response to determining in the virtual machine that the user data corresponding to the repaired table entry to which the offset value has been added has not been restored, the offset value is adjusted within the preset offset interval, the adjusted offset value is added to the target index value of the repaired table entry, and the step of restarting the virtual machine to which the virtual disk belongs is entered.
  18. 一种元数据修复装置,其特征在于,包括:A metadata repair device, characterized by comprising:
    备份表项查询模块,用于在确定虚拟磁盘的二级地址表中存在损坏表项时,在所述虚拟磁盘对应的备份磁盘中查询与所述损坏表项对应的备份表项;所述二级地址表的各个表项用于记录所述虚拟磁盘中各个用户数据的地址;A backup entry query module, used for querying a backup entry corresponding to the damaged entry in a backup disk corresponding to the virtual disk when it is determined that there is a damaged entry in the secondary address table of the virtual disk; each entry in the secondary address table is used to record the address of each user data in the virtual disk;
    第一修复模块,用于当查找到所述备份表项时,利用所述备份表项对所述损坏表项进行修复;和A first repair module, configured to repair the damaged entry by using the backup entry when the backup entry is found; and
    第二修复模块,用于当未查找到所述备份表项时,通过扫描所述虚拟磁盘中所有用户数据的地址构建反推二级地址表,并利用所述反推二级地址表对所述损坏表项进行修复。The second repair module is used to construct a reverse secondary address table by scanning the addresses of all user data in the virtual disk when the backup table entry is not found, and repair the damaged table entry using the reverse secondary address table.
  19. 一种电子设备,其特征在于,包括:An electronic device, comprising:
    存储器,用于存储计算机可读指令;a memory for storing computer-readable instructions;
    一个或多个处理器,用于执行所述计算机可读指令时实现如权利要求1至17任一项所述的元数据修复方法。One or more processors are used to implement the metadata repair method as described in any one of claims 1 to 17 when executing the computer-readable instructions.
  20. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行如权利要求1至17任一项所述的元数据修复方法。 One or more non-volatile computer-readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the metadata repair method according to any one of claims 1 to 17.
PCT/CN2023/084812 2022-11-15 2023-03-29 Metadata restoration method and apparatus, and electronic device and storage medium WO2024103596A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211420802.9A CN115470049B (en) 2022-11-15 2022-11-15 Metadata repairing method and device, electronic equipment and storage medium
CN202211420802.9 2022-11-15

Publications (1)

Publication Number Publication Date
WO2024103596A1 true WO2024103596A1 (en) 2024-05-23

Family

ID=84338208

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/084812 WO2024103596A1 (en) 2022-11-15 2023-03-29 Metadata restoration method and apparatus, and electronic device and storage medium

Country Status (2)

Country Link
CN (1) CN115470049B (en)
WO (1) WO2024103596A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115470049B (en) * 2022-11-15 2023-02-28 浪潮电子信息产业股份有限公司 Metadata repairing method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704208A (en) * 2017-10-13 2018-02-16 郑州云海信息技术有限公司 A kind of restorative procedure of metadata, device and medium
CN112256464A (en) * 2020-10-20 2021-01-22 湖南国科微电子股份有限公司 Hard disk data error correction method and device, electronic equipment and storage medium
CN114840358A (en) * 2022-04-28 2022-08-02 济南浪潮数据技术有限公司 Data restoration method, device and medium based on virtual disk
CN115470049A (en) * 2022-11-15 2022-12-13 浪潮电子信息产业股份有限公司 Metadata repairing method and device, electronic equipment and storage medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101980195B (en) * 2010-10-26 2012-08-01 青岛海信移动通信技术股份有限公司 Mobile communication terminal-based database index repair method and device
US20170371551A1 (en) * 2016-06-23 2017-12-28 Linkedin Corporation Capturing snapshots of variable-length data sequentially stored and indexed to facilitate reverse reading
US10445195B2 (en) * 2017-08-07 2019-10-15 Micron Technology, Inc. Performing data restore operations in memory
CN110225351B (en) * 2019-06-20 2021-07-09 北京达佳互联信息技术有限公司 Video file processing method and device, electronic equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704208A (en) * 2017-10-13 2018-02-16 郑州云海信息技术有限公司 A kind of restorative procedure of metadata, device and medium
CN112256464A (en) * 2020-10-20 2021-01-22 湖南国科微电子股份有限公司 Hard disk data error correction method and device, electronic equipment and storage medium
CN114840358A (en) * 2022-04-28 2022-08-02 济南浪潮数据技术有限公司 Data restoration method, device and medium based on virtual disk
CN115470049A (en) * 2022-11-15 2022-12-13 浪潮电子信息产业股份有限公司 Metadata repairing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN115470049B (en) 2023-02-28
CN115470049A (en) 2022-12-13

Similar Documents

Publication Publication Date Title
US20220413706A1 (en) Data Storage Method, Apparatus and Storage System
WO2024103596A1 (en) Metadata restoration method and apparatus, and electronic device and storage medium
CN102541757B (en) Write cache method, cache synchronization method and device
US10061693B2 (en) Method of generating secondary index and apparatus for storing secondary index
US9471622B2 (en) SCM-conscious transactional key-value store
KR101549220B1 (en) Method and System for Managing Database, and Tree Structure for Database
WO2020119143A1 (en) Database deleted record recovery method and system
WO2007046324A1 (en) Estimator, table managing device, selecting device, table managing method, program for allowing computer to execute the table managing method, and recording medium where the program is recorded
JPH11134235A (en) Method for supporting recovery from fault of external storage device
CN115145697A (en) Database transaction processing method and device and electronic equipment
CN115705152A (en) Metadata management in non-volatile memory devices using in-memory logs
CN106909514B (en) Method and device for positioning snapshot disk address
CN116909939A (en) LSM tree-based key value separation storage engine garbage recycling method, system and equipment
WO2016122318A1 (en) A computer implemented method for generating a variant call file
US20100169572A1 (en) Data storage method, apparatus and system for interrupted write recovery
CN115705153A (en) Conditional update and deferred lookup
CN109325005A (en) A kind of data processing method and electronic equipment
CN111736778B (en) Data updating method, device and system and electronic equipment
US20050131855A1 (en) Data cleaning
CN110209530B (en) Method and system for recovering IO data of CDP system
CN113704277A (en) Database-based breakpoint continuous transmission method and related device
CN111048141B (en) Error processing method, device, equipment and computer readable storage medium
JP2639349B2 (en) Error correction line search device
KR20150044013A (en) Method and System for Managing Database, and Tree Structure for Database
CN106648989B (en) Repair method for western digital hard disk stalling fault

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23890041

Country of ref document: EP

Kind code of ref document: A1