CN110955557A

CN110955557A - Data verification processing method and device, storage medium and processor

Info

Publication number: CN110955557A
Application number: CN201811136916.4A
Authority: CN
Inventors: 彭海林; 佘海斌
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Cloud Computing Ltd
Priority date: 2018-09-27
Filing date: 2018-09-27
Publication date: 2020-04-03
Anticipated expiration: 2038-09-27
Also published as: CN110955557B

Abstract

The invention discloses a data verification processing method, a data verification processing device, a storage medium and a processor. Wherein, the method comprises the following steps: acquiring snapshot data; restoring and writing the snapshot data into a second disk prepared in advance; and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback. The method and the device solve the technical problem that hidden danger exists in the security guarantee of the data because the disk snapshot cannot be verified in the related technology, and improve the security and the reliability of the backed-up data.

Description

Data verification processing method and device, storage medium and processor

Technical Field

The present invention relates to the field of data backup, and in particular, to a data verification processing method, apparatus, storage medium, and processor.

Background

At present, more and more data are provided by using a disk snapshot, which can backup a real physical disk and various storage devices such as a disk or a block storage device on a virtual machine, so that a disk snapshot technology is widely applied.

Because more and more traditional small and medium-sized enterprises are migrated from the data center to the cloud computing platform, the corresponding data backup requirements are more and more in the cloud computing, and the data can be backed up in a disk snapshot mode. In a cloud computing platform, for a cloud disk carrying important services, a user may make a periodic data backup (i.e., create a disk snapshot) plan in order to restore data when a subsequent disaster occurs, thereby ensuring the availability of the services.

The inventor finds that, before the "data recovery after disaster" really happens, the current disk snapshot mechanism does not allow the user to see or confirm that his cloud disk can actually recover the business data by rolling back to a snapshot created before, and the cloud service provider does not provide a mechanism to verify the integrity and correctness of the data after the snapshot is rolled back, for example, how to prove that the data of the cloud disk after the rolling back is identical to the data at the moment of creating the snapshot after a cloud disk creates a data backup (i.e. snapshot) and the cloud disk is rolled back to the snapshot created before at a subsequent time?

Therefore, the inventor finds that disk snapshots cannot be verified in the related art, so that a potential risk exists in security guarantee of data.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the invention provides a data verification processing method, a data verification processing device, a storage medium and a processor, and at least solves the technical problem that hidden dangers exist in security guarantee of data due to the fact that disk snapshots cannot be verified in the related technology.

According to an aspect of an embodiment of the present invention, there is provided a data verification processing method, including: acquiring snapshot data, wherein the snapshot data is data obtained by taking a snapshot of a first disk and used for backing up data of the first disk; restoring and writing the snapshot data into a second disk prepared in advance; and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback.

Further, when the snapshot data is data obtained by snapshot of a snapshot device based on a log structure, restoring and writing the snapshot data to the second disk includes: writing the snapshot data into the second disk; replaying all input/output (IO) operations occurring on the first disk from the time of creating the snapshot data on the second disk, and applying all IO operations to the second disk, wherein the IO operations include: IO write operations.

Further, replaying all IO operations on the second disk and applying all IO operations to the second disk includes: replaying all IO operations on the second disk, and acquiring logs corresponding to all IO operations; applying the log to the second disk.

Further, the second disk is a temporary disk, and before the snapshot data is restored and written to the second disk, the method further includes: creating the temporary disk as the second disk; after obtaining the result of the comparison, the method further comprises: and deleting the temporary disk.

Further, in the case that the first disk supports the merge function compact, the method further includes: forbidding the merging function of the first disk in the processes of creating the snapshot data and executing the data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

Further, after comparing the data of the first disk corresponding to the snapshot data with the data restored to be written to the second disk, the method further includes: determining to restore the integrity and correctness of the data written to the second disk according to the result of the comparison; presenting the integrity and the correctness to a user.

According to another aspect of the embodiments of the present invention, there is also provided a data verification processing apparatus, including: the system comprises an acquisition module, a backup module and a storage module, wherein the acquisition module is used for acquiring snapshot data, and the snapshot data is obtained by taking a snapshot of a first disk and is used for backing up data of the first disk; the recovery module is used for recovering and writing the snapshot data into a second disk prepared in advance; and the comparison module is used for comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback.

Further, in a case that the snapshot data is data obtained by snapshot of a snapshot device based on a log structure, the recovery module includes: a writing module, configured to write the snapshot data into the second disk; a replay module, configured to replay, on the second disk, all the input/output IO operations that have occurred on the first disk from a time when the snapshot data is created, and apply all the IO operations to the second disk, where the IO operations include: IO write operations.

Further, the replay module is configured to replay the all IO operations on the second disk, obtain logs corresponding to the all IO operations, and apply the logs to the second disk.

Further, the second disk is a temporary disk, and the apparatus further includes: a creating module, configured to create the temporary disk as the second disk before the snapshot data is restored and written to the second disk; and the deleting module is used for deleting the temporary disk after the comparison result is obtained.

Further, in the case that the first disk supports the merge function compact, the apparatus further includes: a merge function control module, configured to prohibit a merge function of the first disk during the processes of creating the snapshot data and executing the data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

Further, the apparatus further comprises: and the display module is used for determining the integrity and the correctness of the data which is restored to the second disk according to the comparison result and displaying the integrity and the correctness to a user.

According to another aspect of the embodiments of the present invention, there is also provided a storage medium including a stored program, wherein the program performs the method of any one of the above when executed.

According to another aspect of the embodiments of the present invention, there is also provided a processor for executing a program, wherein the program performs the method of any one of the above when running.

In the embodiment of the invention, the following mode is adopted: acquiring snapshot data; restoring and writing the snapshot data into a second disk prepared in advance; and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback. The embodiment of the invention solves the technical problem that hidden danger exists in the security guarantee of data because the disk snapshot cannot be verified in the related technology, and improves the security and reliability of the backed-up data.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a block diagram of an alternative hardware configuration of a computing device for implementing a data validation processing method;

fig. 2 is a flowchart of a data verification processing method according to embodiment 1 of the present invention;

FIG. 3 is a schematic diagram of a cloud computing system architecture according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a disk snapshot according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of the composition of a log block device according to an embodiment of the invention;

FIG. 6 is a schematic diagram of a log block device and snapshot according to an embodiment of the present invention;

fig. 7 is a block diagram of the structure of a data verification processing apparatus according to an embodiment of the present invention;

FIG. 8 is a block diagram of an alternative configuration of a data verification processing apparatus according to an embodiment of the present invention;

FIG. 9 is a block diagram of an alternative configuration of a data verification processing apparatus according to an embodiment of the present invention;

FIG. 10 is a block diagram of an alternative configuration of a data verification processing apparatus according to an embodiment of the present invention;

FIG. 11 is a block diagram of an alternative configuration of a data verification processing apparatus according to an embodiment of the present invention;

FIG. 12 is a schematic diagram of a disk snapshot and rollback flow, according to an embodiment of the present invention; and the number of the first and second groups,

fig. 13 is a block diagram of an alternative computer terminal according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

a virtual machine disk: a virtual machine block device;

log structure block device: block devices constructed based on a log manner;

and (3) carrying out disk snapshot: snapshot of the virtual machine disk block device;

disk backup: copying the virtual machine disk to another cluster for long-term storage;

disk mirroring: the method comprises the steps of creating a snapshot of a virtual machine disk in a delayed loading mode;

and (3) delayed loading: when the virtual machine is started, the data of the disk is slowly loaded from a disk snapshot, and the virtual machine can run without completing the loading;

and (4) snapshot center: the storage system is responsible for storing and managing the disk snapshots of the virtual machine;

and (3) snapshot rollback: rolling back the disk of the virtual machine to a previous disk snapshot version;

data merging: a plurality of logs of the log block device are merged, so that the storage space is saved;

LBA: a Logical Block Address (Logical Block Address);

VM: a virtual machine;

device: a virtual machine block device;

EBS: english Elastic Block service, Block device storage cluster;

LSBD: an English Log Structured Block Device;

snapshot: carrying out disk snapshot;

lazyload: delaying loading;

backup: backing up a magnetic disk;

and (3) Rollback: rolling back the snapshot;

compact: merging data;

image: mirroring a disk;

block: disk blocks in the disk snapshot;

image Center: a mirror center;

distributed File System: a distributed file system;

HardLink: and the hard link technology is used for sharing data by multiple files.

Example 1

There is also provided, in accordance with an embodiment of the present invention, an embodiment of a data validation processing method, to note that the steps illustrated in the flowchart of the figure may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.

The data verification processing method can be operated on a real computing device and can also be operated on a virtual computing device. For a virtual computing device, which may be a virtual operating system running on a real computing device, it may logically occupy real physical computing resources that are logically exclusive to the virtual computing device. Whether real or virtual, ultimately rely on real physical computing resources. The resource may be one server or a plurality of distributed servers. With the development of technology, personal mobile terminals, computer terminals, etc. can also provide these real physical computing resources. The devices of the physical computing resources that these providers are may be collectively referred to as a computing apparatus or computing device.

Fig. 1 shows a hardware configuration block diagram of a computing device for implementing a data verification processing method. As shown in fig. 1, computing device 10 may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, computing device 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuitry may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in computing device 10. As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the data verification processing method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 104. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, which may be connected to computing device 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of such networks may include wireless networks provided by a communications provider of computing device 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of computing device 10. The display is optional equipment, and can be remotely connected with other equipment for displaying under the condition that the display is not equipped; in addition, some computing devices do not require display functionality, in which case the computing device need not be equipped with a display.

The hardware configurations of the devices that currently provide these computing resources are similar, but new architecture devices that appear as technology advances are not excluded, and these devices should be within the scope of the present embodiments.

The hardware block diagram shown in fig. 1 may be taken as an exemplary block diagram of not only the computing device 10 described above but also a server, and the computing device 10 may be taken as a server to execute the network service. The network server is a network-based user service, such as a social network, a cloud resource, an email, an online payment or other online applications, and the network service may use the method in this embodiment in a case where a snapshot backup of data is required.

Under the operating environment, the application provides a data verification processing method as shown in fig. 2. Fig. 2 is a flowchart of a data verification processing method according to a first embodiment of the present invention, as shown in fig. 2, the method includes the following steps:

step S202, snapshot data is obtained, wherein the snapshot data is data obtained by performing snapshot on a first disk and used for backing up data of the first disk;

step S204, restoring and writing the snapshot data into a second disk prepared in advance;

step S206, comparing the data of the first disk corresponding to the snapshot data with the data restored and written to the second disk, where the comparison result is used to verify the status of the snapshot data after rollback.

Through the steps, after the first disk is subjected to snapshot, the data of the snapshot can be restored on another disk, and then the condition after the rollback is carried out in the future can be determined according to the restored data. Compared with the related art, the embodiment has the advantages that verification steps are added, the comparison result is obtained through verification, and the comparison result can be used as the basis for verifying the rollback condition. Therefore, the technical problem that hidden danger exists in the security guarantee of the data because the disk snapshot cannot be verified in the related technology is solved through the embodiment, and the security and the reliability of the backed-up data are improved.

The method in fig. 2 may be applied in various environments, and a storage and computing separated cloud computing architecture to which the embodiment may be applied is described below with reference to the drawings.

Fig. 3 is a schematic diagram of a cloud computing system architecture according to an embodiment of the present invention, and as shown in fig. 3, in a cloud computing architecture with separated storage and computation, a computing cluster provides VM resources, a block device storage cluster (EBS) is responsible for providing storage resources of disk block devices, and a Snapshot Center (Snapshot Center) is responsible for storing disk snapshots. The disk snapshots are divided into disk blocks (blocks) of the same size, and when a VM cloud disk creates a snapshot, a snapshot service in the storage cluster uploads the disk block (block) data of the disk snapshots to a snapshot center in sequence. And when the VM cloud disk rolls back to a certain snapshot, downloading the disk block of the corresponding snapshot from the snapshot center to the storage cluster, and writing the disk block into the disk.

Fig. 4 is a schematic diagram of a disk snapshot according to an embodiment of the present invention, as shown in fig. 4, the snapshot is divided into blocks (blocks) with equal length, and each block is sequentially assigned with a sequence number (BlockIndex), so that during uploading of the disk snapshot, the snapshot center takes the block as a storage unit.

The present embodiment may also be applied to other architectures and other types of storage systems, which are not shown one by one again.

As an alternative embodiment, an appropriate time may be selected to execute the data verification processing method, for example, a software interface may be provided for a data administrator, the software interface is used to configure a period for generating snapshot data, an option may be provided in the software interface, and the option is used to configure whether to verify the snapshot data after the snapshot is generated, or to perform verification after the snapshot data is generated for a predetermined time. If the administrator configures the option, the execution may be performed according to the configuration of the option. If the administrator does not configure the option, the data validation processing method is not performed.

In the software interface, a display interface may also be provided, which may be used to show the results of the comparison. The result of the comparison may include various results, for example, the integrity and correctness of the data may be included, that is, the integrity and correctness of the data written to the second disk may be restored and then presented to the user according to the result of the comparison.

It may also be preferable to provide missing or incorrect data in the event that the data is incomplete or incorrect.

The processing of the comparison results may be used better than the presentation to the administrator, for example, it may be determined whether the previous snapshot needs to be discarded and a new snapshot may be generated based on the presented results. The function is configured by an administrator, and the administrator can configure various conditions (for example, integrity is lower than a threshold value and/or correctness is lower than a threshold value, etc.) that the snapshot needs to be regenerated, regenerate a new snapshot as long as the conditions are triggered, and after the new snapshot is generated, the verification can still be performed by using the method in the embodiment, and as long as the conditions configured by the administrator are met, the snapshot meeting the requirements is reserved. The processing mode can be automated, and a record is recorded each time a new snapshot is generated, and the record can be reserved for being checked by an administrator or pushed to the mobile equipment of the administrator in real time.

The data verification processing method can be applied to verification of various snapshots, and adjustment of different degrees can be selected according to different implementation situations. The method can be applied to a log block device as a preferred embodiment.

Fig. 5 is a schematic composition diagram of a log block device according to an embodiment of the present invention, and as shown in fig. 5, the log block device stores data using a flat log file stored in a distributed manner in the bottom layer, where the flat log file can only be additionally written and cannot be overwritten. The LSBD device is partitioned into multiple segments according to a logical address space to improve concurrent processing capability. The Index file is used for storing metadata of segment data modification, and the data file is used for storing data of segment data modification. Both the Index file and the data file are flat log files, and after one file is written, the seal is established, and then the next new file is created to be written, similar to LSM.

Fig. 6 is a schematic diagram of a log block device and snapshot according to an embodiment of the present invention, and as shown in fig. 6, the LSBDsnapshot and the LSBD device are also constructed based on an underlying flat log file. Multiple devices and snapshots may share the same log file through hardlink.

The present embodiment may be applied to the above-mentioned log block device, and of course, may also be applied to other log block devices, which are not illustrated herein.

Under the condition that snapshot data is data obtained by snapshot of a snapshot device based on a log structure, due to the nature of the log structure, the snapshot data can be written into the second disk, then all input/output (IO) operations occurring on the first disk from the moment of creating the snapshot data are replayed on the second disk, and all the IO operations are applied to the second disk, wherein the IO operations include: IO write operations. The embodiment can be used for better verifying the snapshot data of the log-structured equipment.

There are also many ways to apply IO operations to the second disk, and one way to achieve higher efficiency is: replaying all IO operations on the second disk, and acquiring logs corresponding to all IO operations; the log is applied to the second disk. In this way, all IO operations can be applied to the second disk more efficiently.

In order to save resources, the second disk used in the above steps may be a temporary disk, and at this time, before the snapshot data is restored and written onto the second disk, the temporary disk may be created as the second disk; after the result of the comparison is obtained, the temporary disk is deleted. By using the temporary disk, more resources are not needed to verify the snapshot data, so that hardware resources can be saved.

In some cases, in a case where the first disk may support the merge function compact, in order to make verification of the snapshot more accurate, the merge function of the first disk may be disabled during creating snapshot data and executing the data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

With the above alternative implementation, when the cloud disk creates a snapshot, the cloud disk compact is temporarily disabled to completely record all logs. After the snapshot is completed, a new disk is cloned by the snapshot, all logs from the moment of creating the snapshot are completely replayed (replayed) on the new disk, and then whether the data of the original cloud disk and the data of the new disk are completely consistent or not is compared, so that the integrity and the correctness of the data after the snapshot is rolled back can be verified. And after the verification is completed, deleting the temporarily created new disk, and then reopening the cloud disk compact.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for causing a computing device to execute the methods according to the embodiments of the present invention or a processor to execute the methods according to the embodiments of the present invention.

Example 2

In this embodiment, there is further provided an apparatus for implementing the method in embodiment 1, and fig. 7 is a block diagram of a data verification processing apparatus according to an embodiment of the present invention, and as shown in fig. 7, the apparatus includes:

an obtaining module 72, configured to obtain snapshot data, where the snapshot data is data obtained by taking a snapshot of the first disk and used for backing up data of the first disk;

a restoring module 74, configured to restore and write the snapshot data to a second disk prepared in advance;

and a comparing module 76, configured to compare the data of the first disk corresponding to the snapshot data with the data restored and written to the second disk, where a result of the comparison is used to verify a status of the snapshot data after rollback.

The technical problem that hidden dangers exist in the security guarantee of data due to the fact that disk snapshots cannot be verified in the related technology is solved through the modules, and the security and the reliability of the backed-up data are improved.

Fig. 8 is a block diagram of an optional structure of the data verification processing apparatus according to the first embodiment of the present invention, as shown in fig. 8, optionally, in a case that the snapshot data is data obtained by taking a snapshot of a snapshot device based on a log structure, the restoring module 76 may include: a writing module 762, configured to write the snapshot data into the second disk; a replay module 764, configured to replay, on the second disk, all the IO operations that occurred on the first disk since the time of creating the snapshot data, and apply all the IO operations to the second disk, where the IO operations include: IO write operations.

Optionally, the replay module 764 is configured to replay all IO operations on the second disk, obtain logs corresponding to all IO operations, and apply the logs to the second disk.

Fig. 9 is a block diagram of an alternative structure of a data verification processing apparatus according to an embodiment of the present invention, and as shown in fig. 9, in a case where the second disk is a temporary disk, the apparatus may further include: a creating module 92, configured to create a temporary disk as the second disk before the snapshot data is restored and written to the second disk; and a deleting module 94, configured to delete the temporary disk after obtaining the comparison result.

Fig. 10 is a block diagram of an alternative structure of a data verification processing apparatus according to an embodiment of the present invention, and as shown in fig. 10, in the case where the first disk supports the merge function compact, the apparatus further includes: a merge function control module 102, configured to disable a merge function of the first disk during the process of creating snapshot data and executing a data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

Fig. 11 is a block diagram of an alternative structure of a data verification processing apparatus according to an embodiment of the present invention, and as shown in fig. 11, the apparatus further includes: and the presentation module 112 is configured to determine, according to the comparison result, to restore the integrity and correctness of the data written to the second disk, and present the integrity and correctness to the user.

It should be noted here that the modules are the same as the example and the application scenario realized by the steps corresponding to embodiment 1, but the modules are not limited to the content disclosed in embodiment 1, and the obtained effects are also described in embodiment 1 in detail, and are not described again.

The modules may be understood as software modules, and in the actual development process, the modules may be integrated into different software or different services, and the following description is given by way of example, in which different services provide the functions of the modules.

Fig. 12 is a schematic diagram of a disk Snapshot and rollback flow according to an embodiment of the present invention, as shown in fig. 12, a Block master (bm) is responsible for managing LSBD metadata and an overall Snapshot and rollback control flow, a Block Server (BS) is responsible for segment service of device, and a Snapshot Server (SS) is responsible for segment upload, download, and rollback of Snapshot; client represents the IO Client (e.g., VM) of LSBD device, and Admin represents the operation initiator that creates snapshot and rollback verification. The LSBD snapshot creation (steps 1-8) and rollback verification (steps 9-17) flows are as follows:

step 1, Admin initiates a request of creating a snapshot to a Block Master.

Step 2, the Block Master requests to create segment snapshots from all Block servers where the disk is located; the Block Server first disables the compact of the disk.

And 3, continuously writing IO into the disk by the Client.

Step 4, the Block Server ensures that these transaction logs are preserved (not corrupted by compactions) while writing IOs to disk.

And 5, all Block servers successfully create segment snapshots, and the disk snapshots are successfully created.

And 6, the Block Master informs the Snapshot Server to backup and upload the disk Snapshot data.

And 7, reading the Snapshot data from the segment of the disk by the Snapshot Server.

And 8, uploading the data of the disk Snapshot to a Snapshot center by the Snapshot Server.

And 9, the Block Master requires the Block Server to create a temporary disk for snapshot rollback.

Step 10, the Block Server creates a temporary disk (Rollback Device) for snapshot Rollback.

And step 11, the Block Master informs the Snapshot Server to roll back the temporary disk to a specific Snapshot.

And step 12, the Snapshot Server downloads the data of the specific Snapshot from the Snapshot center.

And step 13, the Snapshot Server writes the Snapshot data into a temporary disk.

Step 14, the Block Master informs the Block Server to replay (replay) all write IO operations that occurred to the original disk from the moment the snapshot was created on the temporary disk.

And step 15, the Block Server reads out transaction logs corresponding to all IO write operations.

Step 16, the Block Server applies transaction logs to the temporary disk, and then compares whether the data of the temporary disk (Rollback Device) and the original disk are completely consistent. If so, it indicates that snapshot rollback can ensure data integrity and correctness. After the test verification is finished, the Block Server is responsible for deleting the temporary disk and opening the compact of the original disk again.

Step 17, Admin queries the Block Master for the verification result of the snapshot rollback and then displays the result to the user.

The embodiment utilizes the log additional writing function of the log block device, the write IO operation before playback (replay) is performed on the temporary disk after snapshot rollback, the verification of the snapshot rollback is ensured to be effective by comparing the final data consistency of the temporary disk and the original disk, a verification mechanism of the snapshot rollback is realized in the log block device system through the embodiment, the integrity and the correctness of the data after the snapshot rollback can be immediately verified when the snapshot creation is completed, and a more effective data backup scheme is provided for a user.

Example 3

The present embodiments may provide a computing device that may be any one of a group of computing devices. Optionally, in this embodiment, the computing device may also be replaced with a terminal device such as a mobile terminal. Optionally, in this embodiment, the computing device may be located in at least one network device of a plurality of network devices of a computer network.

The computing device may be used to execute the method in embodiment 1, or the computing device may include the modules or respective programs in embodiment 2.

The embodiment of the invention also provides a storage medium or a memory which stores the program, or a processor which can execute the program. The program may be used to execute the method in embodiment 1 or include the functions of the modules or respective programs in embodiment 2.

Optionally, in this embodiment, the storage medium or the memory or the processor may be located in any one of computing devices in a computing device group in a computer network, or in any one of mobile terminals in a mobile terminal group.

The storage medium can be used for storing software programs and modules, such as program instructions/modules corresponding to the methods and apparatuses in the above embodiments of the present invention, and the processor executes various functional applications and data processing by executing the software programs and modules stored in the memory. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory located remotely from the processor, and these remote memories may be connected to the terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Example 3

The embodiment of the invention can provide a computer terminal which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile device.

Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.

Fig. 13 is a block diagram of an alternative computer terminal according to an embodiment of the present invention. As shown in fig. 13, the computer terminal 130 may include: one or more processors (only one shown) and memory.

The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the data verification processing method and apparatus in the embodiments of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the data verification processing method. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory located remotely from the processor, which may be connected to the terminal 130 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring snapshot data, wherein the snapshot data is data obtained by taking a snapshot of a first disk and used for backing up data of the first disk; restoring and writing the snapshot data into a second disk prepared in advance; and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback.

Optionally, the processor may further execute the program code of the following steps: when the snapshot data is data obtained by snapshot of the snapshot device based on the log structure, the restoring and writing of the snapshot data to the second disk includes: writing the snapshot data into a second disk; replaying all Input and Output (IO) operations which occur on the first disk from the moment of creating the snapshot data on the second disk, and applying all IO operations to the second disk, wherein the IO operations comprise: IO write operations.

Optionally, the processor may further execute the program code of the following steps: replaying all IO operations on the second disk and applying all IO operations to the second disk includes: replaying all IO operations on the second disk, and acquiring logs corresponding to all IO operations; the log is applied to the second disk.

Optionally, the processor may further execute the program code of the following steps: the second disk is a temporary disk, and before the snapshot data is restored and written to the second disk, the method further includes: creating a temporary disk as a second disk; after obtaining the result of the comparison, the method further comprises: and deleting the temporary disk.

Optionally, the processor may further execute the program code of the following steps: in the case that the first disk supports the merge function compact, the method further comprises: forbidding the merging function of the first disk in the processes of creating snapshot data and executing the data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

Optionally, the processor may further execute the program code of the following steps: after comparing the data of the first disk corresponding to the snapshot data with the data restored to be written to the second disk, the method further includes: determining to restore the integrity and correctness of the data written to the second disk according to the comparison result; integrity and correctness are presented to the user.

The embodiment of the invention provides a scheme of a computer terminal for realizing a data verification processing method. By acquiring snapshot data; restoring and writing the snapshot data into a second disk prepared in advance; and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback. The embodiment of the invention solves the technical problem that hidden danger exists in the security guarantee of data because the disk snapshot cannot be verified in the related technology, and improves the security and reliability of the backed-up data.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Example 4

The embodiment of the invention also provides a storage medium. Optionally, in this embodiment, the storage medium may be configured to store a program code executed by the data verification processing method provided in the first embodiment.

Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring snapshot data, wherein the snapshot data is data obtained by taking a snapshot of a first disk and used for backing up data of the first disk; restoring and writing the snapshot data into a second disk prepared in advance; and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: when the snapshot data is data obtained by snapshot of the snapshot device based on the log structure, the restoring and writing of the snapshot data to the second disk includes: writing the snapshot data into a second disk; replaying all Input and Output (IO) operations which occur on the first disk from the moment of creating the snapshot data on the second disk, and applying all IO operations to the second disk, wherein the IO operations comprise: IO write operations.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: replaying all IO operations on the second disk and applying all IO operations to the second disk includes: replaying all IO operations on the second disk, and acquiring logs corresponding to all IO operations; the log is applied to the second disk.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the second disk is a temporary disk, and before the snapshot data is restored and written to the second disk, the method further includes: creating a temporary disk as a second disk; after obtaining the result of the comparison, the method further comprises: and deleting the temporary disk.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: in the case that the first disk supports the merge function compact, the method further comprises: forbidding the merging function of the first disk in the processes of creating snapshot data and executing the data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: after comparing the data of the first disk corresponding to the snapshot data with the data restored to be written to the second disk, the method further includes: determining to restore the integrity and correctness of the data written to the second disk according to the comparison result; integrity and correctness are presented to the user.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, a division of a unit is merely a division of a logic function, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A data verification processing method comprises the following steps:

acquiring snapshot data, wherein the snapshot data is data obtained by taking a snapshot of a first disk and used for backing up data of the first disk;

restoring and writing the snapshot data into a second disk prepared in advance;

and comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback.

2. The method of claim 1, wherein, in the case that the snapshot data is data obtained by snapshotting a snapshot device based on a log structure, the restoring the snapshot data to be written on the second disk comprises:

writing the snapshot data into the second disk;

replaying all input/output (IO) operations occurring on the first disk from the time of creating the snapshot data on the second disk, and applying all IO operations to the second disk, wherein the IO operations include: IO write operations.

3. The method of claim 2, wherein replaying and applying all IO operations on the second disk comprises:

replaying all IO operations on the second disk, and acquiring logs corresponding to all IO operations;

applying the log to the second disk.

4. The method of any of claims 1-3, wherein the second disk is a temporary disk,

before resuming writing the snapshot data to the second disk, the method further comprises: creating the temporary disk as the second disk;

after obtaining the result of the comparison, the method further comprises: and deleting the temporary disk.

5. The method of any of claims 1 to 3, wherein in the case that the first disk supports a merge function compact, the method further comprises:

forbidding the merging function of the first disk in the processes of creating the snapshot data and executing the data verification processing method;

after the result of the comparison is obtained, the merge function is turned back on.

6. The method of any of claims 1 to 3, wherein after comparing the data of the first disk corresponding to the snapshot data with the data restored to the second disk, the method further comprises:

determining to restore the integrity and correctness of the data written to the second disk according to the result of the comparison;

presenting the integrity and the correctness to a user.

7. A data verification processing apparatus comprising:

the system comprises an acquisition module, a backup module and a storage module, wherein the acquisition module is used for acquiring snapshot data, and the snapshot data is obtained by taking a snapshot of a first disk and is used for backing up data of the first disk;

the recovery module is used for recovering and writing the snapshot data into a second disk prepared in advance;

and the comparison module is used for comparing the data of the first disk corresponding to the snapshot data with the data restored and written into the second disk, wherein the comparison result is used for verifying the condition of the snapshot data after rollback.

8. The apparatus according to claim 7, wherein, in the case that the snapshot data is data obtained by snapshotting a snapshot device based on a log structure, the recovery module includes:

a writing module, configured to write the snapshot data into the second disk;

a replay module, configured to replay, on the second disk, all the input/output IO operations that have occurred on the first disk from a time when the snapshot data is created, and apply all the IO operations to the second disk, where the IO operations include: IO write operations.

9. The apparatus of claim 8, wherein,

the replay module is configured to replay all the IO operations on the second disk, obtain logs corresponding to all the IO operations, and apply the logs to the second disk.

10. The apparatus of any of claims 7 to 9, wherein the second disk is a temporary disk, the apparatus further comprising:

a creating module, configured to create the temporary disk as the second disk before the snapshot data is restored and written to the second disk;

and the deleting module is used for deleting the temporary disk after the comparison result is obtained.

11. The apparatus according to any one of claims 7 to 9, wherein in a case where the first disk supports a merge function compact, the apparatus further comprises:

a merge function control module, configured to prohibit a merge function of the first disk during the processes of creating the snapshot data and executing the data verification processing method; after the result of the comparison is obtained, the merge function is turned back on.

12. The apparatus of any of claims 7 to 9, wherein the apparatus further comprises:

and the display module is used for determining the integrity and the correctness of the data which is restored to the second disk according to the comparison result and displaying the integrity and the correctness to a user.

13. A storage medium, wherein the storage medium comprises a stored program, wherein the program when executed performs the method of any one of claims 1 to 6.

14. A processor, wherein the processor is configured to execute a program, wherein the program performs the method of any one of claims 1 to 6 when executed.