CN113254268B

CN113254268B - Data processing method and device, equipment and storage medium

Info

Publication number: CN113254268B
Application number: CN202110552937.XA
Authority: CN
Inventors: 李凡
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-05-20
Filing date: 2021-05-20
Publication date: 2024-05-24
Anticipated expiration: 2041-05-20
Also published as: CN113254268A

Abstract

The application discloses a data processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring the target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N; and based on the N backup tasks, backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk, wherein the backup speed is higher.

Description

Data processing method and device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of data processing, and relates to a data processing method, a device, equipment and a storage medium.

Background

With the continuous development of virtualization technology, virtual machines have been widely used in computer technology. The Virtual Machine (VM) backup can improve the reliability and stability of the VM, so it has a very important meaning.

The virtual machine backup, namely a process that the virtual machine reads target data from a target file of a Virtual Disk (VDISK) through a backup task (backup job) and writes the target data into a backup image (backup image).

For a backup task, the virtual machine reads the data of one data block from the target file of the virtual disk, then writes the data of the data block into the backup mirror image, reads the data of the next data block into the backup mirror image, traverses all the data blocks of the target file in sequence, and writes all the target data into the backup mirror image.

The existing virtual machine backup scheme is as follows: setting the number of backup tasks as a fixed value N, and enabling the virtual machine to read data of N data blocks from the target data in parallel according to the N backup tasks, then writing the data of the N data blocks into a backup mirror image, traversing all the target data in sequence, and finishing backup.

However, in the above scheme, a fixed number of backup tasks are used to backup the target file, and the backup rate cannot be guaranteed.

Disclosure of Invention

The embodiment of the application provides a data processing method, a data processing device, data processing equipment and a storage medium, which have higher backup rate.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a data processing method, which comprises the following steps:

acquiring the target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N;

And backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

The embodiment of the application provides a data processing device, which comprises:

The acquisition module is used for acquiring the target number N of the backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N;

and the backup module is used for backing up the target data stored in at least one virtual disk to the backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

The embodiment of the application also provides electronic equipment, which comprises: the data processing system comprises a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor realizes the data processing method when executing the program.

The embodiment of the application also provides a storage medium, on which a computer program is stored, which when being executed by a processor, implements the above-mentioned data processing method.

The data processing method, device, equipment and storage medium provided by the embodiment of the application comprise the following steps: acquiring the target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N; and backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks. In the scheme of the application, the target number N of the backup tasks is obtained first, and then the target data is backed up based on the N backup tasks. Therefore, the backup rate of the N backup tasks is higher than that of other backup tasks, so that the scheme of the application has higher backup rate in the backup process.

Drawings

FIG. 1 is a schematic diagram of an alternative architecture of a data processing system according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of an alternative method for processing data according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of an alternative method for processing data according to an embodiment of the present application;

FIG. 4 is a schematic flow chart of an alternative method for processing data according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of an alternative data processing method according to an embodiment of the present application;

FIG. 6A is a schematic flow chart of an alternative method for processing data according to an embodiment of the present application;

FIG. 6B is a schematic flow chart of an alternative method for processing data according to an embodiment of the present application;

FIG. 7 is a schematic flow chart of an alternative data processing method according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an alternative backup process according to an embodiment of the present application;

FIG. 9 is a schematic diagram of an alternative backup process according to an embodiment of the present application;

FIG. 10 is a schematic diagram of an alternative backup process according to an embodiment of the present application;

FIG. 11 is a schematic diagram of an alternative configuration of a data processing apparatus according to an embodiment of the present application;

Fig. 12 is a schematic structural diagram of an alternative electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application more clear, the specific technical solutions of the application will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are illustrative of the application and are not intended to limit the scope of the application.

The embodiment of the application can provide a data processing method, a data processing device, data processing equipment and a storage medium. In practical application, the data processing method may be implemented by a data processing device, and each functional entity in the data processing device may be cooperatively implemented by a hardware resource of an electronic device (such as a terminal device), a computing resource such as a processor, and a communication resource (such as for supporting communications in various manners such as implementing an optical cable and a cellular).

The data processing method provided by the embodiment of the application is applied to a data processing system, and the data processing system is composed of a client and a data processing end.

As an example, the structure of a data processing system may be as shown in FIG. 1, comprising: a client 10 and a data processing terminal 20.

In an example, the client 10 and the data processing 20 may be the same physical entity; in an example, as shown in fig. 1, the client 10 and the data processing terminal 20 may be different physical entities, and interaction between the client 10 and the data processing terminal 20 is performed through the network 30.

Here, the client 10 is configured to receive an operation of a user, and send a backup request to the data processing terminal 20 based on the operation of the user. The data processing end 20 is configured to receive a backup request sent by the client 10, and backup target data.

In the embodiment of the present application, based on the data processing system shown in fig. 1, a client sends an instruction for backing up data to a data processing end, and the data processing end receives the instruction for backing up data and executes: acquiring the target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N; and backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

Embodiments of a data processing method, apparatus, device, and storage medium according to embodiments of the present application are described below with reference to the schematic diagram of a data processing system shown in fig. 1.

The embodiment provides a data processing method, which is applied to a data processing device, wherein the data processing device can be implemented on an electronic device serving as a data processing end. The functions performed by the method may be performed by a processor in an electronic device, which may of course be stored in a computer storage medium, as will be seen, comprising at least a processor and a storage medium.

The electronic device may be any device having information processing capabilities, and in one embodiment, the electronic device may be an intelligent terminal, such as a notebook or other electronic device having wireless communication capabilities, an AR/VR device, a mobile terminal. In another embodiment, the electronic device may also be a terminal device with computing capabilities that is not portable, such as a desktop computer, or the like.

Of course, the embodiments of the present application are not limited to the provided methods and hardware, but may be implemented in various ways, such as being provided as a storage medium (storing instructions for performing the data processing methods provided by the embodiments of the present application).

Fig. 2 is a flowchart of a data processing method according to an embodiment of the present application, which is used for backing up a target file in at least one virtual disk, and a detailed description will be given of the backup process.

As shown in fig. 2, the method comprises the steps of:

S201, the electronic equipment acquires the target number N of backup tasks.

The backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; m is different from N. In short, the backup rate corresponding to the target number N of backup tasks is optimal.

And the backup rate is the size of the data volume of the backup mirror image to backup the data of the virtual disk in unit time.

In one example, the optimal backup rate is related to the type of stored software. In particular, for a known storage type, its backup rate at different numbers of backup tasks is known, i.e. the optimal number of backup tasks corresponding to the optimal backup rate is known. That is, for this known storage type, the backup rate is higher for the optimal number of backup tasks than for the other non-optimal number of backup tasks.

In this case, S201 may be implemented as: the electronic equipment acquires the type of the storage software, searches the backup task number corresponding to the optimal backup rate from a plurality of backup task numbers in the storage software type, and takes the backup task number as a target number N of the backup tasks.

In another example, there are many factors related to the optimal backup rate, and specific factors affecting the optimal backup rate cannot be determined. In this case, S201 may be implemented as: the electronic device sets a plurality of numbers, and for the backup rate under each of the plurality of backup tasks, the number of backup tasks with the largest backup rate in the plurality of backup rates is used as a target number N of backup tasks.

S202, the electronic equipment backs up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

Wherein the backup image is used for storing backup data. Specifically, for each virtual disk, a backup image may be established for storing backup data of the virtual disk.

The target data is part or all of the data to be backed up. In an embodiment of the present application, at least one virtual disk stores target data.

The electronic equipment configures the number of backup tasks as N, and starts N backup tasks in parallel through a backup manager (backup manager), and adopts the N backup tasks to backup target data of at least one virtual disk. Specifically, the electronic device reads N data blocks of the target data from at least one virtual disk, stores the N data blocks in the backup image corresponding to each virtual disk in the at least one virtual disk, then reads another N data blocks of the target data from the at least one virtual disk, stores the N data blocks in the backup image corresponding to each virtual disk in the at least one virtual disk, and repeatedly reads and stores the N data blocks until all the target data are stored in the corresponding backup images, thereby completing the backup of the target data.

It should be noted that, if the test data does not belong to the data to be backed up, the target data stored in at least one virtual disk is all the data to be backed up; and if the test data belong to the data to be backed up, the target data stored in the at least one virtual disk are the data except the test data in the data to be backed up.

The data processing method provided by the embodiment of the application obtains the target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N; and backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks. In the scheme of the application, the target number N of the backup tasks is obtained first, and then the target data is backed up based on the N backup tasks. Therefore, the backup rate of the N backup tasks is higher than that of other backup tasks, so that the scheme of the application has higher backup rate in the backup process.

Specific implementation of the electronic device to acquire the target number N of backup tasks in S201 is described below. Among them, depending on the manner of acquiring N, the following manner a or manner B may be included but not limited thereto.

As shown in fig. 3, the mode a may include, but is not limited to, S2011A to S2013A described below.

S2011A, the electronic equipment sets at least two test numbers.

The electronic equipment can set the number of the test quantity according to actual requirements.

S2011A may be implemented as: the electronic device may set at least two test quantities according to a preset algorithm.

The preset algorithm for setting the test quantity may include: a random algorithm, an arithmetic increment algorithm, and the like.

For example, the electronic device may set 10 test numbers, 2,4, 6, 8, 10, 12, 14, 16, 18, 20, respectively, according to an arithmetic progression algorithm.

And S2012A, the electronic equipment respectively counts the backup rate of each test number backup task in the at least two test numbers.

Specifically, the electronic device counting the backup rate under the first number of backup tasks includes: and starting a first number of backup tasks in parallel, backing up test data by adopting the first number of backup tasks, and counting the backup rate when backing up the test data by adopting the first number of backup tasks.

The first number is any one of at least two test numbers.

The electronic device traverses each of the at least two test quantities, and for each test quantity, operates with reference to the first number of execution procedures to obtain a backup rate corresponding to each test quantity.

In one example, the test data may be a set of known data that is dedicated to determining the target task number N. In this case, the test data does not belong to the data to be backed up.

In another example, the test data belongs to the data to be backed up.

The electronic device performing backup of the test data by adopting the first number of backup tasks may include: the electronic device starts from the initial section of the test data, reads a first number of data blocks, stores the first number of data blocks in the backup image, reads the next first number of data blocks of the test data, stores the next first number of data blocks in the backup image, and repeatedly reads and stores the next first number of data blocks.

The electronic device counting backup rate when the first number of backup tasks is used for backing up the test data may include: the electronic equipment counts the data volume backed up in the first time period, and takes the quotient of the data volume and the duration of the first time period as the backup rate under the first number of backup tasks.

Or the electronic equipment adopts a speed measurement algorithm to measure the real-time backup rate under the first number of backup tasks in real time, and calculates the average value of all the real-time backup rates in a period of time to serve as the backup rate under the first number of backup tasks.

For example, the test quantity includes: 2. 4, 6, 8, 10, 12, 14, 16, 18, 20; the electronic equipment adopts 2 backup tasks to backup test data, and the backup rate under the 2 backup tasks is counted to be 20 megabytes per second (MB/s); the electronic equipment adopts 4 backup tasks to backup test data, and the backup rate under the 4 backup tasks is counted to be 30MB/s; the electronic equipment adopts 6 backup tasks to backup test data, and the backup rate under the 6 backup tasks is counted to be 40MB/s; the electronic equipment adopts 8 backup tasks to backup test data, and the backup rate under the 8 backup tasks is counted to be 50MB/s; the electronic equipment adopts 10 backup tasks to backup test data, and the backup rate under the 10 backup tasks is counted to be 60MB/s; the electronic equipment adopts 12 backup tasks to backup test data, and the backup rate under the 12 backup tasks is counted to be 65MB/s; the electronic equipment adopts 14 backup tasks to backup test data, and the backup rate under the 14 backup tasks is counted to be 75MB/s; the electronic equipment adopts 16 backup tasks to backup test data, and the backup rate under the 16 backup tasks is counted to be 70MB/s; the electronic equipment adopts 18 backup tasks to backup test data, and the backup rate under the 18 backup tasks is counted to be 60MB/s; the electronic equipment adopts 20 backup tasks to backup the test data, and the backup rate under the 20 backup tasks is counted to be 60MB/s.

For example, the number of tests for the backup tasks and the backup rate under the number of tests may be saved as table 1.

TABLE 1

Number of tests	Testing backup rate under a number of backup tasks
		2	20MB/s
4	30MB/s
		6	40MB/s
8	50MB/s
		10	60MB/s
12	65MB/s
		14	75MB/s
16	70MB/s
		18	60MB/s
20	60MB/s

S2013A, the electronic equipment takes the test number with the largest corresponding backup rate in the at least two test numbers as the N.

The electronic device compares the at least two backup rates acquired in S2012A to obtain a maximum backup rate of the at least two backup rates, checks the test number corresponding to the maximum backup rate, and uses the test number corresponding to the maximum backup rate as the target number N of backup tasks.

For example, assuming that the backup rate for each test number of backup tasks is shown in table 1 at S2012A, S2013A may be implemented as: the electronic equipment reads the table 1 to obtain the maximum backup rate of 75MB/s, searches the test number corresponding to the backup rate of 75MB/s to be 14, and considers 14 as the target number N of the backup tasks.

As shown in fig. 4, the mode B may include, but is not limited to, S2011B to S2017B described below.

It should be noted that the following initial measurement number and the measurement number after the increment are a relative concept. That is, for one cycle described below, a pair of an initial measurement number and an incremented measurement number are corresponded.

And S2011B, the electronic equipment acquires the initial test quantity.

Wherein the initial test number is greater than or equal to 1.

S2011B may be implemented as: the electronic equipment acquires the default number of the equipment as the initial measurement number; or the electronic equipment selects one quantity from a plurality of preset test quantities based on the operation of a user to serve as an initial test quantity; or the electronic device may assign the number of other ways as the initial test number.

And S2012B, the electronic equipment counts the backup rates under the initial test number of backup tasks to obtain an initial rate.

The specific implementation of S2012B may refer to the implementation process of the electronic device in S2012A to count the backup rate under the first number of backup tasks, which is not described herein.

And S2013B, the electronic equipment increases the initial test quantity to the increased test quantity according to an increasing rule.

The specific content of the increment rule can be configured according to actual requirements, and the embodiment of the application is not limited to the specific content.

For example, the increment rule may include: the difference is incremented by 2.

For another example, the increment rule may include: the equal ratio is increased, and the ratio is 2.

Specifically, S2013B may be implemented as: and the electronic equipment performs incremental calculation on the initial measurement quantity according to the limiting conditions of the incremental rule to obtain the incremental test quantity.

For example, assuming that the initial measurement number is 2, the increment rule includes equal difference increment, and the difference is 2, the electronic device calculates 2 plus 2 to be equal to 4 after the initial measurement number is subjected to equal difference increment with the difference of 2; the number of tests after increment was 4.

S2014B, the electronic equipment counts the backup rate of the increased test number of backup tasks to obtain the increased rate.

The implementation of S2014B may refer to the implementation process of the electronic device in S2012A to count the backup rate under the first number of backup tasks, which is not described herein.

And S2015B, the electronic equipment judges whether the increased speed is smaller than or equal to the initial speed.

The electronic device determines whether the incremented rate is less than or equal to the initial rate, and if the incremented rate is less than or equal to the initial rate, performs S2016B described below; if the incremented rate is greater than the initial rate, the following S2017B is performed.

S2016B, the electronic device takes the initial measurement quantity corresponding to the initial rate as N.

And the electronic equipment takes the initial measurement quantity corresponding to the initial rate in the cycle as the target quantity N of the backup tasks.

S2017B, the electronic equipment takes the increased test quantity as a new initial test quantity.

And the electronic equipment takes the increased test quantity as new initial test quantity, and continuously counts the backup rate of the new initial test quantity under the backup tasks until the increased rate is smaller than or equal to the initial rate. In other words, the electronic apparatus re-executes the above-described S2011B to S2017B in order with the incremented number of tests as a new initial number of tests.

Exemplary, as shown in fig. 5, implementation of mode B may include, but is not limited to, S501 to S510 described below.

S501, the electronic equipment acquires a first test quantity.

S502, the electronic equipment counts backup rates under the first test number of backup tasks to serve as a first rate.

S503, the electronic device increases the first test quantity to a second test quantity.

S504, the electronic equipment counts the backup rate under the second test number of backup tasks to be used as a second rate.

S505, the electronic device judges whether the second rate is smaller than or equal to the first rate.

If the electronic device determines that the second rate is less than or equal to the first rate, the following S506 is performed.

If the electronic device determines that the second rate is greater than the first rate, the following S507 is executed.

S506, the electronic device takes the first rate as N.

S507, the electronic equipment increases the second test quantity to a third test quantity.

S508, the electronic equipment counts the backup rate under the third test number of backup tasks to be used as a third rate.

S509, the electronic device judges whether the third rate is smaller than or equal to the second rate.

If the electronic device determines that the third rate is less than or equal to the second rate, the following S510 is executed.

If the electronic device determines that the third rate is greater than the second rate, the electronic device still refers to S507 to S510 to perform the cyclic operation; until N is obtained.

S510, the electronic equipment takes the second rate as N.

Specific implementations of S202 may include, but are not limited to, scheme a, scheme b, or scheme c.

If only one virtual disk (the first virtual disk) includes the target data, S202 may be implemented as an embodiment a, and if P virtual disks include the target data, S202 may be implemented as an embodiment b. Wherein P is greater than 1.

And (c) backing up the target data stored by the virtual disk into a backup image corresponding to the virtual disk by the electronic equipment based on the N backup tasks.

Specifically, the electronic device starts N backup tasks in parallel through the backup manager, and uses the N backup tasks to backup the target data of the first virtual disk.

The process of backing up the target data of the first virtual disk by the electronic device through N backup tasks comprises the following steps: the electronic device reads N data blocks of target data from the first virtual disk through N backup tasks, stores the N data blocks into a backup image corresponding to the first virtual disk, then reads another N data blocks of the target data from the first virtual disk, stores the other N data blocks into the backup image corresponding to the first virtual disk, and repeatedly reads and stores until all the target data in the first virtual disk are stored into the backup image corresponding to the first virtual disk.

Scheme b as shown in fig. 6A may include, but is not limited to, S2021b to S2022b described below.

S2021b, the electronic device distributes the N backup tasks to the P virtual disks.

And the electronic equipment distributes N backup tasks to the P virtual disks according to the distribution rule.

The specific content of the allocation rule may be configured according to actual requirements, which is not limited in the embodiment of the present application. For example, the allocation rules may include: average allocation, random allocation, or priority allocation of virtual disks.

Assuming that the allocation rule is average allocation, the target number of backup tasks acquired in S201 is 20, which are respectively: backup task 1, backup task 2 … … backup task 20; the 5 virtual disks comprise target data, namely a virtual disk 1 and a virtual disk 2 … … and a virtual disk 5; the electronic device may evenly distribute 20 backup tasks to 5 virtual disks, i.e., 4 backup tasks for each virtual disk, according to an average distribution rule, as shown in table 2.

TABLE 2

Virtual disk	Backup tasks corresponding to virtual disk
		Virtual disk 1	Backup task 1, backup task 2, backup task 3, and backup task 4
Virtual disk 2	Backup task 5, backup task 6, backup task 7 and backup task 8
		Virtual disk 3	Backup task 9, backup task 10, backup task 11 and backup task 12
Virtual disk 4	Backup task 13, backup task 14, backup task 15, and backup task 16
		Virtual disk 5	Backup task 17, backup task 18, backup task 19, and backup task 20

S2022b, for each virtual disk in the P virtual disks, the electronic device backs up the target data stored by the virtual disk to the backup image corresponding to the virtual disk based on the backup task allocated to the virtual disk.

Specifically, the electronic device executes the following operations for each of the P virtual disks until all the target data in the P virtual disks are backed up. Taking the second virtual disk as an example, the operations performed include: and the electronic equipment parallelly starts a backup task corresponding to the second virtual disk through the backup manager, and backups target data of the second virtual disk by adopting the backup task corresponding to the second virtual disk.

In the scheme a, a specific implementation of the electronic device adopting the backup task corresponding to the second virtual disk to backup the target data of the second virtual disk may refer to a process that the electronic device adopts N backup tasks to backup the target data of the first virtual disk, which is not described herein.

It can be seen that, in the scheme b, when the target data is backed up, the backup task corresponding to each virtual disk is fixed.

The implementation of scheme c is described below.

In the scheme c, when the target data is backed up, the backup task corresponding to each virtual disk can be dynamically adjusted. For example, after the at least one virtual disk finishes the backup, the backup task corresponding to the virtual disk that does not finish the backup may be dynamically adjusted. As shown in fig. 6B, implementation of scheme c may include, but is not limited to, S2021c to S2024c described below.

S2021c, the electronic device distributes the N backup tasks to the P virtual disks.

The specific implementation of S2021c may refer to S2021b.

S2022c, the electronic device backs up the target data stored by the virtual disk to the backup image corresponding to the virtual disk based on the backup task allocated to the virtual disk for each virtual disk in the P virtual disks.

An implementation of S2022c may refer to S2022b, unlike S2022b, in scheme b, S2022b performs until the backup of all virtual disks is completed. In scenario c, S2022c is performed, and the electronic device performs S2023c after performing the first time.

Wherein the first time may be a preset time period. The application is not limited to the duration of the first time, and can be configured according to the time requirement. For example, the first time may be set to a smaller period of time, which corresponds to performing S2023c immediately after performing S2022c, i.e., the detection in S2023c is real-time.

S2023c, the electronic device detects the backup states of the P virtual disks.

Wherein the backup state includes: the unrepeated, the backed-up neutralization and the backed-up are completed.

The electronic device may detect the backup status of the virtual disk using a detection algorithm, and implementation of S2023c may include: the electronic equipment adopts a detection algorithm to detect the backup states of the P virtual disks.

Assuming that P is 5, the P virtual disks are virtual disk 1, virtual disk 2, virtual disk 3, virtual disk 4, and virtual disk 5, respectively; the backup states of the 5 virtual disks detected by the electronic device may be as shown in table 3.

TABLE 3 Table 3

S2024c, the electronic device distributes the N backup tasks to the P-K virtual disks.

And the P-K virtual disks are virtual disks which are not backed up in the P virtual disks.

For example, if the backup states of the P virtual disks obtained in S2023c are shown in table 3, and the backup task corresponding to each virtual disk is shown in table 2, the electronic device allocates the backup task 17, the backup task 18, the backup task 19, and the backup task 20 corresponding to the virtual disk 5 to the virtual disk 1, the virtual disk 2, the virtual disk 3, and the virtual disk 4 on average; the backup tasks corresponding to the reassigned virtual disk may be as shown in table 4.

TABLE 4 Table 4

Where/represents no backup tasks.

S2025c, the electronic device backups the target data stored by the virtual disk to the backup image corresponding to the virtual disk based on the backup task allocated to the virtual disk for each virtual disk in the P-K virtual disks.

The implementation of S2025c may refer to S2022b, and will not be described herein.

It can be understood that the electronic device may detect the backup progress of each virtual disk in real time, and when detecting that the third virtual disk completes the backup, obtain at least one backup task corresponding to the third virtual disk, and assign the at least one backup task corresponding to the third virtual disk to the fourth virtual disk.

The fourth virtual disk is a virtual disk with unfinished backup. In one example, the fourth virtual disk is any virtual disk that did not complete the backup. In another example, the fourth virtual disk is the highest priority virtual disk of any of the outstanding backup virtual disks. In yet another example, the fourth virtual disk is a plurality of virtual disks that have not been backed up.

The data processing method provided by the embodiment of the application is further described through a specific application scene.

The number of reads/writes per second (Input/Output Operations Persecond, IOPS) is used to measure an indicator of the performance of the storage device.

The Input Output (IO) depth may refer to the number of IOs that are currently concurrently committed to the storage device. The IO depth may also be referred to as IO queue depth, which is equal to the number of backup tasks in the embodiment of the present application. For example, 1 IO is submitted to the storage device, and 3 IOs are simultaneously submitted before the IO returns, and the IO depth of the storage device is 4. Assuming that 1 IO is submitted to the storage device, after the IO returns, the next 1 IO can be submitted, and the IO depth of the storage device is 1.

The optimal IO depth may refer to the IO depth corresponding to the highest IOPS performance.

The IO performance of the currently mainstream storage device has a feature that the IO performance of the storage device increases with the increase of the IO depth, and the IOPS starts to decrease after reaching a certain value, for example, table 6 illustrates the IO performance of one storage device, and it can be seen that the optimal IO depth of the storage device is 32.

TABLE 6

IO depth	1	2	4	8	16	32	64	128
									IOPS	4235	6233	8568	10531	11921	12756	12168	12119

In practice, the optimal IO depth for different types of storage devices may be different, for example, the optimal IO depth for NAS type storage devices may be 16, the optimal IO depth for SAN type storage devices may be 32, and the optimal IO depth for distributed virtual storage type storage devices may be 64.

IO depth control for virtual machine backup may include scheme one or scheme two as follows.

Scheme one: 1 VDISK corresponds to a fixed 1 backup job, i.e. the backup IO depth is fixed to 1. Under the scheme, the IOPS performance of the storage device is poor, the backup time is long, and the backup rate is low.

Scheme II: 1 VDISK corresponds to a fixed X backup jobs, i.e. the backup IO depth is fixed to X. Because the optimal IO depth of various storage devices is different, the fixed X value is difficult to correspond to the optimal IO depth, so that the backup rate is low.

The VM backup rate is lower under the two schemes, and the invention provides a scheme with higher backup rate.

Assuming that the VM includes a virtual disk A, the target data in virtual disk A needs to be backed up. As shown in fig. 7, the data processing procedure may include S701 to S712 described below.

S701, the electronic equipment acquires the number 4 of default backup tasks.

S702, the electronic equipment starts 4 backup tasks through the backup manager, and target data in the virtual disk A are stored in the backup mirror image.

S703, the electronic equipment counts the average backup rate of 4 backup tasks within 20 seconds through the backup manager.

The electronic equipment calculates the average backup rate of 30MB/s under 4 backup tasks within 20 seconds through the backup manager.

By way of example, FIG. 8 illustrates backup data flow for 4 backup tasks.

S704, the electronic equipment adds 2 backup tasks to the virtual disk A through the backup manager.

And S705, the electronic equipment runs 6 backup tasks through the backup manager and continues to backup the target data in the virtual disk A.

S706, the electronic equipment counts the average backup rate of 6 backup tasks within 20 seconds through the backup manager.

The electronic equipment calculates the average backup rate of 50MB/s under 6 backup tasks within 20 seconds through the backup manager.

By way of example, FIG. 9 illustrates backup data flow for 6 backup tasks.

And S707, the electronic equipment compares and determines that the backup rate is not reduced through the backup manager.

The average backup rate of the electronic device under the 6 backup tasks is greater than the average backup rate under the 4 backup tasks through the backup manager.

And S708, the electronic equipment adds 2 backup tasks to the virtual disk A through the backup manager.

And 709, the electronic equipment runs 8 backup tasks through the backup manager and continues to backup the target data in the virtual disk A.

S710, the electronic equipment counts the average backup rate of 8 backup tasks in 20 seconds through the backup manager.

The electronic equipment calculates the average backup rate of 45MB/s under 8 backup tasks within 20 seconds through the backup manager.

By way of example, FIG. 10 illustrates backup data flow for 8 backup tasks.

S711, the electronic equipment determines that the backup rate is reduced through the comparison of the backup manager.

The average backup rate 45MB/s of the electronic equipment under the 8 backup tasks is smaller than the average backup rate 50MB/s under the 6 backup tasks through the backup manager.

And S712, the electronic equipment runs 6 backup tasks through the backup manager, and continues to backup the target data in the virtual disk A until the backup is completed.

Compared with the traditional scheme in the industry, the data processing method can continuously and dynamically adjust the number of backup tasks in the backup process, and monitor the backup rate to judge whether the optimal IO depth of the storage device is reached. And finally, the number of VM backup tasks can be adaptively adjusted, so that the optimal IO depth of the storage device is achieved, and the VM backup rate is optimal. Therefore, the backup method of the application improves the backup rate and shortens the backup time.

Fig. 11 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, and as shown in fig. 11, a data processing apparatus 110 may include an obtaining module 1101 and a backup module 1102. Wherein:

an obtaining module 1101, configured to obtain a target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks; the M is different from the N.

And the backup module 1102 is configured to backup, based on the N backup tasks, target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk.

In some embodiments, the obtaining module 1101 is specifically configured to:

setting at least two test numbers;

Respectively counting the backup rate of each of the at least two test numbers under the backup tasks;

and taking the test number with the largest corresponding backup rate of the at least two test numbers as the N.

In some embodiments, the obtaining module 1101 is specifically configured to obtain an initial test number;

counting the backup rates of the initial test number of backup tasks to obtain an initial rate;

Increasing the initial test quantity to the increased test quantity according to an increasing rule;

Counting the backup rate of the increased test number of backup tasks to obtain an increased rate;

If the increased speed is greater than the initial speed, taking the increased test quantity as a new initial test quantity, and continuously counting the backup speed of the new initial test quantity under a plurality of backup tasks until the increased speed is less than or equal to the initial speed;

And taking the initial measurement quantity corresponding to the initial rate as N under the condition that the increased rate is smaller than or equal to the initial rate.

In some embodiments, if the at least one virtual disk includes a virtual disk, the backup module 1102 is specifically configured to:

And based on the N backup tasks, backing up the target data stored by the virtual disk into a backup image corresponding to the virtual disk.

In some embodiments, if the at least one virtual disk includes P virtual disks, where P is greater than 1, the backup module 1102 is specifically configured to:

distributing the N backup tasks to the P virtual disks;

And for each virtual disk in the P virtual disks, backing up target data stored by the virtual disk into a backup image corresponding to the virtual disk based on a backup task allocated to the virtual disk.

In some embodiments, the backup module 1102 is further configured to:

If the K virtual disks included in the P virtual disks are backed up, distributing the N backup tasks to the P-K virtual disks; the P-K virtual disks are virtual disks which are not backed up in the P virtual disks;

And for each virtual disk in the P-K virtual disks, backing up target data stored by the virtual disk into a backup image corresponding to the virtual disk based on backup tasks allocated to the virtual disk.

It should be noted that, the data processing apparatus provided in the embodiment of the present application includes each unit included, which may be implemented by a processor in an electronic device; of course, the method can also be realized by a specific logic circuit; in an implementation, the Processor may be a central processing unit (CPU, central Processing Unit), a microprocessor (MPU, micro Processor Unit), a digital signal Processor (DSP, digital Signal Processor), or a Field-Programmable gate array (FPGA), etc.

The description of the apparatus embodiments above is similar to that of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus of the present application, please refer to the description of the embodiments of the method of the present application.

It should be noted that, in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software functional module, and sold or used as a separate product, the data processing method may also be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be embodied essentially or in a part contributing to the related art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes. Thus, embodiments of the application are not limited to any specific combination of hardware and software.

Correspondingly, an embodiment of the application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program which can be run on the processor, and the processor realizes the steps in the data processing method provided in the embodiment when executing the program.

Accordingly, an embodiment of the present application provides a storage medium, that is, a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the data processing method provided in the above embodiment.

It should be noted here that: the description of the storage medium and apparatus embodiments above is similar to that of the method embodiments described above, with similar benefits as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and the apparatus of the present application, please refer to the description of the method embodiments of the present application.

It should be noted that fig. 12 is a schematic diagram of a hardware entity of an electronic device according to an embodiment of the present application, as shown in fig. 12, the electronic device 120 includes: a processor 1201, at least one communication bus 1202, a user interface 1203, at least one external communication interface 1204, and a memory 1205. Wherein the communication bus 1202 is configured to enable connected communications between these components. The user interface 1203 may include a display screen, among other things, and the external communication interface 1204 may include standard wired and wireless interfaces.

The memory 1205 is configured to store instructions and applications executable by the processor 1201, and may also cache data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or processed by various modules in the processor 1201 and the electronic device, and may be implemented by a FLASH memory (FLASH) or a random access memory (Random Access Memory, RAM).

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in some embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application. The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units; can be located in one place or distributed to a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read Only Memory (ROM), a magnetic disk or an optical disk, or the like, which can store program codes.

Or the above-described integrated units of the application may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the embodiments of the present application may be embodied essentially or in a part contributing to the related art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

The foregoing is merely an embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of data processing, the method comprising:

Acquiring the target number N of backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks, and is the optimal backup rate; the M is different from the N; the obtaining the target number N of backup tasks includes: setting at least two test numbers; respectively counting the backup rate of each of the at least two test numbers under the backup tasks; taking the test number with the largest corresponding backup rate of the at least two test numbers as the N;

And starting the N backup tasks, and backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk.

2. The method of claim 1, wherein the obtaining the target number N of backup tasks comprises:

Acquiring initial test quantity;

3. The method of claim 1, wherein if the at least one virtual disk comprises P virtual disks, the P is greater than 1; the starting the N backup tasks, backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk, including:

distributing the N backup tasks to the P virtual disks;

4. A method according to claim 3, characterized in that the method further comprises:

5. A data processing apparatus, the apparatus comprising:

The acquisition module is used for acquiring the target number N of the backup tasks; the backup rate of the N backup tasks is greater than or equal to the backup rate of the M backup tasks, and is the optimal backup rate; the M is different from the N; the obtaining the target number N of backup tasks includes: setting at least two test numbers; respectively counting the backup rate of each of the at least two test numbers under the backup tasks; taking the test number with the largest corresponding backup rate of the at least two test numbers as the N;

and the backup module is used for starting the N backup tasks and backing up the target data stored in at least one virtual disk to the backup image corresponding to each virtual disk in the at least one virtual disk.

6. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, the processor implementing the data processing method of any one of claims 1 to 4 when the program is executed.

7. A storage medium having stored thereon a computer program which, when executed by a processor, implements the data processing method of any of claims 1 to 4.