CN114780296B - Data backup method, device and system of database cluster - Google Patents
Data backup method, device and system of database cluster Download PDFInfo
- Publication number
- CN114780296B CN114780296B CN202210499026.XA CN202210499026A CN114780296B CN 114780296 B CN114780296 B CN 114780296B CN 202210499026 A CN202210499026 A CN 202210499026A CN 114780296 B CN114780296 B CN 114780296B
- Authority
- CN
- China
- Prior art keywords
- backup
- target
- task
- node
- database cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 93
- 230000008569 process Effects 0.000 claims abstract description 38
- 238000011084 recovery Methods 0.000 claims description 78
- 238000003860 storage Methods 0.000 claims description 23
- 238000004140 cleaning Methods 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 21
- 230000002159 abnormal effect Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000000717 retained effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000014759 maintenance of location Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a data backup method, device and system of a database cluster, which are used for solving the problems of complex backup process, low backup efficiency and success rate, influence on the production operation of the database cluster and the like in the conventional backup scheme. The method comprises the following steps: determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the running state information and the load capacity information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster; determining a task execution plan of the target backup task based on task related information of the backup task currently being executed; and scheduling the backup node to execute the target backup task based on the task execution plan.
Description
Technical Field
The present application relates to the field of database technologies, and in particular, to a method, an apparatus, and a system for backing up data in a database cluster.
Background
At present, for data backup of a database cluster, a backup node in the database cluster is usually designated manually, and a backup task for the database cluster is executed at a designated time point by using a timing mechanism such as a crontab. However, once the backup node is abnormal or the backup time is improperly selected, the backup failure can be caused, the generation and operation of the database cluster are affected, and all backup tasks need to be manually involved and the backup node and the backup time are adjusted, so that the complex backup process and the low backup efficiency are caused.
Disclosure of Invention
The embodiment of the application aims to provide a data backup method, device and system for a database cluster, which are used for solving the problems of complex backup process, low backup efficiency and success rate, influence on the production operation of the database cluster and the like in the conventional backup scheme.
In order to achieve the above object, the embodiment of the present application adopts the following technical scheme:
in a first aspect, an embodiment of the present application provides a data backup method for a database cluster, including:
determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the running state information and the load capacity information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster;
determining a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and scheduling the backup node to execute the target backup task based on the task execution plan.
In a second aspect, an embodiment of the present application provides a data backup system of a database cluster, including: scheduling a platform and a database cluster;
The scheduling platform is configured to determine, from among the serviceable nodes in the database cluster, a backup node for executing a target backup task based on operational status information of the serviceable nodes in the database cluster, where the target backup task is used to backup target data of the database cluster, determine a task execution plan of the target backup task based on task related information of a backup task currently being executed, and send a scheduling request to the backup node based on the task execution plan;
And the backup node is used for responding to the scheduling request and executing the target backup task.
In a third aspect, an embodiment of the present application provides a data backup apparatus for a database cluster, including:
The first determining module is used for determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the operation state information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster;
The second determining module is used for determining a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and the scheduling module is used for scheduling the backup node to execute the target backup task based on the task execution plan.
In a fourth aspect, an embodiment of the present application provides an electronic device, including:
A processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method according to the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, characterized in that instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method according to the first aspect.
According to the scheme of the embodiment of the application, the running state information and the load capacity information of the nodes are considered to reflect the carrying capacity of the nodes to the backup task, and the backup node for executing the backup task is determined from the available service nodes of the database cluster based on the running state information and the load capacity information of the available service nodes in the database cluster, so that the manually appointed backup node is replaced, the labor is saved, the backup process is simplified, the backup efficiency is improved, the backup node can be ensured to run normally, and the problems of backup failure or influence on the production and the running of the database cluster caused by abnormal conditions such as incapacity of service, overlarge load or insufficient resources of the backup node are avoided; the task execution plan of the target backup task is determined based on the task related information of the backup task currently being executed, and the backup node is further scheduled to execute the target backup task according to the task execution plan, so that not only can the labor force be saved, the backup process be simplified and the backup efficiency be improved, but also the problems of large consumption of processing resources, increase of processing time and the like caused by excessive backup tasks executed simultaneously can be avoided, and a guarantee is provided for the generation and the operation of the backup task and the database cluster.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:
FIG. 1 is a schematic diagram of an implementation environment to which a data backup method of a database cluster according to an embodiment of the present application is applicable;
FIG. 2 is a schematic diagram of an implementation environment to which a data backup method of a database cluster according to another embodiment of the present application is applicable;
FIG. 3 is a flowchart illustrating a method for backing up data in a database cluster according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating a method for backing up data of a database cluster according to another embodiment of the present application;
FIG. 5 is a schematic diagram illustrating data recovery of a database cluster according to an embodiment of the present application;
FIG. 6 is a schematic diagram illustrating a data backup apparatus for a database cluster according to an embodiment of the present application;
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the application may be practiced otherwise than as specifically illustrated or described herein. Furthermore, in the present specification and claims, "and/or" means at least one of the connected objects, and the character "/" generally means a relationship in which the associated object is an "or" before and after.
Partial conceptual description:
database cluster: a virtual single database logical image is formed using at least two or more database servers (also referred to as nodes) that can provide transparent data services to clients as a single database system.
MySQL database: is a relational database.
Binary log: the data changes generated on the database are written to the file in binary format.
NFS (Network FILE SYSTEM): network file systems that enable users to access files on a network just as if they were using their own computer.
In order to solve the problems that the backup process is complicated, the backup efficiency and the success rate are low, the production operation of a database cluster is influenced and the like in the conventional backup scheme, the embodiment of the application aims to provide the data backup scheme of the intelligent scheduling database cluster, and the problems that the backup failure or the production operation of the database cluster is influenced due to the abnormal conditions of incapacity of service, overlarge load or insufficient resources and the like of the backup node are avoided because the running state information and the load capacity information of the node can reflect the carrying capacity of the backup task and the backup node for executing the backup task is determined from the available service nodes of the database cluster based on the running state information and the load capacity information of the available service nodes of the database cluster, so that the manually appointed backup node is replaced; the task execution plan of the target backup task is determined based on the task related information of the backup task currently being executed, and the backup node is further scheduled to execute the target backup task according to the task execution plan, so that not only can the labor force be saved, the backup process be simplified and the backup efficiency be improved, but also the problems of large consumption of processing resources, increase of processing time and the like caused by excessive backup tasks executed simultaneously can be avoided, and a guarantee is provided for the generation and the operation of the backup task and the database cluster.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
For ease of understanding, an implementation environment to which a data backup method for a database cluster according to an embodiment of the present application is applicable will be described first with reference to fig. 1. As shown in fig. 1, an implementation environment to which a data backup method of a database cluster according to an embodiment of the present application is applicable may include a database cluster 1, a scheduling platform 2, and a backup pool 3.
The database cluster 1 comprises a plurality of nodes 11, and the nodes 11 are matched with each other to provide data access service for the service system, so that the normal operation of the service system is ensured. In practical applications, any type of database cluster may be used as the database cluster 1, such as MySQL database cluster, etc., and in addition, the number of database clusters 1 may be plural, and fig. 1 illustrates only one database cluster 1. As shown in fig. 2, in the backup process of the database cluster 1, the node 11 (i.e., the backup node) selected to perform the backup task of the database cluster 1 generates a corresponding backup set based on the data to be backed up of the database cluster 1 and stores the backup set in the backup pool 3, and in addition, the backup node returns the execution result of the target backup task to the scheduling platform 2, and 1 generates a backup log corresponding to the performed backup task and writes the backup log into the backup pool 3 for use in the subsequent recovery of the database cluster 2. Specifically, the backup node may write the generated backup set into the backup pool 3 in a streaming backup manner.
The scheduling platform 2 may include a Server (Server) or a Server cluster formed by a plurality of servers, and is capable of providing functions such as calculation and scheduling based on services of hardware resources and software resources. Specifically, as shown in fig. 2, in an embodiment of the present application, the scheduling platform 2 may determine a backup node for executing a backup task for the database cluster 1 from the serviceable nodes of the database cluster 1 based on a pre-configured intelligent scheduling mechanism and determine a task execution plan of the backup task, and then schedule the backup node to execute a target backup task according to the task execution plan. Of course, the scheduling platform 2 may also periodically maintain the backup pool 3, and when the database cluster 1 needs to be restored to a certain historical state, acquire a backup set and a backup log related to the historical state from the backup pool 3, and then perform a restoration operation on the database cluster 1 based on the acquired backup set and backup log.
The backup pool 3 can provide a data storage function that can store backup sets generated by backup nodes in the course of performing backup tasks. Optionally, as shown in fig. 2, the backup pool 3 may include, but is not limited to, a local backup pool and/or a remote backup pool (such as NFS), where the local backup pool receives a backup set sent by a backup node, and the local backup pool may compress and/or encrypt the received backup set and send the compressed and/or encrypted backup set to the remote backup pool. In practical applications, the backup pool 3 may adopt any suitable storage architecture, which is not limited in this embodiment of the present application. Illustratively, the backup pool 3 may employ GlusterFS, mooseFS, HDFS, object storage (such as ceph), S3 room storage, and so on, so that dynamic lateral expansion of space, replacement of failed disks, and so on may be implemented.
The data backup method of the database cluster provided by the embodiment of the application is used for describing the interaction process among the database cluster 1, the dispatching platform 2 and the backup pool 3 in detail, realizing the intelligent and automatic backup dispatching of the database cluster 1, further realizing uniform coordination of resources, avoiding the situation of extremely surplus or deficient generated by average allocation, avoiding the manual intervention allocation principle, simplifying the backup process, improving the backup efficiency and the backup success rate, ensuring that the production operation of the database cluster 1 can be normally performed, and the like.
Based on the above implementation environment, the method for backing up data of a database cluster according to the embodiment of the present application is described below.
Referring to fig. 3, a flowchart of a method for backing up data of a database cluster according to an embodiment of the present application is shown, and the method can be applied to the scheduling platform 2 in the above-mentioned implementation environment. As shown in fig. 3, the method may include the steps of:
S302, determining backup nodes for executing the target backup tasks from the serviceable nodes of the target database cluster based on the operation state information and the load capacity information of the serviceable nodes in the target database cluster.
The target database cluster refers to a database cluster needing to be backed up. The serviceable nodes in the target database cluster refer to nodes in the target database cluster which can normally provide data services. For example, the target database cluster includes a node 1, a node 2 and a node 3, where the node 1 is currently in a downtime state, and the node 2 and the node 3 are currently normal, and the node 2 and the node 3 are serviceable nodes.
In practical applications, the scheduling platform may periodically monitor whether each node in the target database cluster is normal by adopting various manners commonly used in the art, so as to determine the serviceable node in the target database cluster, for example, by periodically monitoring a heartbeat packet sent by a node in the target database cluster, so as to determine whether each node in the target database cluster is normal.
The operation state information of the serviceable node refers to information for reflecting the operation state of the serviceable node, which may include, but is not limited to, at least one of the following information in particular: roles of serviceable nodes, business attributes (such as number of job concurrency, job type) that can service the executing business, etc. The roles of the serviceable nodes include, but are not limited to, a writing node, a reading and writing node and the like, wherein the writing node refers to the serviceable node for executing writing operation in the production process of the database cluster, the reading node refers to the serviceable node for executing reading operation in the production process of the database cluster, and the reading and writing node refers to the serviceable node for executing reading operation and writing operation in the production process of the database cluster.
The load of the serviceable node is capability information, which refers to information for reflecting the capacity of the serviceable node to accept the load, and may specifically include, but is not limited to, at least one of the following information: master-slave replication delay time of the serviceable node, size of load received, size of available resources, and the like.
The target backup task is used for backing up target data of the target database cluster, wherein the target data refers to data needing to be backed up in the target database cluster. In order to avoid the problems of failure in backup or influence on the production operation of the database cluster caused by abnormal conditions such as incapacity of service, overlarge load or insufficient resources of the backup nodes, and the like, so as to ensure the normal execution of the target backup task, the backup nodes which can provide service and can accept the target backup task can be selected from the serviceable nodes in the target server cluster based on the operation state information and the load capacity information of the serviceable nodes in the target server cluster, and the target backup task is executed by the backup nodes.
In addition, if the node executing the backup task has serious master-slave copy delay, the execution time of the backup task is increased, if the execution time of the backup task is time-consuming, the backup task may intersect with the production peak period of the database cluster, and the production operation of the database cluster may be affected. To this end, in an alternative implementation, as shown in fig. 4, S302 may be specifically implemented as: and aiming at each serviceable node in the target database cluster, if the role of the serviceable node indicates that the serviceable node does not belong to the write node and the master-slave copy delay time length of the serviceable node is smaller than a preset time length threshold, determining the serviceable node as a backup node.
For example, assuming that the serviceable nodes in the target database cluster include the serviceable node 1, the serviceable receiving node 2 and the serviceable node 3, if the serviceable node 1 is a writing node, that is, the serviceable node 1 is used to perform writing operation in the production process of the target database cluster, and the master-slave copy delay time length of the serviceable node 2 exceeds the preset time length threshold, the serviceable node 3 may be determined as a backup node.
It should be noted that, in practical application, if the target database cluster does not belong to the write node and the master-slave copy delay time length is less than the preset time length threshold, one of the serviceable nodes may be selected to be determined as the backup node. For example, any one of the plurality of serviceable nodes may be selected to be determined as a backup node, or one serviceable node with the smallest master-slave copy delay duration may be selected from the plurality of serviceable nodes to be determined as a backup node, and so on.
It can be understood that the serviceable node with the role not belonging to the writing node and the master-slave copy delay time length smaller in the target database cluster is determined as the backup node, so that the backup node can be prevented from being used for executing writing operation in the production process of the database cluster to influence the production process of the database cluster and the execution process of the backup task, the time consumption for executing the backup task can be reduced, the intersection with the production peak time period of the database cluster can be avoided, and the normal production operation of the database cluster can be ensured.
An embodiment of the present application is herein shown a specific implementation of S302 described above. Of course, it should be understood that S302 may be implemented in other manners, which are not limited by the embodiment of the present application.
S304, determining a task execution plan of the target backup task based on the task related information of the backup task currently being executed.
The task related information of the current backup task refers to information related to the current backup task, which may specifically include, but is not limited to, the concurrency number of the current backup task, occupied processing resource information (such as occupied network traffic and disk IO), backup time, backup data volume, and the like.
The task execution plan of the target backup task may specifically include, but is not limited to, a backup start-stop time of the target backup task, a target processing resource to be occupied, and the like. The backup start-stop time comprises a backup start time and a backup end time.
The task related information of the backup task currently being executed can reflect the execution condition of the backup task currently being executed by the scheduling platform, and the task execution plan of the target backup task is determined based on the task related information of the backup task currently being executed, so that the problems of large consumption of processing resources, increase of processing time and the like caused by excessive backup tasks simultaneously being executed can be avoided, and further, a guarantee is provided for the generation and the operation of the backup task and the database cluster.
Specifically, the backup end time of the backup task currently being executed may be predicted based on the number of concurrency of the backup task currently being executed and the amount of backup data, and the backup start time of the target backup task may be further determined based on the amount of backup data of the target backup task and the backup end time of the backup task currently being executed. In addition, the processing resource information required to be occupied by the target backup task can be determined based on the concurrency number of the backup tasks currently being executed, the occupied processing resource information, the backup data volume of the target backup task and the like.
For example, if it is determined that the remaining memory of the scheduling platform is insufficient based on the memory usage rate of the scheduling platform occupied by the backup task currently being executed, 50% of the remaining memory size may be determined as the memory size allocated to the target backup task; for another example, if the network traffic occupied by the backup task currently being executed exceeds 50%, the maximum network traffic occupied by the target backup task is set to be 50MB/s, otherwise, the maximum network traffic occupied by the target backup task is set to be 100MB/s, and so on.
An embodiment of the present application is herein shown a specific implementation of S304 described above. Of course, it should be understood that S304 may be implemented in other manners, which are not limited by the embodiment of the present application.
S306, based on the task execution plan, the backup node is scheduled to execute the target backup task.
After the task execution plan of the target backup task is determined, the backup node can be scheduled according to the task execution plan, and the target backup task is executed by the backup node, so that the backup of the target data in the target database cluster is realized. Specifically, as shown in fig. 4, agent clients (agents) are deployed on each node in the target database cluster. The scheduling platform can schedule the Agent on the backup node to execute the target backup task.
In an alternative implementation, to ensure that the backup node can automatically execute the target backup task according to the task execution plan, S306 may include: generating a target backup program for executing the target backup task at least based on the task execution plan, sending the target backup program to the backup node, and triggering the backup node to execute the target backup program, so that the target backup program obtains target data of the target database cluster in the running process to generate a backup set corresponding to the target backup task.
For example, the scheduling platform may call a preset backup program template, generate a target backup program based on the task execution plan and the preset backup program template, send the target backup program to an Agent deployed on the backup node, and trigger the Agent to automatically run the target backup program. As shown in fig. 4, in the process of running the target backup program, if the target backup task is executed successfully, the Agent deployed on the backup node may store the generated backup set corresponding to the target backup task into the local backup pool and return the execution result to the scheduling platform; further, the Agent deployed on the backup node may perform preset processing (such as compression and/or encryption) on the backup set, and store the backup set after the preset processing in the remote backup pool, if the preset processing fails on the backup set or the backup set after the preset processing fails to be stored in the remote backup pool, the Agent deployed on the backup node returns an execution result indicating the failure to the scheduling platform, so that the scheduling platform continues to schedule the Agent to execute the target backup task in the next scheduling period.
Optionally, in order to ensure that the backup node can store the related backup data for the target database cluster in the backup pool for subsequent use, the scheduling platform may acquire the identification information of the backup pool to be stored, and generate the target backup program based on the identification information of the backup pool, the task execution plan of the target backup task and the preset backup program template, so that the target backup program can acquire the target data of the target database cluster in the running process to generate a backup set corresponding to the target backup task and store the backup set in the backup pool.
An embodiment of the present application is herein shown as a specific implementation of S306 described above. Of course, it should be understood that S306 may be implemented in other manners, which are not limited by the embodiment of the present application.
In order to ensure that the target backup task is executed at a proper time, so as to reduce the cross between the backup time of the target backup task and the peak period of the production process of the target database cluster as much as possible, and ensure that the target backup task can be executed successfully, in an alternative implementation, as shown in fig. 4, S306 may be specifically implemented as follows: and determining a target scheduling period matched with the target backup task based on a task execution plan of the target backup task, and further scheduling the backup node to execute the target backup task based on the task execution plan in the target scheduling period. Accordingly, after S306 described above, the data backup method for a database cluster according to the embodiment of the present application may further include: the following processes are repeatedly executed until the execution result of the target backup task indicates that the target backup task is successfully executed:
And acquiring an execution result of the target backup task, and if the execution result indicates that the target backup task fails to be executed, scheduling the backup node to execute the target backup task based on a task execution plan in the next scheduling period of the current scheduling period.
For example, taking the time length of each scheduling period as 1 hour as an example, if the current time is 2022, 2, 1, 8, 05 am, and the backup start time indicated by the task execution plan is 23, 05, then it may be determined that the target scheduling period of the target backup task is 11 th period (i.e. 22:00-23:00), and then the scheduling platform may schedule the backup node to execute the target backup task in the time period of 2022, 2, 1, 20:00-23:00. After executing the target backup task, the backup node returns the execution result of the target backup task to the scheduling platform, and if the execution result of the target backup task indicates that the target backup task fails to execute, the scheduling platform will perform the following steps: the backup node is scheduled again to execute the target backup task within the time period of 00-24:00; if the execution result of the target backup task indicates that the target backup task is successfully executed, ending executing the target backup task, otherwise, repeating the processing from 0:00-1:00 of the next day until the execution result of the target backup task indicates that the target backup task is successfully executed.
Because the backup node locally generates some intermediate file data after executing the target backup task, in order to avoid the influence of the intermediate file data on the execution process of the target backup task in the next scheduling period, further, as shown in fig. 4, if the backup node fails to execute the target backup task in the target scheduling period, the scheduling platform further controls the backup node to clean the local intermediate file data.
It should be noted that, in practical applications, the number of target database clusters may be plural, in this case, the data backup method of the database cluster described above may be executed for each target database cluster, so as to implement data backup for the target database cluster.
According to the data backup method of the database cluster, provided by the embodiment of the application, the running state information and the load capacity information of the nodes are considered to reflect the carrying capacity of the nodes to the backup task, and the backup node for executing the backup task is determined from the available service nodes of the database cluster based on the running state information and the load capacity information of the available service nodes in the database cluster, so that the manually appointed backup node is replaced, the labor force is saved, the backup process is simplified, the backup efficiency is improved, the backup node can be ensured to normally run, and the problems that the backup failure or the production operation of the database cluster is influenced due to abnormal conditions such as incapacity of service, overlarge load or insufficient resources of the backup node are avoided; the task execution plan of the target backup task is determined based on the task related information of the backup task currently being executed, and the backup node is further scheduled to execute the target backup task according to the task execution plan, so that not only can the labor force be saved, the backup process be simplified and the backup efficiency be improved, but also the problems of large consumption of processing resources, increase of processing time and the like caused by excessive backup tasks executed simultaneously can be avoided, and a guarantee is provided for the generation and the operation of the backup task and the database cluster.
On the basis of the above embodiment, the data backup method of the database cluster according to the embodiment of the present application may further include data recovery of the database cluster. Specifically, after S306, the data backup method of the database cluster according to the embodiment of the present application may further include:
S308, receiving a recovery request of the target database cluster.
Wherein, the recovery request carries a recovery time point. The restore request is for requesting that the target database cluster be restored to the state at the restore point in time.
S310, performing recovery processing on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool and the backup log of the historical backup task corresponding to the historical backup set.
The historical backup set is a backup set obtained by executing the corresponding historical backup task, and the backup log of the historical backup task comprises backup start-stop time information of the historical backup task. The backup start-stop time information of the historical backup task includes a backup start time (i.e., a time when the backup task starts to be executed) and a backup end time (i.e., a time when the backup task stops to be executed) of the historical backup task. For example, when the backup node starts to execute the historical backup task 1 from 2:00 in early morning and stops executing the historical backup task 1 at 4:00 in early morning, the backup start time of the historical backup task 1 is 2:00 in early morning, and the backup end time is 4:00 in early morning.
Based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool and the backup log of the historical backup task corresponding to the historical backup set, the data required for recovering the target database cluster to the recovery time point can be determined from the historical backup set, and then the recovery processing is carried out on the target database cluster based on the determined data.
In an alternative implementation manner, to ensure that the target database cluster can accurately recover to the state corresponding to the recovery time point, S310 may include:
s311, selecting a historical backup set meeting a first preset backup condition from the backup pool as a first candidate backup set based on the recovery time point and the backup log of the historical backup task corresponding to the historical backup set.
The first preset backup condition includes that the backup ending time point is located before the recovery time point and is nearest to the recovery time point.
S312, taking the backup ending time point of the first candidate backup set as a reference time point, and selecting a historical backup set with the backup starting and ending time information meeting a second preset backup condition from the backup pool as a second candidate backup set based on the backup log of the historical backup task corresponding to the historical backup set.
The second preset backup condition includes that the backup start time point is located before the reference time point and the backup end time point is located after the reference time point.
S313, based on the backup logs of the historical backup tasks corresponding to the historical backup sets, selecting the historical backup sets with the backup start-stop time information meeting the third preset backup conditions from the backup pool as third candidate backup sets.
The third preset backup condition includes that a backup start time point is located before a recovery time point and a backup end time point is located after the recovery time point.
And S314, generating a target recovery backup set based on the first candidate backup set, the second candidate backup set and the third candidate backup set, and carrying out recovery processing on the target database cluster based on the target recovery backup set.
Specifically, the target restore backup set may be generated based on the first candidate backup set, the data corresponding to the first target time period in the second candidate backup set, and the data corresponding to the second target time period in the third candidate backup set. Wherein the first target period is a period from a reference time point to a backup end time point of the second candidate backup set, and the second target period is a period from a backup start time point to a restoration time point of the third candidate backup set.
For example, as shown in fig. 5, assuming that the recovery time t1 is 15:00 pm, the historical backup sets corresponding to the target database clusters stored in the backup pool include the historical backup sets 1 to 5, and the backup start and end time points of the historical backup sets are 4:00 to 5:00 am, 5:00 to 6:00 am, 6:00 to 8:00 am, 7 am: 00-9:00 and 14:00-16:00 pm on the same day, the historical backup set 3 can be determined as a first candidate backup set, the backup ending time of the historical backup set 3 is determined as a reference time point t_ backupset, the historical backup set 4 is further selected from the backup pool to be used as a second candidate backup set, and the historical backup set 5 is selected to be used as a third candidate backup set; and then, generating a target recovery backup set based on the total data of the historical backup set 3, the data corresponding to the first target time period in the historical backup set 4 and the data corresponding to the second target time period in the historical backup set 5, and carrying out recovery processing on the target database cluster based on the target recovery backup set.
It can be understood that in the above implementation manner, the historical backup set that ends the last backup from the recovery time point and the backup data generated in the period between the backup end time point and the recovery time point of the historical backup set are considered, so that the comprehensiveness of the backup data for recovering the target database cluster can be ensured, and the target database cluster can be ensured to be accurately recovered to the state corresponding to the recovery time point.
Further, in order to save the occupation of the backup pool resources, save the storage space of the backup pool, and simultaneously ensure the effective and efficient recovery of the target database cluster, before S310, the data backup method of the database cluster according to the embodiment of the present application may further include: according to a preset polling period, polling a historical backup set of a target database cluster stored in a backup pool, and executing the following operation if one historical backup set is polled: based on the backup logs of the history backup tasks corresponding to the currently polled history backup sets and the preset storage strategies corresponding to the target database clusters, determining whether the history backup sets meet preset cleaning conditions, and if the history backup sets meet the preset cleaning conditions, deleting the history backup sets from the backup pool.
The preset storage policy corresponding to the target database cluster may specifically include a retention period, a storage manner, and the like of the backup set for the target database cluster. In practical application, the preset polling period, the preset storage strategy corresponding to the target database cluster and the preset cleaning condition can be set according to practical needs, which is not limited by the embodiment of the application.
For example, the preset polling period is once for each month, and the preset retention period corresponding to the target database cluster is a sparse retention policy, that is, all backup sets of about 1 month are retained, 1 backup set is retained every 2 days of about 1-3 months, 1 backup set is retained every 7 days of about 3-6 months, 1 backup set is retained every 30 days of about 6-12 months, and so on. If the current polling period is 11 months 1-12 months 1, based on the backup logs of the historical backup tasks corresponding to the current polled historical backup set, determining that the backup end time of the historical backup set is 10 months 20 days and belongs to the backup set of nearly 1 month, and retaining the historical backup set; if the backup end time of the historical backup set is determined to be 9 months and 29 days, the historical backup set belongs to the backup set generated in the last 1-3 months, and if the backup end time is far from the current time compared with other historical backup sets generated in the last 9 months and 28 days-9 months and 30 days, the historical backup set is deleted.
It can be understood that, based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool and the backup log of the historical backup task corresponding to the historical backup set, the recovery process is performed on the target database cluster, so that the target database cluster can be accurately recovered to the state corresponding to the recovery time point.
In addition, corresponding to the data backup method of the database cluster shown in fig. 3, the embodiment of the application further provides a data backup device of the database cluster, which can be applied to the scheduling platform 2 in the implementation environment. Fig. 6 is a schematic structural diagram of a data backup apparatus 600 of a database cluster according to an embodiment of the present application, where the apparatus includes:
A first determining module 610, configured to determine, from among the serviceable nodes of the target database cluster, a backup node for performing a target backup task, based on operation status information of the serviceable nodes in the target database cluster, where the target backup task is used to backup target data of the target database cluster;
a second determining module 620, configured to determine a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and the scheduling module 630 is configured to schedule the backup node to execute the target backup task based on the task execution plan.
According to the data backup device of the database cluster, provided by the embodiment of the application, the running state information and the load capacity information of the nodes are considered to reflect the carrying capacity of the nodes to the backup task, and the backup node for executing the backup task is determined from the available service nodes of the database cluster based on the running state information and the load capacity information of the available service nodes in the database cluster, so that the manually appointed backup node is replaced, the labor force is saved, the backup process is simplified, the backup efficiency is improved, the backup node can be ensured to normally run, and the problems that the backup failure or the production operation of the database cluster is influenced due to abnormal conditions such as incapacity of service, overlarge load or insufficient resources of the backup node are avoided; the task execution plan of the target backup task is determined based on the task related information of the backup task currently being executed, and the backup node is further scheduled to execute the target backup task according to the task execution plan, so that not only can the labor force be saved, the backup process be simplified and the backup efficiency be improved, but also the problems of large consumption of processing resources, increase of processing time and the like caused by excessive backup tasks executed simultaneously can be avoided, and a guarantee is provided for the generation and the operation of the backup task and the database cluster.
Optionally, the running state information of the node includes a role of the node, and the load capacity information of the node includes a master-slave copy delay duration of the node;
The first determining module includes:
and the backup node determining submodule is used for determining the serviceable node as the backup node for each serviceable node in the target database cluster if the role of the serviceable node indicates that the serviceable node does not belong to a write node and the master-slave copy delay time length of the serviceable node is smaller than a preset time length threshold value.
Optionally, the scheduling module includes:
The generation sub-module is used for generating a target backup program for executing the target backup task at least based on the task execution plan;
A sending sub-module, configured to send the target backup program to the backup node;
And the triggering sub-module is used for triggering the backup node to execute the target backup program.
Optionally, the generating submodule is specifically configured to:
acquiring identification information of a backup pool;
And generating the target backup program based on the identification information of the backup pool, the task execution plan and a preset backup program template, wherein the target backup program acquires target data of the target database cluster in the running process to generate a backup set corresponding to the target backup task and stores the generated backup set into the backup pool.
Optionally, the apparatus further comprises:
the receiving module is used for receiving a recovery request of the target database cluster after the scheduling module schedules the backup node to execute the target backup task based on the task execution plan, wherein the recovery request carries a recovery time point;
And the recovery module is used for carrying out recovery processing on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster and the backup log of the historical backup task corresponding to the historical backup set, wherein the historical backup set is a backup set obtained by executing the corresponding historical backup task, and the backup log of the historical backup task comprises backup start-stop time information of the historical backup task.
Optionally, the recovery module includes:
a first backup set determining submodule, configured to select, from the backup pool, a historical backup set that meets a first preset backup condition as a first candidate backup set based on the recovery time point and a backup log of a historical backup task corresponding to the historical backup set, where the first preset backup condition includes that a backup end time point is located before the recovery time point and is closest to the recovery time point;
A second backup set determining sub-module, configured to take a backup end time point of the first candidate backup set as a reference time point, and select, from the backup pool, a historical backup set with backup start-stop time information meeting a second preset backup condition as a second candidate backup set based on a backup log of a historical backup task corresponding to the historical backup set, where the second preset backup condition includes that a backup start time point is located before the reference time point and a backup end time point is located after the reference time point;
A third backup set determining submodule, configured to select, based on a backup log of a historical backup task corresponding to the historical backup set, a historical backup set whose backup start-stop time information satisfies a third preset backup condition from the backup pool as a third candidate backup set, where the third preset backup condition includes that a backup start time point is located before the recovery time point and a backup end time point is located after the recovery time point;
and the restoration sub-module is used for generating a target restoration backup set based on the first candidate backup set, the second candidate backup set and the third candidate backup set, and carrying out restoration processing on the target database cluster based on the target restoration backup set.
Optionally, before the recovery module performs recovery processing on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool, and the backup log of the historical backup task corresponding to the historical backup set, polling the historical backup set of the target database cluster in the backup pool according to a preset polling period, and executing the following operations if one historical backup set is polled:
determining whether the currently polled historical backup set meets a preset cleaning condition or not based on the backup log of the historical backup task corresponding to the currently polled historical backup set and a preset storage strategy corresponding to the target database cluster;
and if the currently polled historical backup set meets the preset cleaning condition, deleting the currently polled historical backup set from the backup pool.
Optionally, the scheduling module is specifically configured to determine, based on the task execution plan, a target scheduling period that matches the target backup task, and schedule, in the target scheduling period, the backup node to execute the target backup task based on the task execution plan, and repeatedly execute the following processes until an execution result of the target backup task indicates that the target backup task is executed successfully: and if the execution result indicates that the execution of the target backup task fails, scheduling the backup node to execute the target backup task based on the task execution plan in the next scheduling period of the current scheduling period.
Obviously, the data backup device of the database cluster according to the embodiment of the present application may be used as an execution body of the data backup method of the database cluster shown in fig. 3, so that the function of the data backup method of the database cluster implemented in fig. 3 can be implemented. Since the principle is the same, the description is not repeated here.
The embodiment of the application also provides a data backup system of the database cluster, which comprises a dispatching platform (such as the dispatching platform 2 in the implementation environment shown in fig. 1) and the database cluster (such as the database cluster 1 in the implementation environment shown in fig. 1).
The scheduling platform is used for determining a backup node for executing a target backup task from the serviceable nodes of the database cluster based on the running state information of the serviceable nodes in the database cluster, wherein the target backup task is used for backing up target data of the database cluster; determining a task execution plan of a target backup task based on task related information of the backup task currently being executed, and sending a scheduling request to a backup node based on the task execution plan;
and the backup node is used for responding to the scheduling request and executing the target backup task.
Optionally, the running state information of the node includes a role of the node, and the load capacity information of the node includes a master-slave copy delay duration of the node;
The scheduling platform determines a backup node for executing a target backup task from the serviceable nodes of a target database cluster based on the operation state information and the load capacity information of the serviceable nodes in the target database cluster, and the scheduling platform comprises the following steps:
And aiming at each serviceable node in the target database cluster, if the role of the serviceable node indicates that the serviceable node does not belong to a write node and the master-slave copy delay time length of the serviceable node is smaller than a preset time length threshold, determining the serviceable node as the backup node.
Optionally, the scheduling platform schedules the backup node to execute the target backup task based on the task execution plan, including:
generating a target backup program for executing the target backup task based at least on the task execution plan;
Transmitting the target backup program to the backup node;
triggering the backup node to execute the target backup program.
Optionally, the scheduling platform is further configured to, after the task execution plan is based, schedule the backup node to execute the target backup task, receive a recovery request of the target database cluster, where the recovery request carries a recovery time point, and perform recovery processing on the target database cluster based on the recovery time point, a historical backup set corresponding to the target database cluster stored in the backup pool, and a backup log of the historical backup task corresponding to the historical backup set, where the historical backup set is a backup set obtained by executing the corresponding historical backup task, and the backup log of the historical backup task includes backup start-stop time information of the historical backup task.
Optionally, the scheduling platform performs recovery processing on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool, and a backup log of a historical backup task corresponding to the historical backup set, including:
Selecting a historical backup set meeting a first preset backup condition from the backup pool as a first candidate backup set based on the recovery time point and a backup log of a historical backup task corresponding to the historical backup set, wherein the first preset backup condition comprises that a backup ending time point is positioned before the recovery time point and is nearest to the recovery time point;
Taking a backup ending time point of the first candidate backup set as a reference time point, and selecting a historical backup set with backup start-stop time information meeting a second preset backup condition from the backup pool as a second candidate backup set based on a backup log of a historical backup task corresponding to the historical backup set, wherein the second preset backup condition comprises that the backup starting time point is positioned before the reference time point and the backup ending time point is positioned after the reference time point;
Selecting a historical backup set with backup start-stop time information meeting a third preset backup condition from the backup pool as a third candidate backup set based on a backup log of a historical backup task corresponding to the historical backup set, wherein the third preset backup condition comprises that a backup start time point is positioned before the recovery time point and a backup end time point is positioned after the recovery time point;
And generating a target recovery backup set based on the first candidate backup set, the second candidate backup set and the third candidate backup set, and carrying out recovery processing on the target database cluster based on the target recovery backup set.
Optionally, the scheduling platform is further configured to, before performing a recovery process on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool, and the backup log of the historical backup task corresponding to the historical backup set, poll the historical backup set of the target database cluster in the backup pool according to a preset polling period, and if one historical backup set is polled, execute the following operations:
determining whether the currently polled historical backup set meets a preset cleaning condition or not based on the backup log of the historical backup task corresponding to the currently polled historical backup set and a preset storage strategy corresponding to the target database cluster;
and if the currently polled historical backup set meets the preset cleaning condition, deleting the currently polled historical backup set from the backup pool.
Optionally, the scheduling platform schedules the backup node to execute the target backup task based on the task execution plan, including:
Determining a target scheduling period matched with the target backup task based on the task execution plan;
scheduling the backup node to execute the target backup task based on the task execution plan in the target scheduling period;
In the target scheduling period, after the backup node is scheduled to execute the target backup task based on the task execution plan, the method further comprises:
repeating the following processes until the execution result of the target backup task indicates that the target backup task is successfully executed:
acquiring an execution result of the target backup task;
And if the execution result indicates that the target backup task fails to be executed, the backup node is scheduled to execute the target backup task based on the task execution plan in the next scheduling period of the current scheduling period.
Optionally, the backup node is further configured to:
in the process of executing the target backup task, generating a backup log corresponding to the target backup task based on the execution process information of the target backup task;
and after the target backup task is executed, returning the execution process information and the execution result of the target backup task to the scheduling platform.
Optionally, the system further comprises a backup pool;
And the backup node is used for acquiring the target data of the target database cluster, generating a backup set corresponding to the target backup task based on the acquired target data, and storing the generated backup set into the backup pool.
Optionally, the backup node is further configured to:
after a backup set corresponding to the target backup task is generated based on the obtained target data, carrying out preset processing on the backup set, and storing the backup set after the preset processing into a backup pool, wherein the preset processing comprises compression processing and/or encryption processing.
According to the data backup system of the database cluster, provided by the embodiment of the application, the running state information and the load capacity information of the nodes are considered to reflect the carrying capacity of the nodes to the backup task, and the backup node for executing the backup task is determined from the available service nodes of the database cluster based on the running state information and the load capacity information of the available service nodes in the database cluster, so that the manually appointed backup node is replaced, the labor force is saved, the backup process is simplified, the backup efficiency is improved, the backup node can be ensured to normally run, and the problems that the backup failure or the production operation of the database cluster is influenced due to abnormal conditions such as incapacity of service, overlarge load or insufficient resources of the backup node are avoided; the task execution plan of the target backup task is determined based on the task related information of the backup task currently being executed, and the backup node is further scheduled to execute the target backup task according to the task execution plan, so that not only can the labor force be saved, the backup process be simplified and the backup efficiency be improved, but also the problems of large consumption of processing resources, increase of processing time and the like caused by excessive backup tasks executed simultaneously can be avoided, and a guarantee is provided for the generation and the operation of the backup task and the database cluster.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present specification. Referring to fig. 7, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 7, but not only one bus or type of bus.
And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory to the memory and then runs, and forms a data backup device of the database cluster on a logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:
determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the running state information and the load capacity information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster;
determining a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and scheduling the backup node to execute the target backup task based on the task execution plan.
The method performed by the data backup device of the database cluster disclosed in the embodiment shown in fig. 3 of the present specification may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
It should be understood that the electronic device according to the embodiment of the present application may implement the functions of the data backup device of the database cluster in the embodiment shown in fig. 3. Because the principles are the same, the embodiments of the present application are not described herein.
Of course, in addition to the software implementation, the electronic device in this specification does not exclude other implementations, such as a logic device or a combination of software and hardware, that is, the execution subject of the following process is not limited to each logic unit, but may also be hardware or a logic device.
The embodiments of the present application also provide a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device comprising a plurality of application programs, enable the portable electronic device to perform the method of the embodiment shown in fig. 3, and in particular to perform the operations of:
determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the running state information and the load capacity information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster;
determining a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and scheduling the backup node to execute the target backup task based on the task execution plan.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In summary, the foregoing description is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the protection scope of the present specification.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
Claims (15)
1. A method for backing up data of a database cluster, comprising:
Determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the running state information and the load capacity information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster; the running state information of the node comprises a role of the node, the load capacity information of the node comprises master-slave copy delay time length of the node, and the backup node comprises a serviceable node of which the role does not belong to a writing node and the master-slave copy delay time length is smaller than a preset time length threshold;
determining a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and scheduling the backup node to execute the target backup task based on the task execution plan.
2. The method of claim 1, wherein determining a backup node for performing a target backup task from among the serviceable nodes of the target database cluster based on the operational status information and the load capacity information of the serviceable nodes in the target database cluster comprises:
And aiming at each serviceable node in the target database cluster, if the role of the serviceable node indicates that the serviceable node does not belong to a write node and the master-slave copy delay time length of the serviceable node is smaller than a preset time length threshold, determining the serviceable node as the backup node.
3. The method of claim 1, wherein scheduling the backup node to perform the target backup task based on the task execution plan comprises:
generating a target backup program for executing the target backup task based at least on the task execution plan;
Transmitting the target backup program to the backup node;
triggering the backup node to execute the target backup program.
4. A method according to claim 3, wherein generating a target backup program for performing the target backup task based at least on the task execution plan comprises:
acquiring identification information of a backup pool;
And generating the target backup program based on the identification information of the backup pool, the task execution plan and a preset backup program template, wherein the target backup program acquires target data of the target database cluster in the running process to generate a backup set corresponding to the target backup task and stores the generated backup set into the backup pool.
5. The method of claim 4, wherein after scheduling the backup node to perform the target backup task based on the task execution plan, the method further comprises:
receiving a recovery request of the target database cluster, wherein the recovery request carries a recovery time point;
And carrying out recovery processing on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster and the backup log of the historical backup task corresponding to the historical backup set, wherein the historical backup set is obtained by executing the corresponding historical backup task, and the backup log of the historical backup task comprises backup start-stop time information of the historical backup task.
6. The method according to claim 5, wherein the restoring the target database cluster based on the restoration time point, the historical backup set corresponding to the target database cluster stored in the backup pool, and the backup log of the historical backup task corresponding to the historical backup set includes:
Selecting a historical backup set meeting a first preset backup condition from the backup pool as a first candidate backup set based on the recovery time point and a backup log of a historical backup task corresponding to the historical backup set, wherein the first preset backup condition comprises that a backup ending time point is positioned before the recovery time point and is nearest to the recovery time point;
Taking a backup ending time point of the first candidate backup set as a reference time point, and selecting a historical backup set with backup start-stop time information meeting a second preset backup condition from the backup pool as a second candidate backup set based on a backup log of a historical backup task corresponding to the historical backup set, wherein the second preset backup condition comprises that the backup starting time point is positioned before the reference time point and the backup ending time point is positioned after the reference time point;
Selecting a historical backup set with backup start-stop time information meeting a third preset backup condition from the backup pool as a third candidate backup set based on a backup log of a historical backup task corresponding to the historical backup set, wherein the third preset backup condition comprises that a backup start time point is positioned before the recovery time point and a backup end time point is positioned after the recovery time point;
And generating a target recovery backup set based on the first candidate backup set, the second candidate backup set and the third candidate backup set, and carrying out recovery processing on the target database cluster based on the target recovery backup set.
7. The method of claim 5, wherein prior to performing recovery processing on the target database cluster based on the recovery time point, the historical backup set corresponding to the target database cluster stored in the backup pool, and the backup log of the historical backup task corresponding to the historical backup set, the method further comprises:
According to a preset polling period, polling a historical backup set of the target database cluster in the backup pool, and executing the following operations if one historical backup set is polled:
determining whether the currently polled historical backup set meets a preset cleaning condition or not based on the backup log of the historical backup task corresponding to the currently polled historical backup set and a preset storage strategy corresponding to the target database cluster;
and if the currently polled historical backup set meets the preset cleaning condition, deleting the currently polled historical backup set from the backup pool.
8. The method of claim 1, wherein scheduling the backup node to perform the target backup task based on the task execution plan comprises:
Determining a target scheduling period matched with the target backup task based on the task execution plan;
scheduling the backup node to execute the target backup task based on the task execution plan in the target scheduling period;
In the target scheduling period, after the backup node is scheduled to execute the target backup task based on the task execution plan, the method further comprises:
repeating the following processes until the execution result of the target backup task indicates that the target backup task is successfully executed:
acquiring an execution result of the target backup task;
And if the execution result indicates that the target backup task fails to be executed, the backup node is scheduled to execute the target backup task based on the task execution plan in the next scheduling period of the current scheduling period.
9. A data backup system for a database cluster, comprising: scheduling a platform and a database cluster;
the scheduling platform is used for determining a backup node for executing a target backup task from the serviceable nodes of the database cluster based on the operation state information of the serviceable nodes in the database cluster, wherein the target backup task is used for backing up target data of the database cluster; determining a task execution plan of the target backup task based on task related information of the backup task currently being executed, and sending a scheduling request to the backup node based on the task execution plan; the running state information of the node comprises a role of the node, the load capacity information of the node comprises master-slave copy delay time length of the node, and the backup node comprises a serviceable node of which the role does not belong to a writing node and the master-slave copy delay time length is smaller than a preset time length threshold;
And the backup node is used for responding to the scheduling request and executing the target backup task.
10. The system of claim 9, wherein the backup node is further configured to:
In the process of executing the target backup task, generating a backup log corresponding to the target backup task based on the execution process information of the target backup task;
And after the target backup task is executed, returning the execution process information and the execution result of the target backup task to the dispatching platform.
11. The system of claim 9, wherein the system further comprises a backup pool;
The backup node is configured to obtain target data of the target database cluster, generate a backup set corresponding to the target backup task based on the obtained target data, and store the generated backup set in the backup pool.
12. The system of claim 11, wherein the backup node is further configured to:
After a backup set corresponding to the target backup task is generated based on the obtained target data, carrying out preset processing on the backup set, and storing the backup set after the preset processing into the backup pool, wherein the preset processing comprises compression processing and/or encryption processing.
13. A data backup apparatus for a database cluster, comprising:
The first determining module is used for determining a backup node for executing a target backup task from the serviceable nodes of the target database cluster based on the operation state information of the serviceable nodes in the target database cluster, wherein the target backup task is used for backing up target data of the target database cluster; the running state information of the node comprises a role of the node, the load capacity information of the node comprises master-slave copy delay time length of the node, and the backup node comprises a serviceable node of which the role does not belong to a writing node and the master-slave copy delay time length is smaller than a preset time length threshold;
The second determining module is used for determining a task execution plan of the target backup task based on task related information of the backup task currently being executed;
and the scheduling module is used for scheduling the backup node to execute the target backup task based on the task execution plan.
14. An electronic device, comprising:
A processor;
a memory for storing the processor-executable instructions;
Wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 8.
15. A computer readable storage medium, characterized in that instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210499026.XA CN114780296B (en) | 2022-05-09 | 2022-05-09 | Data backup method, device and system of database cluster |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210499026.XA CN114780296B (en) | 2022-05-09 | 2022-05-09 | Data backup method, device and system of database cluster |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114780296A CN114780296A (en) | 2022-07-22 |
CN114780296B true CN114780296B (en) | 2024-06-25 |
Family
ID=82436223
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210499026.XA Active CN114780296B (en) | 2022-05-09 | 2022-05-09 | Data backup method, device and system of database cluster |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114780296B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116795602B (en) * | 2023-08-17 | 2023-11-10 | 北京星云医学检验实验室有限公司 | Method and device for constructing biological sample information database |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107231437A (en) * | 2017-07-18 | 2017-10-03 | 郑州云海信息技术有限公司 | A kind of task backup management method and device |
CN107329853A (en) * | 2017-06-13 | 2017-11-07 | 上海微烛信息技术有限公司 | Backup method, standby system and the electronic equipment of data-base cluster |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9372854B2 (en) * | 2010-11-08 | 2016-06-21 | Hewlett Packard Enterprise Development Lp | Load balancing backup jobs in a virtualized storage system having a plurality of physical nodes |
CN103019889A (en) * | 2012-12-21 | 2013-04-03 | 曙光信息产业(北京)有限公司 | Distributed file system and failure processing method thereof |
CN106528324A (en) * | 2015-09-10 | 2017-03-22 | 华为技术有限公司 | Fault recovery method and apparatus |
CN107145406A (en) * | 2017-05-14 | 2017-09-08 | 四川盛世天成信息技术有限公司 | A kind of disaster-tolerant backup method and system based on Clustering |
CN107528917B (en) * | 2017-09-13 | 2020-12-15 | 马上消费金融股份有限公司 | File storage method and device |
CN108459926B (en) * | 2018-02-26 | 2020-05-15 | 新华三技术有限公司 | Data remote backup method and device and computer readable medium |
CN113297231A (en) * | 2020-07-28 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Database processing method and device |
CN112190924A (en) * | 2020-12-04 | 2021-01-08 | 腾讯科技(深圳)有限公司 | Data disaster tolerance method, device and computer readable medium |
-
2022
- 2022-05-09 CN CN202210499026.XA patent/CN114780296B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107329853A (en) * | 2017-06-13 | 2017-11-07 | 上海微烛信息技术有限公司 | Backup method, standby system and the electronic equipment of data-base cluster |
CN107231437A (en) * | 2017-07-18 | 2017-10-03 | 郑州云海信息技术有限公司 | A kind of task backup management method and device |
Also Published As
Publication number | Publication date |
---|---|
CN114780296A (en) | 2022-07-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3399692B1 (en) | Method and apparatus for upgrading distributed storage system | |
CN111143133B (en) | Virtual machine backup method and backup virtual machine recovery method | |
CN103152390B (en) | The node configuration method of distributed memory system, device, node and system | |
CN107506266B (en) | Data recovery method and system | |
CN111966449B (en) | Virtual machine backup management method, system, terminal and storage medium | |
CN114780296B (en) | Data backup method, device and system of database cluster | |
US7315959B2 (en) | Real-time remote backup system and related method | |
CN112527746A (en) | Log storage method, device, equipment and storage medium | |
US10261718B1 (en) | Adaptively controlling data copy operations that copy data in an electronic environment | |
CN111966531B (en) | Data snapshot method and device, computer equipment and storage medium | |
CN116483546B (en) | Distributed training task scheduling method, device, equipment and storage medium | |
CN111625402A (en) | Data recovery method and device, electronic equipment and computer readable storage medium | |
CN112527561B (en) | Data backup method and device based on Internet of things cloud storage | |
CN113302593A (en) | Task processing method, device and system, electronic equipment and storage medium | |
CN113900856A (en) | Data processing method and related equipment | |
CN112148420B (en) | Abnormal task processing method based on container technology, server and cloud platform | |
CN112286727A (en) | Space-time isolation domain fast recovery method and system based on incremental snapshot | |
CN110618863A (en) | Operation scheduling method based on Raft algorithm | |
CN113452726A (en) | Data transmission method, terminal and computer readable storage medium | |
CN110908821A (en) | Method, device, equipment and storage medium for task failure management | |
CN112463514A (en) | Monitoring method and device for distributed cache cluster | |
CN117539639B (en) | Video memory resource scheduling method, device, system, storage medium and electronic equipment | |
CN110365775B (en) | Service data uploading method and device and storage medium | |
CN110351386B (en) | Increment synchronization method and device between different copies | |
CN118069416A (en) | Data processing method and device, storage medium and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |