CN107526838B - Method and device for database cluster capacity expansion - Google Patents
Method and device for database cluster capacity expansion Download PDFInfo
- Publication number
- CN107526838B CN107526838B CN201710805786.8A CN201710805786A CN107526838B CN 107526838 B CN107526838 B CN 107526838B CN 201710805786 A CN201710805786 A CN 201710805786A CN 107526838 B CN107526838 B CN 107526838B
- Authority
- CN
- China
- Prior art keywords
- triple
- task
- instance
- triple task
- execution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for database cluster expansion, and relates to the technical field of computers. One embodiment of the method comprises: creating a triple task according to a received capacity expansion request of the database cluster; and executing the triple task to complete the expansion of the database cluster. According to the embodiment, the triple task is created through the received capacity expansion request, the capacity expansion of the database cluster is completed, the user does not need to conduct additional control on the capacity expansion, the automatic capacity expansion effect of the database cluster can be achieved by directly submitting the cluster capacity expansion request, and the performance of compatible cascade capacity expansion or other complex capacity expansion is achieved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for database cluster expansion.
Background
A database cluster uses at least two or more database servers to form a virtual single database logical image, such as a single database system, to provide transparent data services to clients. In practical application, the existing database cluster often has a situation that the business requirements cannot be met, and in such a situation, a new database instance needs to be added to the existing cluster, that is, the database cluster is expanded.
In the prior art, when a database cluster is expanded, all expansion operations in which a target instance in the cluster is the same data source instance are generally submitted and executed as a task. The mount master library instance of the target instance may be the same database instance or may be different database instances. And if the capacity of a plurality of target instances starts to be expanded simultaneously, the mounted main library instances of the target instances are required to be the same database instance.
In the process of implementing the present invention, the inventor finds that the method for database cluster capacity expansion in the prior art has at least the following problems: firstly, the capacity expansion of a database cluster is carried out by adopting the prior art, and all capacity expansion conditions need to be considered, so that on one hand, the design is complicated, and on the other hand, along with the service upgrade, when a new capacity expansion task is added, the code is very difficult to upgrade in the previous overall design; secondly, the existing technology is adopted to carry out database cluster capacity expansion, if a certain step in the capacity expansion task fails to be executed, the step which is executed wrongly cannot be directly recovered, and the whole capacity expansion task fails; third, the prior art does not support cascade expansion or other complicated expansion.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for database cluster capacity expansion, which can create a triple task through a received capacity expansion request, complete capacity expansion of a database cluster, and achieve an effect of automatic capacity expansion of the database cluster by directly submitting a cluster capacity expansion request without additional control of a user on capacity expansion.
To achieve the above object, according to an aspect of the embodiments of the present invention, a method for database cluster expansion is provided.
The method for expanding the database cluster in the embodiment of the invention comprises the following steps: creating a triple task according to a received capacity expansion request of a database cluster, wherein the triple task comprises the following three elements of information: a target instance, a data source instance of the target instance, and a mount master library instance of the target instance; and executing the triple task to complete the expansion of the database cluster.
Optionally, creating the triple task according to the received capacity expansion request of the database cluster includes: and creating one or more triple tasks according to one or more target instances contained in the capacity expansion request.
Optionally, creating the triple task further comprises: saving at least one of the following information in the triple task: the method comprises the steps of executing the triple tasks currently, executing states of the triple tasks, adding time of the triple tasks and updating time of the triple tasks.
Optionally, the execution state of the triple task includes at least one of: non-executing, successful execution, failed execution; and when the triple task is executed, selecting the triple task with the execution state of non-execution, and starting to execute from the current execution step of the triple task with the execution state of non-execution.
Optionally, the method further comprises: if the triple task fails to execute, updating the execution state of the triple task to be execution failure; and after a new triple task identical to the triple task failed to be executed is created according to the received capacity expansion request, the new triple task is not stored, and the execution state of the triple task failed to be executed is modified into non-execution state.
Optionally, creating the triple task further comprises: and if the target instance has the brother node, storing the brother node information of the target instance in the triple task corresponding to the target instance. Wherein the sibling node refers to other target instances having the same data source instance as the target instance.
Optionally, before executing the triplet task, the method further comprises: and judging whether the sibling node of the target example stored in the triple task is the target example of other executed triple tasks, and if not, executing the triple task.
Optionally, in the process of executing the triple task, the method further includes: after generating a backup file on a target instance of the triple task, judging whether the backup file exists on a brother node of the target instance, if not, copying the backup file to the brother node of the target instance; and before removing the backup file from the target instance of the triple task, judging whether the copying is finished, if so, removing the backup file, and if not, continuing to wait until the copying is finished.
Optionally, the method further comprises: before the triple task is executed, whether a data source instance and/or a mounting main library instance of the triple task is/are a target instance of other uncompleted triple tasks is judged, and if not, the triple task is executed.
To achieve the above object, according to another aspect of the embodiments of the present invention, an apparatus for database cluster expansion is provided.
The device for expanding the database cluster in the embodiment of the invention comprises the following components: the creation module is configured to create a triple task according to a received capacity expansion request of a database cluster, where the triple task includes the following three elements: a target instance, a data source instance of the target instance, and a mount master library instance of the target instance; and the execution module is used for executing the triple tasks to complete the expansion of the database cluster.
Optionally, the creating module is further configured to: and creating one or more triple tasks according to one or more target instances contained in the capacity expansion request.
Optionally, the creating module is further configured to: saving at least one of the following information in the triple task: the method comprises the steps of executing the triple tasks currently, executing states of the triple tasks, adding time of the triple tasks and updating time of the triple tasks.
Optionally, the execution state of the triple task includes at least one of: non-executing, successful execution, failed execution; and the execution module is further configured to: when the triple task is executed, selecting the triple task with the execution state of non-execution, and starting to execute from the current execution step of the triple task with the execution state of non-execution.
Optionally, the creating module is further configured to: if the triple task fails to execute, updating the execution state of the triple task to be execution failure; and after a new triple task identical to the triple task failed to be executed is created according to the received capacity expansion request, the new triple task is not stored, and the execution state of the triple task failed to be executed is modified into non-execution state.
Optionally, the creating module is further configured to: and if the target instance has the brother node, storing the brother node information of the target instance in the triple task corresponding to the target instance. Wherein the sibling node refers to other target instances having the same data source instance as the target instance.
Optionally, the execution module is further configured to: before the triple task is executed, judging whether the sibling node of the target instance stored in the triple task is the target instance of other executed triple tasks, and if not, executing the triple task.
Optionally, in the process of executing the triple task, the execution module is further configured to:
after generating a backup file on a target instance of the triple task, judging whether the backup file exists on a brother node of the target instance, if not, copying the backup file to the brother node of the target instance; and before removing the backup file from the target instance of the triple task, judging whether the copying is finished, if so, removing the backup file, and if not, continuing to wait until the copying is finished.
Optionally, the execution module is further configured to: before the triple task is executed, whether a data source instance and/or a mounting main library instance of the triple task is/are a target instance of other uncompleted triple tasks is judged, and if not, the triple task is executed.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided an electronic apparatus.
An electronic device of an embodiment of the present invention includes: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by one or more processors, the one or more processors implement the method for expanding the database cluster of the embodiment of the invention.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided a computer-readable medium.
A computer-readable medium of an embodiment of the present invention stores thereon a computer program, and when the computer program is executed by a processor, the method for expanding a database cluster of an embodiment of the present invention is implemented.
One embodiment of the above invention has the following advantages or benefits: the automatic capacity expansion of the database cluster is completed by creating the triple task according to the received capacity expansion request and executing the triple task, so that the automatic capacity expansion of the database cluster can be achieved only by directly submitting the cluster capacity expansion request without additionally controlling the capacity expansion by a user (for example, complicated capacity expansion task design needs to be carried out by considering all capacity expansion conditions, and the designed capacity expansion task needs to be integrally modified when the capacity expansion requirement is changed); in the embodiment of the invention, the triple task is used as the minimum unit of cluster expansion, and one or more triple tasks are created according to one or more target examples contained in the expansion request, so that the complexity of the design of the expansion task in the prior art can be reduced, and the cascade expansion or other complex expansion conditions can be compatible; by saving the execution state of the triple task, the triple task failed to be executed can be submitted again according to the execution state; in addition, by saving the current execution steps of the triple tasks, when a certain triple task fails, the triple task can be directly executed from the failure step after the failure reason is solved, and other tasks in the cluster are not influenced; in addition, by storing the current execution step of the triple task, the current state of the triple task can be checked at any time, and the progress of the triple task is tracked; by storing brother node information of the triple tasks, backup files can be shared with the brother nodes; before the triple tasks are executed, whether brother nodes of the target examples stored in the triple tasks are the target examples of other triple tasks being executed is verified, so that the complexity of executing the related capacity expansion tasks is reduced under the condition that brother nodes exist, and the overall capacity expansion time is further shortened; before the triple tasks are executed, whether the triple tasks can be executed or not can be further judged by judging whether the data source examples and/or the mount main library examples of the triple tasks are the target examples of other uncompleted triple tasks or not, and the capacity expansion efficiency is improved.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram illustrating the main steps of a method for database cluster capacity expansion according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a process of creating a triple task according to a method for database cluster capacity expansion according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the principal architecture of a system suitable for implementing the method of database cluster capacity expansion of an embodiment of the present invention;
FIG. 4 is a schematic diagram of a main flow of a method for database cluster capacity expansion to check whether a triple task may start to be executed according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the major modules of an apparatus for database cluster capacity expansion according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 7 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The embodiment of the invention introduces the technical scheme of database cluster capacity expansion in detail, completes the capacity expansion of the database cluster by establishing the triple task through the received capacity expansion request, does not need a user to perform additional control on the capacity expansion, can achieve the effect of automatic capacity expansion of the database cluster by directly submitting the cluster capacity expansion request, and has the performance of being compatible with cascade capacity expansion or other complex capacity expansion. The technical scheme for database cluster capacity expansion of the embodiment of the invention can be suitable for various database clusters. In the following description of the specific embodiment, a MySQL database cluster is taken as an example for detailed description, but the technical solution of the embodiment of the present invention is not limited to MySQL, and for the same support of other types of databases, the implementation process of the embodiment of the present invention may be referred to for specific implementation.
The following is a detailed explanation of technical terms involved in the examples of the present invention:
a database instance, which refers to a stand-alone database service, may be understood as a database process, and a database server may run one or more database instances.
And the triple task refers to an extension task comprising a target instance, a data source instance and a mount main library instance.
The target instance refers to the database instance being expanded.
The data source instance refers to a database instance of a copy data source in the capacity expansion process.
And the mounting master library instance refers to a database instance to be mounted by the target instance, so that the data synchronization of the target instance and the mounting master library instance is ensured.
Sibling refers to other target instances that have the same data source instance as the target instance.
Fig. 1 is a schematic diagram of main steps of a method for database cluster capacity expansion according to an embodiment of the present invention, and as shown in fig. 1, the method for database cluster capacity expansion according to the embodiment of the present invention mainly includes the following steps:
step S101: and creating a triple task according to the received capacity expansion request of the database cluster. Wherein the triple task may include information of the following three elements: a target instance, a data source instance of the target instance, and a mount master library instance of the target instance. In this embodiment of the present invention, specifically, the creating a triple task according to the received capacity expansion request of the database cluster may include: one or more triple tasks are created based on one or more target instances included in the capacity expansion request.
Step S102: and executing the triple task to complete the expansion of the database cluster. The method comprises the steps that a triple task is created through a received capacity expansion request, and the capacity expansion of a database cluster is completed, so that a user does not need to perform extra control on capacity expansion, and the effect of automatic capacity expansion of the database cluster is achieved by directly submitting a cluster capacity expansion task; in the embodiment of the invention, the triple task is used as the minimum unit of cluster expansion, so that the cascade expansion or other complex expansion conditions can be compatible.
In this embodiment of the present invention, creating a triple task may further include: saving at least one of the following information in the triple task: the method comprises the steps of currently executing the triple tasks, executing states of the triple tasks, adding time of the triple tasks and updating time of the triple tasks.
In this embodiment of the present invention, the execution state of the triple task may include at least one of the following: not executed, executing, successful execution, failed execution.
In addition, in the embodiment of the present invention, when the triple task is executed, the triple task whose execution state is not executed is selected, and the triple task is executed from the current execution step of the triple task whose execution state is not executed. By saving the execution state of the triple task, the triple task failed to be executed can be submitted again according to the execution state; in addition, by storing the current execution steps of the triple tasks, the current state of the triple tasks can be checked at any time, the progress of the triple tasks is tracked, and when a certain triple task fails, the triple task can be executed directly from the failure step after the failure reason is solved, so that other tasks in the cluster are not influenced.
The method for expanding the database cluster according to the embodiment of the present invention may further include: if the triple task fails to execute, updating the execution state of the triple task to be execution failure; and after a new triple task identical to the triple task failed to be executed is created according to the received capacity expansion request, the new triple task is not stored, and the execution state of the triple task failed to be executed is modified into non-execution state.
In this embodiment of the present invention, creating a triple task may further include: and if the target instance has the brother node, storing the brother node information of the target instance in the triple task corresponding to the target instance. Backup files can be shared with sibling nodes by saving sibling node information of the triple task.
The method for expanding the database cluster according to the embodiment of the present invention may further include: before the triple task is executed, judging whether the sibling node of the target instance stored in the triple task is the target instance of other executed triple tasks, and if not, executing the triple task.
In the process of executing the triple task, the method for database cluster capacity expansion according to the embodiment of the present invention may further include: after generating a backup file on a target instance of the triple task, judging whether the backup file exists on a brother node of the target instance, if not, copying the backup file to the brother node of the target instance; and before clearing the backup file from the target instance of the triple task, judging whether the copying is finished, if so, clearing the backup file, and if not, continuing to wait until the copying is finished. Because the backup file is copied to the brother node of the target instance and is operated in the background process, the backup file is shared with the brother file, and the capacity expansion time is reduced.
The method for expanding the database cluster according to the embodiment of the present invention may further include: before the triple tasks are executed, whether the data source instances and/or the mounting main library instances of the triple tasks are target instances of other uncompleted triple tasks or not is judged, and if not, the triple tasks are executed. In the capacity expansion process, the mounting main library instance and the data source instance of the target instance are required to be existing database instances in the current cluster, so that before the triple task is executed, whether the triple task can be executed or not can be further judged by judging whether the data source instance and/or the mounting main library instance of the triple task are/is the target instances of other uncompleted triple tasks, and the capacity expansion efficiency is improved.
Fig. 2 is a schematic diagram of a process of creating a triple task by a method for database cluster capacity expansion according to an embodiment of the present invention, where each node represents one database instance, and the database instances in the cluster implement data synchronization in a master-slave replication manner, that is, data is replicated from one database instance (master) to multiple independent database instances (slave). Theoretically, any database instance in the cluster can be selected as a mount master instance, and in practice, selection is made according to service requirements, such as the distance of a machine room, reading and writing pressure and the like. In general, the data source instance selects a database instance with no write operations, a small number of connections, and low CPU utilization. As shown in fig. 2, an example of simultaneously expanding two database instances in an existing database cluster, a "new slave library D" and a "new slave library E" are target instances to be expanded, and two triple tasks are created according to the expansion request. The mount master library instance of the target instance "new slave library D" is "slave library A" and the data source instance is "slave library C". The mount master library instance of the target instance "new slave library E" is "slave library A" and the data source instance is "slave library C". Since the data source instances of the target instance "new slave library D" and "new slave library E" are the same, the target instance "new slave library D" and "new slave library E" are nodes that are brother nodes of each other, and the backup file is shared in the process of generating the backup file.
In the method for expanding the database cluster according to the embodiment of the present invention, the method for implementing data synchronization may be, but is not limited to, implementing data synchronization in a master-slave copy manner as shown in fig. 2, and may also implement data synchronization in combination with specific different service requirements.
Fig. 3 is a schematic diagram of the main architecture of a system suitable for implementing the method for database cluster capacity expansion according to the embodiment of the present invention. In the embodiment of the invention, the method for expanding the database cluster can be specifically realized by designing the database cluster expansion system. As shown in fig. 3, the architecture of the database cluster capacity expansion system in the embodiment of the present invention mainly includes the following four parts: the task adding unit, the task storage unit, the task selection unit and the task execution unit.
The task adding unit is used for receiving single or batch capacity expansion requests, creating one or more triple tasks according to the capacity expansion requests, and adding the triple tasks to the task storage unit.
The task adding process can be that an interface capable of adding the triple task is exposed outwards, and the interface is called when the task needs to be added from the outside. When adding the task, the capacity expansion task including one triple can be submitted each time, and the capacity expansion task including a plurality of triple tasks can also be submitted at one time. In addition, for the triple tasks which restart after the failure, the user calls the same interface as the added task to submit the failed triple tasks again after processing according to the displayed error log. After receiving the request, judging whether the triple task exists according to the triple task record stored in the task storage unit, if not, directly adding an unexecuted task record, otherwise, modifying the execution state of the previous triple task into unexecuted, and not modifying other information.
In the task storage unit, the triple task may be stored by using a storage database (which may be, but is not limited to, a MySQL database, etc.) and information such as a current execution step, an execution state, an addition time, an update time, etc. of the triple task is stored in order to control and track an execution process of the triple task. In addition, in order to save the file backup time for processing the expansion task of the brother node, the brother node information can be stored again, and all target instances of the data copied from the same data source instance in one cluster are recorded.
In the method for expanding the database cluster according to the embodiment of the present invention, the triple task is stored by using the "storage database", and the "storage database" is not used to limit the protection scope of the technical scheme of the present invention.
And reading all unexecuted triple tasks in the task storage unit at the task selection unit, and processing each executable triple task by taking the triple task as a unit.
The process of selecting the task can be scanning a storage database, and reading all the triple tasks with the execution states of not being executed; and checking whether the triple task in the execution state of non-execution can be executed, if so, starting a capacity expansion process, otherwise, skipping and continuing to process the next triple task.
The sequence of processing the triple tasks is relatively free, the triple tasks which are submitted firstly can be simply processed firstly, the priority of each triple task can be set according to the service requirement, and the triple tasks which fail to be executed before the triple tasks are processed firstly and the like can be processed firstly. In the method for database cluster capacity expansion according to the embodiment of the present invention, the sequence for processing the triple tasks may be, but is not limited to, the above, and the triple tasks to be processed may also be selected according to the requirements of specific different services.
Fig. 4 is a schematic main flowchart of a method for database cluster capacity expansion according to an embodiment of the present invention, which checks whether a triple task may start to be executed. As shown in fig. 4, the main process of checking whether a triple task may start to be executed according to the method for database cluster capacity expansion in the embodiment of the present invention includes: and judging whether the data source instance and/or the mount master library instance of the triple task are/is the target instances of other uncompleted triple tasks, if not, judging whether the brother node of the target instance stored in the triple task is the target instance of other executing triple tasks, and if not, starting a capacity expansion process.
In the method for database cluster capacity expansion according to the embodiment of the present invention, the main process for checking whether the triple task can start to be executed may be, but is not limited to, the process shown in fig. 4, and may also check whether the triple task can start to be executed in combination with specific different service requirements.
After the task execution unit successfully starts the capacity expansion process, the triple task is executed, which may specifically include, but is not limited to, the following twelve steps: (1) carrying out expansion environment inspection; (2) installing capacity expansion software; (3) generating a backup file; (4) preprocessing a backup file; (5) modifying the database configuration file; (6) closing the monitoring; (7) closing the database service; (8) restoring the backup data; (9) starting a database service; (10) establishing database synchronization; (11) starting monitoring; (12) and clearing the backup file. For the triple task which is recovered to be executed after the error occurs, the triple task directly jumps to the previous step, and the triple task is executed from the previous step until the triple task is completed, so that the function of recovering from the error can be realized.
When the backup file is generated in step (3), if the target instance of the executing triple task has other sibling nodes on which the file has not been backed up yet, the target instance of the executing triple task may start a background process to copy the backup file to the other sibling nodes. Before the backup file is cleared in the step (12), whether the copying is completed or not is judged, if so, the backup file is cleared, and if not, the process of waiting until the copying is completed is continued.
According to the technical scheme of the database cluster capacity expansion, the automatic capacity expansion of the database cluster is completed by creating the triple task according to the received capacity expansion request and executing the triple task, so that a user does not need to perform additional control on capacity expansion (for example, complicated capacity expansion task design needs to be performed by considering all capacity expansion conditions, and when the capacity expansion requirement is changed, the designed capacity expansion task needs to be integrally modified), and the effect of automatic capacity expansion of the database cluster can be achieved only by directly submitting the cluster capacity expansion request; in the embodiment of the invention, the triple task is used as the minimum unit of cluster expansion, and one or more triple tasks are created according to one or more target examples contained in the expansion request, so that the complexity of the design of the expansion task in the prior art can be reduced, and the cascade expansion or other complex expansion conditions can be compatible; by saving the execution state of the triple task, the triple task failed to be executed can be submitted again according to the execution state; in addition, by saving the current execution steps of the triple tasks, when a certain triple task fails, the triple task can be directly executed from the failure step after the failure reason is solved, and other tasks in the cluster are not influenced; in addition, by storing the current execution step of the triple task, the current state of the triple task can be checked at any time, and the progress of the triple task is tracked; by storing brother node information of the triple tasks, backup files can be shared with the brother nodes; before the triple tasks are executed, whether brother nodes of the target examples stored in the triple tasks are the target examples of other triple tasks being executed is verified, so that the complexity of executing the related capacity expansion tasks is reduced under the condition that brother nodes exist, and the overall capacity expansion time is further shortened; before the triple tasks are executed, whether the triple tasks can be executed or not can be further judged by judging whether the data source examples and/or the mount main library examples of the triple tasks are the target examples of other uncompleted triple tasks or not, and the capacity expansion efficiency is improved.
Fig. 5 is a schematic diagram of main modules of an apparatus for database cluster capacity expansion according to an embodiment of the present invention. As shown in fig. 5, the apparatus 500 for database cluster capacity expansion of the present invention mainly includes the following modules: creating module 501 and executing module 502.
The creating module 501 may be configured to create a triple task according to a received capacity expansion request of a database cluster, where the triple task includes information of the following three elements: the method comprises the steps that a target instance, a data source instance of the target instance and a mount master library instance of the target instance are installed; the execution module 502 may be used to execute triple tasks to complete the expansion of the database cluster.
In this embodiment of the present invention, the creating module 501 may further be configured to: one or more triple tasks are created based on one or more target instances included in the capacity expansion request.
In this embodiment of the present invention, the creating module 501 may further be configured to: saving at least one of the following information in the triple task: the method comprises the steps of currently executing the triple tasks, executing states of the triple tasks, adding time of the triple tasks and updating time of the triple tasks.
In the embodiment of the present invention, the execution state of the triple task includes at least one of the following states: not executed, executing, successful execution, failed execution. The execution module 502 may also be to: when the triple task is executed, the triple task with the execution state of unexecuted is selected, and the triple task with the execution state of unexecuted is executed from the current execution step of the triple task.
In this embodiment of the present invention, the creating module 501 may further be configured to: if the triple task fails to execute, updating the execution state of the triple task to be execution failure; and after a new triple task identical to the triple task failed to be executed is created according to the received capacity expansion request, the new triple task is not stored, and the execution state of the triple task failed to be executed is modified into non-execution state.
In this embodiment of the present invention, the creating module 501 may further be configured to: and if the target instance has the brother node, storing the brother node information of the target instance in the triple task corresponding to the target instance.
In this embodiment of the present invention, the execution module 502 may further be configured to: before the triple task is executed, judging whether the sibling node of the target instance stored in the triple task is the target instance of other executed triple tasks, and if not, executing the triple task.
In this embodiment of the present invention, in the process of executing the triple task, the execution module 502 may further be configured to: after generating a backup file on a target instance of the triple task, judging whether the backup file exists on a brother node of the target instance, if not, copying the backup file to the brother node of the target instance; and before clearing the backup file from the target instance of the triple task, judging whether the copying is finished, if so, clearing the backup file, and if not, continuing to wait until the copying is finished.
In this embodiment of the present invention, the execution module 502 may further be configured to: before the triple tasks are executed, whether the data source instances and/or the mounting main library instances of the triple tasks are target instances of other uncompleted triple tasks is judged, and if not, the triple tasks are executed.
As can be seen from the above description, the automatic capacity expansion of the database cluster is completed by creating the triple task according to the received capacity expansion request and executing the triple task, so that the user does not need to perform additional control on the capacity expansion (for example, the user needs to consider all capacity expansion conditions to perform complicated capacity expansion task design, and needs to modify the designed capacity expansion task integrally when the capacity expansion requirement changes), and the effect of the automatic capacity expansion of the database cluster can be achieved only by directly submitting the cluster capacity expansion request; in the embodiment of the invention, the triple task is used as the minimum unit of cluster expansion, and one or more triple tasks are created according to one or more target examples contained in the expansion request, so that the complexity of the design of the expansion task in the prior art can be reduced, and the cascade expansion or other complex expansion conditions can be compatible; by saving the execution state of the triple task, the triple task failed to be executed can be submitted again according to the execution state; in addition, by saving the current execution steps of the triple tasks, when a certain triple task fails, the triple task can be directly executed from the failure step after the failure reason is solved, and other tasks in the cluster are not influenced; in addition, by storing the current execution step of the triple task, the current state of the triple task can be checked at any time, and the progress of the triple task is tracked; by storing brother node information of the triple tasks, backup files can be shared with the brother nodes; before the triple tasks are executed, whether brother nodes of the target examples stored in the triple tasks are the target examples of other triple tasks being executed is verified, so that the complexity of executing the related capacity expansion tasks is reduced under the condition that brother nodes exist, and the overall capacity expansion time is further shortened; before the triple tasks are executed, whether the triple tasks can be executed or not can be further judged by judging whether the data source examples and/or the mount main library examples of the triple tasks are the target examples of other uncompleted triple tasks or not, and the capacity expansion efficiency is improved.
Fig. 6 illustrates an exemplary system architecture 600 of a method for database cluster capacity expansion or an apparatus for database cluster capacity expansion to which an embodiment of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 serves to provide a medium for communication links between the terminal devices 601, 602, 603 and the server 605. Network 604 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 601, 602, 603 to interact with the server 605 via the network 604 to receive or send messages or the like. The terminal devices 601, 602, 603 may have installed thereon various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 605 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 601, 602, 603. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the method for database cluster capacity expansion provided in the embodiment of the present invention is generally executed by the server 605, and accordingly, the apparatus for database cluster capacity expansion is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks, and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, shown is a block diagram of a computer system 700 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a creation module and an execution module. The names of these modules do not form a limitation on the module itself in some cases, for example, the creating module may also be described as a module that creates a triple task according to a received capacity expansion request of the database cluster.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: creating a triple task according to a received capacity expansion request of the database cluster, wherein the triple task comprises the following three elements of information: the method comprises the steps that a target instance, a data source instance of the target instance and a mount master library instance of the target instance are installed; and executing the triple task to complete the expansion of the database cluster.
According to the technical scheme of the embodiment of the invention, the automatic capacity expansion of the database cluster is completed by creating the triple task according to the received capacity expansion request and executing the triple task, so that the automatic capacity expansion of the database cluster can be realized only by directly submitting the cluster capacity expansion request without additionally controlling the capacity expansion by a user (for example, complicated capacity expansion task design needs to be carried out by considering all capacity expansion conditions, and the designed capacity expansion task needs to be integrally modified when the capacity expansion requirement is changed); in the embodiment of the invention, the triple task is used as the minimum unit of cluster expansion, and one or more triple tasks are created according to one or more target examples contained in the expansion request, so that the complexity of the design of the expansion task in the prior art can be reduced, and the cascade expansion or other complex expansion conditions can be compatible; by saving the execution state of the triple task, the triple task failed to be executed can be submitted again according to the execution state; in addition, by saving the current execution steps of the triple tasks, when a certain triple task fails, the triple task can be directly executed from the failure step after the failure reason is solved, and other tasks in the cluster are not influenced; in addition, by storing the current execution step of the triple task, the current state of the triple task can be checked at any time, and the progress of the triple task is tracked; by storing brother node information of the triple tasks, backup files can be shared with the brother nodes; before the triple tasks are executed, whether brother nodes of the target examples stored in the triple tasks are the target examples of other triple tasks being executed is verified, so that the complexity of executing the related capacity expansion tasks is reduced under the condition that brother nodes exist, and the overall capacity expansion time is further shortened; before the triple tasks are executed, whether the triple tasks can be executed or not can be further judged by judging whether the data source examples and/or the mount main library examples of the triple tasks are the target examples of other uncompleted triple tasks or not, and the capacity expansion efficiency is improved.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (18)
1. A method for database cluster expansion, comprising:
creating a triple task according to a received capacity expansion request of a database cluster, wherein the triple task comprises the following three elements of information: a target instance, a data source instance of the target instance, and a mount master library instance of the target instance; wherein the content of the first and second substances,
the task of creating the triple comprises the following steps: saving at least one of the following information in the triple task: the current execution step of the triple task, the execution state of the triple task, the adding time of the triple task and the updating time of the triple task;
and executing the triple task to complete the expansion of the database cluster.
2. The method of claim 1, wherein creating the triple task according to the received capacity expansion request of the database cluster comprises:
and creating one or more triple tasks according to one or more target instances contained in the capacity expansion request.
3. The method of claim 1, wherein the execution state of the triplet of tasks comprises at least one of: non-executing, successful execution, failed execution; and
when the triple task is executed, selecting the triple task with the execution state of non-execution, and starting to execute from the current execution step of the triple task with the execution state of non-execution.
4. The method of claim 3, further comprising:
if the triple task fails to execute, updating the execution state of the triple task to be execution failure; and
after a new triple task identical to the triple task failed to be executed is created according to the received capacity expansion request, the new triple task is not stored, and the execution state of the triple task failed to be executed is modified into non-execution state.
5. The method of claim 1, wherein creating a triple task further comprises: and if the target instance has a brother node, storing brother node information of the target instance in the triple task corresponding to the target instance, wherein the brother node refers to other target instances having the same data source instance as the target instance.
6. The method of claim 5, wherein prior to executing the triplet of tasks, the method further comprises: and judging whether the sibling node of the target example stored in the triple task is the target example of other executed triple tasks, and if not, executing the triple task.
7. The method of claim 5, wherein in performing the triplet of tasks, the method further comprises:
after generating a backup file on a target instance of the triple task, judging whether the backup file exists on a brother node of the target instance, if not, copying the backup file to the brother node of the target instance; and
before removing the backup file from the target instance of the triple task, judging whether the copying is finished or not, if so, removing the backup file, and if not, continuing to wait until the copying is finished.
8. The method of claim 1, further comprising:
before the triple task is executed, whether a data source instance and/or a mounting main library instance of the triple task is/are a target instance of other uncompleted triple tasks is judged, and if not, the triple task is executed.
9. An apparatus for database cluster expansion, comprising:
the creation module is configured to create a triple task according to a received capacity expansion request of the database cluster, where the triple task includes information of the following three elements: a target instance, a data source instance of the target instance, and a mount master library instance of the target instance; and the creation module is further configured to: saving at least one of the following information in the triple task: the current execution step of the triple task, the execution state of the triple task, the adding time of the triple task and the updating time of the triple task;
and the execution module is used for executing the triple tasks to complete the expansion of the database cluster.
10. The apparatus of claim 9, wherein the creation module is further configured to:
and creating one or more triple tasks according to one or more target instances contained in the capacity expansion request.
11. The apparatus of claim 9, wherein the execution state of the triplet of tasks comprises at least one of: non-executing, successful execution, failed execution; and
the execution module is further to: when the triple task is executed, selecting the triple task with the execution state of non-execution, and starting to execute from the current execution step of the triple task with the execution state of non-execution.
12. The apparatus of claim 11, wherein the creation module is further configured to: if the triple task fails to execute, updating the execution state of the triple task to be execution failure; and
after a new triple task identical to the triple task failed to be executed is created according to the received capacity expansion request, the new triple task is not stored, and the execution state of the triple task failed to be executed is modified into non-execution state.
13. The apparatus of claim 9, wherein the creation module is further configured to: and if the target instance has a brother node, storing brother node information of the target instance in the triple task corresponding to the target instance, wherein the brother node refers to other target instances having the same data source instance as the target instance.
14. The apparatus of claim 13, wherein the execution module is further configured to: before the triple task is executed, judging whether the sibling node of the target instance stored in the triple task is the target instance of other executed triple tasks, and if not, executing the triple task.
15. The apparatus of claim 13, wherein in executing the triplet of tasks, the execution module is further configured to:
after generating a backup file on a target instance of the triple task, judging whether the backup file exists on a brother node of the target instance, if not, copying the backup file to the brother node of the target instance; and
before removing the backup file from the target instance of the triple task, judging whether the copying is finished or not, if so, removing the backup file, and if not, continuing to wait until the copying is finished.
16. The apparatus of claim 9, wherein the execution module is further configured to:
before the triple task is executed, whether a data source instance and/or a mounting main library instance of the triple task is/are a target instance of other uncompleted triple tasks is judged, and if not, the triple task is executed.
17. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-8.
18. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710805786.8A CN107526838B (en) | 2017-09-08 | 2017-09-08 | Method and device for database cluster capacity expansion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710805786.8A CN107526838B (en) | 2017-09-08 | 2017-09-08 | Method and device for database cluster capacity expansion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107526838A CN107526838A (en) | 2017-12-29 |
| CN107526838B true CN107526838B (en) | 2020-05-01 |
Family
ID=60735830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710805786.8A Active CN107526838B (en) | 2017-09-08 | 2017-09-08 | Method and device for database cluster capacity expansion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107526838B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656709B (en) * | 2017-10-12 | 2021-08-10 | 北京京东尚科信息技术有限公司 | Codis cluster capacity expansion method and device |
| CN110874384B (en) * | 2018-09-03 | 2023-05-02 | 阿里巴巴集团控股有限公司 | Method, device and system for expanding and shrinking capacity of database cluster |
| CN113448940B (en) * | 2020-03-24 | 2023-09-22 | 北京京东振世信息技术有限公司 | Database capacity expansion method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000090002A (en) * | 1998-09-09 | 2000-03-31 | Toshiba Corp | Data base managing system |
| CN103034664A (en) * | 2011-10-10 | 2013-04-10 | 上海盛霄云计算技术有限公司 | Method, system and device for controlling data migration of database |
| CN103959706A (en) * | 2011-11-30 | 2014-07-30 | 微软公司 | Migrating authenticated content towards content consumer |
| CN104142943A (en) * | 2013-05-07 | 2014-11-12 | 上海盛大网络发展有限公司 | Database expansion method and database |
| CN104731896A (en) * | 2015-03-18 | 2015-06-24 | 北京百度网讯科技有限公司 | Data processing method and system |
| KR20160046572A (en) * | 2014-10-21 | 2016-04-29 | 포항공과대학교 산학협력단 | Method for expanding data of database and apparatus therefor |
| CN105760552A (en) * | 2016-03-25 | 2016-07-13 | 北京奇虎科技有限公司 | Data management method and device |
-
2017
- 2017-09-08 CN CN201710805786.8A patent/CN107526838B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000090002A (en) * | 1998-09-09 | 2000-03-31 | Toshiba Corp | Data base managing system |
| CN103034664A (en) * | 2011-10-10 | 2013-04-10 | 上海盛霄云计算技术有限公司 | Method, system and device for controlling data migration of database |
| CN103959706A (en) * | 2011-11-30 | 2014-07-30 | 微软公司 | Migrating authenticated content towards content consumer |
| CN104142943A (en) * | 2013-05-07 | 2014-11-12 | 上海盛大网络发展有限公司 | Database expansion method and database |
| KR20160046572A (en) * | 2014-10-21 | 2016-04-29 | 포항공과대학교 산학협력단 | Method for expanding data of database and apparatus therefor |
| CN104731896A (en) * | 2015-03-18 | 2015-06-24 | 北京百度网讯科技有限公司 | Data processing method and system |
| CN105760552A (en) * | 2016-03-25 | 2016-07-13 | 北京奇虎科技有限公司 | Data management method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107526838A (en) | 2017-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109120678B (en) | Method and apparatus for service hosting of distributed storage system | |
| CN110609872B (en) | Method and apparatus for synchronizing node data | |
| CN109032796B (en) | Data processing method and device | |
| CN110019123B (en) | Data migration method and device | |
| CN109245908B (en) | Method and device for switching master cluster and slave cluster | |
| CN107526838B (en) | Method and device for database cluster capacity expansion | |
| US20220012050A1 (en) | Mini program batch processing method and apparatus, electronic device, and readable storage medium | |
| CN111338834A (en) | Data storage method and device | |
| CN109117199B (en) | Configuration management method and device for distributed program | |
| CN113051055A (en) | Task processing method and device | |
| CN111767126A (en) | System and method for distributed batch processing | |
| CN110221910B (en) | Method and apparatus for performing MPI jobs | |
| CN113760924A (en) | Distributed transaction processing method and device | |
| CN110019059B (en) | Timing synchronization method and device | |
| CN114661420B (en) | Application protection method, device and system based on Kubernetes container platform | |
| US20230093004A1 (en) | System and method for asynchronous backend processing of expensive command line interface commands | |
| CN116208487A (en) | Method, device, equipment and medium for upgrading consensus algorithm in block chain system | |
| CN113761075A (en) | Method, device, equipment and computer readable medium for switching databases | |
| CN114265605A (en) | Version rollback method and device for functional component of business system | |
| CN113127430B (en) | Mirror image information processing method, mirror image information processing device, computer readable medium and electronic equipment | |
| CN114443216A (en) | Cloud platform mirror image management method and management system | |
| CN112379973A (en) | Heavy loading method and device | |
| CN111767113A (en) | Method and device for realizing container eviction | |
| CN112749042B (en) | Application running method and device | |
| CN111984686A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |