CN113342508A

CN113342508A - Task scheduling method and device

Info

Publication number: CN113342508A
Application number: CN202110769168.9A
Authority: CN
Inventors: 张志广; 彭一; 喻俊
Original assignee: Hunan MgtvCom Interactive Entertainment Media Co Ltd
Current assignee: Hunan MgtvCom Interactive Entertainment Media Co Ltd
Priority date: 2021-07-07
Filing date: 2021-07-07
Publication date: 2021-09-03

Abstract

The invention provides a task scheduling method and a task scheduling device, wherein each time a task to be executed is executed, whether a registration center is available is judged; if the task is not available, processing the task to be executed according to a cache history scheduling mode; if the scheduling scheme is available, judging whether to re-formulate the scheduling scheme; if the scheduling scheme does not need to be re-formulated and is the master node, executing the task to be executed and triggering other slave nodes to execute the task to be executed, and if the scheduling scheme needs to be re-formulated, initiating a master node election process to the registration center; if the master node becomes the master node after the master node election process is initiated, a new scheduling scheme is formulated, the task to be executed is executed, and the slave node specified by the new scheduling scheme is triggered to execute the task to be executed; and if the scheduling scheme does not need to be re-formulated and the selected master node is the slave node, or if the selected master node is the slave node after the master node election process is initiated, receiving a trigger signal sent by the master node, and executing the task to be executed. The dependence on the registration center is reduced, and the stability and the usability of the task scheduling system are improved.

Description

Task scheduling method and device

Technical Field

The invention relates to the technical field of internet, in particular to a task scheduling method and a task scheduling device.

Background

When internet applications are developed to a certain scale, a task scheduling system is usually required to be abstracted to perform task scheduling work, and the current task scheduling systems are generally divided into a centralized task scheduling system (e.g., xxl-joba) and a decentralized task scheduling system (e.g., Elastic-joba).

However, the xxl-joba operation depends on the availability of the MySQL database, which can seriously affect the task scheduling capability of xxl-joba when the MySQL database fails, while the Elastic-joba operation depends on the availability of the Zookeeper, which can cause the Elastic-joba to be unavailable when the Zookeeper fails on the whole cluster. That is, the existing task scheduling system has poor stability and usability.

Disclosure of Invention

In view of this, embodiments of the present invention provide a task scheduling method and device, so as to solve the problems of poor stability and poor usability of the existing task scheduling system.

In order to achieve the above purpose, the embodiments of the present invention provide the following technical solutions:

the first aspect of the embodiments of the present invention discloses a task scheduling method, which is applicable to a target service server, where the target service server is any service server in a service server cluster, and the method includes:

judging whether a registration center is available or not when a pre-configured task to be executed starts to be executed each time;

if the registration center is unavailable, processing the task to be executed according to a historical scheduling mode which is cached in the target service server in advance, wherein the historical scheduling mode is a mode of processing the task to be executed in the last scheduling period;

if the registration center is available, judging whether to reformulate a scheduling scheme of the task to be executed based on the states of all service servers in the service server cluster, wherein the scheduling scheme is used for designating the service server which needs to execute the task to be executed in the service server cluster;

if the scheduling scheme is not re-established, executing the following steps;

if the scheduling scheme is re-established, initiating a master node election process to the registration center, and competing for master nodes with other service servers in the service server cluster, and executing the following steps;

the following steps include:

judging whether the target service server is a main node or not;

if the target service server is a master node and the scheduling scheme is not re-formulated, executing the task to be executed, and triggering all slave nodes specified by the scheduling scheme to execute the task to be executed, wherein each available slave node corresponds to any service server in the service server cluster;

if the target service server is a master node and the scheduling scheme is re-formulated, formulating a new scheduling scheme according to all available slave nodes in the registry, executing the task to be executed, and triggering all slave nodes specified by the new scheduling scheme to execute the task to be executed;

and if the target service server is a slave node, receiving a trigger signal sent by a master node, and executing the task to be executed.

Preferably, if the registry is available, determining whether to reformulate the scheduling scheme of the task to be executed based on the states of all the service servers in the service server cluster, including:

if the registration center is available, determining whether to increase or decrease the service servers in the service server cluster based on the states of all the service servers in the service server cluster;

if a service server is newly added or reduced in the service server cluster, determining to reformulate the scheduling scheme of the task to be executed;

and if the service servers are not newly added or reduced in the service server cluster, determining not to reformulate the scheduling scheme.

Preferably, if the scheduling scheme is re-formulated, initiating a master node election process to the registration center to race for master nodes with other service servers in the service server cluster, including:

if the scheduling scheme is re-formulated, initiating a master node election process to the registration center, and determining whether a task allocation mark and task global information of the task to be executed exist in the registration center;

if the task distribution mark and/or the task global information exist in the registry, registering server information of the target service server to the registry so that the target service server becomes a slave node;

if the task allocation mark and the task global information do not exist in the registration center, setting the task allocation mark and registering the task global information to the registration center, and registering the server information of the target service server to the registration center, so that the target service server becomes a main node.

Preferably, if the target service server is a master node and the scheduling scheme is newly formulated, formulating a new scheduling scheme according to all available slave nodes in the registry, executing the task to be executed, and triggering all slave nodes specified by the new scheduling scheme to execute the task to be executed includes:

if the target service server is a master node and the scheduling scheme is re-formulated, capturing the IP of all available slave nodes in the registration center, and formulating a new scheduling scheme according to a preset algorithm;

the task to be executed is executed, and the task to be executed is executed,

and sending a trigger signal to the slave node according to the IP of the slave node for each slave node specified by the new scheduling scheme, so that the slave node executes the task to be executed.

Preferably, the determining whether the registry is available or not when the pre-configured task to be executed starts to be executed each time includes:

inquiring the health index of a registration center when a pre-configured task to be executed starts to be executed each time;

and judging whether the registration center is available according to the health index.

Preferably, if the registry is unavailable, the processing the task to be executed according to a historical scheduling manner cached in the target service server in advance includes:

if the registration center is unavailable, determining whether the target service server executes the task to be executed in the last scheduling period according to a historical scheduling mode cached in the target service server in advance;

if the target service server does not execute the task to be executed in the last scheduling period, the task to be executed is not executed;

and if the target service server executes the task to be executed in the last scheduling period, executing the task to be executed.

Preferably, the method further comprises the following steps:

and uploading the execution information of the task to be executed to a preset database.

Preferably, after the formulating a new scheduling scheme according to all available slave nodes in the registry, the method further includes:

and uploading the new scheduling scheme to the registration center.

A second aspect of the present invention discloses a task scheduling apparatus, which is applicable to a target service server, where the target service server is any service server in a service server cluster, and the apparatus includes:

the first judging unit is used for judging whether the registration center is available or not when a pre-configured task to be executed starts to be executed each time;

a first processing unit, configured to, if the registry is unavailable, process the to-be-executed task according to a historical scheduling manner that is cached in the target service server in advance, where the historical scheduling manner is a manner of processing the to-be-executed task in a previous scheduling period;

a second determining unit, configured to determine, if the registration center is available, whether to reformulate a scheduling scheme for the task to be executed based on states of all service servers in the service server cluster, where the scheduling scheme is used to designate a service server in the service server cluster that needs to execute the task to be executed;

the second processing unit is used for executing the following steps if the scheduling scheme is not re-established; if the scheduling scheme is re-established, initiating a master node election process to the registration center, and competing for master nodes with other service servers in the service server cluster, and executing the following steps;

the following steps include:

judging whether the target service server is a main node or not;

Preferably, the second determining unit is specifically configured to: if the registration center is available, determining whether to increase or decrease the service servers in the service server cluster based on the states of all the service servers in the service server cluster; if a service server is newly added or reduced in the service server cluster, determining to reformulate the scheduling scheme of the task to be executed; and if the service servers are not newly added or reduced in the service server cluster, determining not to reformulate the scheduling scheme.

Based on the above method and apparatus for task scheduling provided by the embodiments of the present invention, the method is applicable to a target service server, the target service server is any service server in a service server cluster, and the method includes: judging whether the registration center is available or not when the task to be executed starts to be executed each time; if the registration center is unavailable, processing the task to be executed according to a historical scheduling mode cached in the target service server; if the registration center is available, judging whether to reformulate the scheduling scheme of the task to be executed; if the scheduling scheme does not need to be re-formulated and is the master node, executing the task to be executed and triggering other slave nodes to execute the task to be executed, and if the scheduling scheme needs to be re-formulated, initiating a master node election process to the registration center; if the master node becomes the master node after the master node election process is initiated, a new scheduling scheme is formulated, the tasks to be executed are executed, and all slave nodes specified by the new scheduling scheme are triggered to execute the tasks to be executed; and if the scheduling scheme does not need to be re-formulated and the master node is a slave node, or if the master node becomes a slave node after initiating the master node election process, receiving a trigger signal sent by the master node, and executing the task to be executed. The dependence on the registration center is reduced, and the stability and the usability of the task scheduling system are improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a flowchart of a task scheduling method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of the relationship between a configuration layer, an execution layer, a support layer, and a storage layer according to an embodiment of the present invention;

fig. 3 is another flowchart of a task scheduling method according to an embodiment of the present invention;

fig. 4 is another flowchart of a task scheduling method according to an embodiment of the present invention;

fig. 5 is a flowchart of initiating a master node election process to a registration center according to an embodiment of the present invention;

fig. 6 is another flowchart of initiating a master node election process to a registry according to an embodiment of the present invention;

fig. 7 is a block diagram of a task scheduling device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

As is known in the art, the categories of existing task scheduling systems are generally divided into centralized task scheduling systems such as xxl-joba and decentralized task scheduling systems such as Elastic-joba. However, the operation of xxl-joba depends on the availability of the MySQL database, which can seriously affect the task scheduling capability of xxl-joba when the MySQL database fails, while the operation of Elastic-joba depends on the availability of Zookeeper, which can cause the Elastic-joba to be unavailable when the Zookeeper fails to complete the cluster, i.e. the existing task scheduling system has poor stability and poor availability.

Therefore, the embodiment of the invention provides a task scheduling method and a task scheduling device, which are suitable for a target service server belonging to any service server in a service server cluster, and monitor whether a registration center is available or not when a task to be executed starts to be executed each time. And when the registration center is unavailable, processing the tasks to be executed by utilizing a pre-cached historical scheduling mode. When the registration center is available, judging whether a scheduling scheme of the task to be executed needs to be re-formulated, if the scheduling scheme does not need to be re-formulated and is a main node, executing the task to be executed and triggering other slave nodes to execute the task to be executed; if the scheduling scheme needs to be re-formulated, initiating a master node election process to the registration center; if the master node becomes the master node after the master node election process is initiated, a new scheduling scheme is formulated, the tasks to be executed are executed, and all slave nodes specified by the new scheduling scheme are triggered to execute the tasks to be executed; and if the scheduling scheme does not need to be re-formulated and the master node is a slave node, or if the master node becomes a slave node after initiating the master node election process, receiving a trigger signal sent by the master node, and executing the task to be executed. The dependence on the registry is reduced, so that the stability and the availability of the task scheduling system are improved.

It should be noted that the following explains english abbreviations and their meanings referred to in the embodiments of the present invention in advance.

And (3) SDK: software development kit, i.e., Software development kit.

SideCar: the clients of distributed task scheduling, provided in the form of SDKs, are usually integrated in the service servers, also referred to as task nodes.

It should be noted here that in the same task, the nodes are divided into the master node and the slave node, but the identity of the same node in different tasks may be different, for example: in task a, node 1 is the master, while in task B, node 1 is the slave.

An execution: and the task Executor is used for realizing an execution interface provided by the SideCar for the service server, and the execution is the execution main body of the task.

Gateway: and the gateway provides a data uploading result for the SideCar and provides an operation interface of an editing task for the Control Panel.

controlPanel: and the console is a management background.

Job Status Management: and task state management, which mainly provides an interface for interacting with the database.

Monitor: and the monitoring management module is mainly used for alarming and collecting the reported data indexes.

Referring to fig. 1, a flowchart of a task scheduling method provided in an embodiment of the present invention is shown, where the task scheduling method is applicable to a target service server, where the target service server is any service server in a service server cluster, and the task scheduling method includes:

step S101: and judging whether the registration center is available or not when the pre-configured task to be executed starts to be executed each time. If the registry is not available, step S102 is performed, and if the registry is available, step S103 is performed.

It should be noted that, the task configuration of the task to be executed is configured in advance, and specifically, a support layer may be defined, through which task configuration of various tasks (task configuration of task configured by using Control Panel) can be completed, and at the same time, the support layer also provides functions such as Gateway (Gateway), task state Management (Job Status Management), task exception compensation, and task monitoring (Monitor).

It can be understood that, the task to be executed is executed according to a preset period, the service server (including the target service server) mainly includes the above mentioned SideCar and Executor, and the service server is set in a predefined execution layer, and the SideCar of the target service server listens to the registration center for events, such as: task configuration change events such as modification, suspension, and manual execution are monitored.

In the process of implementing step S101, each time the task to be executed starts to be executed, the SideCar of the target service server determines whether the registry is available. If the registry is not available, step S102 is performed, and if the registry is available, step S103 is performed.

It should be noted that all the service servers in the service server cluster can be used as nodes and registered with the registry (e.g., Zookeeper) through connection, and each node has health check at the connection level and uploads the health check to the support layer. In some embodiments, the SideCar of the target service server queries the health index of the registry each time when the task to be executed starts to be executed, and determines whether the registry is available according to the health index, wherein the health index is obtained by summarizing the connection health conditions of all the nodes collected by the support layer.

The above-mentioned registration center can be set in a predefined configuration layer, and the configuration layer mainly completes the functions of node selection owner (i.e. election master node mentioned in the following steps), task allocation (corresponding to the scheduling scheme mentioned in the following steps) and configuration change (configuration of tasks in Control Panel, such as new task, modification task, start task and stop task) through the registration center.

It can be understood that node election specifically refers to: a plurality of task nodes (i.e. the sidecars of a plurality of service servers) compete for a distributed lock, and the service server that competes for obtaining the distributed lock is the master node, and it should be noted that each preconfigured task needs to select a master node instead of all tasks sharing a master node.

Step S102: and processing the tasks to be executed according to a historical scheduling mode which is cached in the target service server in advance.

It should be noted that the historical scheduling mode is a mode of processing the to-be-executed task in the last scheduling period, all nodes are triggered to perform a caching task by a timing task frame (for example, quartz), and the mode of processing the to-be-executed task in the last scheduling period is cached, so that each node (i.e., each service server) records whether the to-be-executed task in the last scheduling period is executed by itself.

In the process of implementing the step S102 specifically, if it is determined that the registration center is unavailable, the SideCar of the target service server determines whether the target service server executes the task to be executed in the previous scheduling period according to a historical scheduling manner cached in the target service server in advance; if the target service server does not execute the task to be executed in the last scheduling period, the task to be executed is not executed (equivalent to skipping); and if the target service server executes the task to be executed in the last scheduling period, executing the task to be executed.

For example: and if the target service server is assumed to be a service server A, the service server A judges that the registration center is unavailable, and determines whether to execute the task to be executed in the last scheduling period. And if the task to be executed is determined to be executed by the scheduling unit in the previous scheduling period, executing the task to be executed, and if the task to be executed is determined not to be executed by the scheduling unit in the previous scheduling period, skipping.

Step S103: and judging whether to reformulate the scheduling scheme of the task to be executed again or not based on the states of all the service servers in the service server cluster. If the scheduling scheme is not re-established, step S105 and the following steps are performed. If the scheduling scheme is newly established, step S104 is executed.

It should be noted that the scheduling scheme is used to specify the service servers in the service server cluster that need to execute the tasks to be executed, that is, assign the tasks to be executed to some service servers in the service server cluster for execution (that is, assign the tasks).

In the process of implementing step S103 specifically, if the target service server determines that the registration center is available, it is determined whether to add or subtract a new service server in the service server cluster based on the states of all service servers in the service server cluster (i.e., it is determined whether the service servers in the service server cluster change). If a new service server is added or decreased in the service server cluster, determining to reformulate the scheduling scheme of the task to be executed (which is equivalent to task allocation of the task to be executed again), and executing step S104. If no new service server or no new service server is added to or subtracted from the service server cluster, the scheduling scheme is determined not to be re-formulated, and step S105 and the subsequent steps are executed.

Step S104: and initiating a master node election process to the registration center, competing for master nodes with other service servers in the service server cluster, and executing the step S105 and the subsequent steps.

In the process of implementing step S104, if the target service server determines to reformulate the scheduling scheme, the target service server initiates a master node election process to the registration center, and performs master node election with other service servers in the service server cluster, and after the master node election process is completed, the target service server becomes a master node or a slave node.

It can be understood that the service server serving as the master node needs to complete the operations of making a scheduling scheme (i.e., task allocation), registering task global information of tasks to be executed, setting task allocation flags of the tasks to be executed, and task cleaning.

It should be noted that task cleaning specifically means: and the master node takes the slave node off line, so that the task on the slave node which takes off line is not executed.

Specifically, for concurrent security, when a scheduling scheme of a task is to be formulated, a corresponding task assignment flag needs to be set to indicate that the task is formulating the scheduling scheme.

Step S105: and judging whether the target service server is a main node or not. If the target service server is the master node and the scheduling scheme is not re-established, step S106 is executed, if the target service server is the master node and the scheduling scheme is re-established, step S107 is executed, and if the target service server is the slave node, step S108 is executed.

In the process of implementing step S105, after the target service server initiates the master node election process to the registration center, it may be determined whether the target service server is a master node. At this time, if the target service server is the master node and the scheduling scheme is not re-established, step S106 is performed, if the target service server is the master node and the scheduling scheme is re-established, step S107 is performed, and if the target service server is the slave node, step S108 is performed.

It can be understood that, if the target service server is a slave node, no matter whether the scheduling scheme needs to be re-formulated, step S108 is executed, and as long as the trigger signal sent by the master node is received, the task to be executed is executed.

Step S106: and if the target service server is the master node and does not re-establish the scheduling scheme, executing the tasks to be executed, and triggering all the slave nodes specified by the scheduling scheme to execute the tasks to be executed.

It should be noted that each available slave node corresponds to any traffic server in the traffic server cluster.

In the process of implementing step S106 specifically, if the target service server is the master node and does not re-make the scheduling scheme, that is, it indicates that the current scheduling scheme does not need to be changed, at this time, the target service server executes the task to be executed, and triggers all the slave nodes specified by the scheduling scheme to execute the task to be executed according to the scheduling scheme (equivalent to the originally-made scheduling scheme).

And for each slave node specified by the scheduling scheme, the target service server sends a trigger signal to the slave node through the registration center according to the IP of the slave node, so that the slave node executes the task to be executed.

It should be noted that, the service servers in the service server cluster are registered in the registration center in advance, and the registration center includes registration information (including the IP of the service server) of all service servers that have been registered in advance.

Specifically, the SideCar of the target service server is triggered by a timing task framework (such as quartz), and the executive Executor of the target service server executes the task to be executed.

Step S107: and if the target service server is the master node and the scheduling scheme is re-formulated, formulating a new scheduling scheme according to all available slave nodes in the registration center, executing the task to be executed, and triggering all the slave nodes specified by the new scheduling scheme to execute the task to be executed.

In the process of implementing step S107 specifically, if the target service server is the master node and the scheduling scheme is re-established, the target service server captures the IPs of all available slave nodes in the registration center, and selects a node that needs to execute a task to be executed according to a preset algorithm (e.g., polling, hash, random, failover, and other algorithms), so as to establish and obtain a new scheduling scheme, where the new scheduling scheme specifies which service servers in the service server cluster need to execute the task to be executed (i.e., specifies Job Owner).

After the target service server formulates a new scheduling scheme, the SideCar of the target service server is triggered by a timing task framework (such as quartz), and an executive Executor of the target service server executes the task to be executed.

And for each slave node specified by the new scheduling scheme, the target service server sends a trigger signal to the slave node through the registration center according to the IP of the slave node, so that the slave node executes the task to be executed. That is, the target service server triggers the executors of all the slave nodes specified by the new scheduling scheme to execute the task to be executed through the registry.

Preferably, after the target service server executes the task to be executed, the SideCar of the target service server reports execution information (for example, information about start time, end time, execution state, time consumption, and the like) of the task to be executed to the Gateway of the support layer, and uploads the execution information of the task to be executed to a preset database (for example, Redis) through the Gateway of the support layer. It can be understood that the database is disposed in a predefined storage layer, and when a fault occurs in the support layer and/or the storage layer, the execution information of the task to be executed cannot be uploaded to the database, and at this time, the target service server may cache the execution information of the task to be executed in a local queue, and after the support layer and the storage layer are both normal, the execution information of the task to be executed is uploaded to the storage layer for storage.

It should be noted that after other slave nodes execute the to-be-executed task, the execution information of the to-be-executed task also needs to be uploaded to the storage layer, which may be specifically referred to the relevant content of the execution information of the to-be-executed task uploaded by the target service server, and is not described herein again.

Preferably, after the target service server formulates a new scheduling scheme, the new scheduling scheme is uploaded to the registry.

Step S108: and receiving a trigger signal sent by the main node, and executing a task to be executed.

In the process of implementing step S108 specifically, if the target service server is a slave node, no matter whether the scheduling scheme needs to be re-formulated, when the target service server receives a trigger signal sent by a master node (at this time, the master node is another service server in the service server cluster), the target service server executes the task to be executed, and uploads execution information of the task to be executed to the storage layer, where the specific uploading manner may refer to the content in step S107, and is not described herein again.

In the embodiment of the invention, when the task to be executed starts to execute each time, whether the registration center is available is monitored. And when the registration center is unavailable, processing the tasks to be executed by utilizing a pre-cached historical scheduling mode. And when the registration center is available, judging whether the scheduling scheme of the task to be executed needs to be re-formulated. And if the master node does not need to re-formulate the scheduling scheme, executing the task to be executed and triggering other slave nodes to execute the task to be executed. If the scheduling scheme needs to be re-formulated, a master node election process is initiated to the registration center. And if the master node becomes the master node after the master node election process is initiated, formulating a new scheduling scheme, executing the tasks to be executed and triggering all slave nodes appointed by the new scheduling scheme to execute the tasks to be executed. And if the scheduling scheme does not need to be re-formulated and the master node is a slave node, or if the master node becomes a slave node after initiating the master node election process, receiving a trigger signal sent by the master node, and executing the task to be executed. The dependence on the registration center is reduced, and the stability and the usability of the task scheduling system are improved.

In order to facilitate understanding of the configuration layer, the execution layer, the support layer and the storage layer mentioned in the steps of fig. 1 in the embodiment of the present invention, the description is made by using a schematic diagram of the relationship among the configuration layer, the execution layer, the support layer and the storage layer shown in fig. 2.

As shown in fig. 2, a registry such as Zookeeper is provided in the configuration layer 100, service servers are provided in the execution layer 200, each service server at least includes an executive and a SideCar, Gateway, Control Panel, Job Status Management and Monitor are provided in the support layer 300, and a database such as Redis is provided in the storage layer 400.

The configuration layer 100 is connected to the execution layer 200 and the support layer 300, respectively, the execution layer 200 is connected to the support layer 300, and the support layer 300 is connected to the storage layer 400.

It is to be understood that in order to better explain the contents of the various steps in fig. 1 of the above-described embodiment of the present invention, the description is made by way of example with reference to fig. 3 and 4.

Referring to fig. 3, another flowchart of a task scheduling method provided by the embodiment of the present invention is shown, which includes the following steps:

step S301: and the manager logs in the console in the Control Panel and creates a new task to be executed.

In the process of implementing step S301 specifically, after the administrator logs in the console, a new task to be executed may be created, for example: a 'task is executed once every 5 seconds' is newly established, the task to be executed can be started, and basic information of the task to be executed is saved in the registry.

Step S302: after the SideCar of the target service server is started, the on-line node is registered in the registration center, and the owner selection is completed to become the main node.

Step S303: the target service server determines the namespace controller of the task to be executed.

It should be noted that namespace controller is a namespace, and is used for distinguishing different tasks.

Step S304: the target service server monitors the registration center and makes a scheduling scheme.

In the specific implementation process of step S304, when the scheduling scheme is formulated, two steps of task allocation and Job Owner allocation are mainly included.

It should be noted that, if no scheduling scheme is defined for the task to be executed (i.e. no task assignment flag is present), the default value of the scheduling scheme is 1, and a Job Owner needs to be assigned.

Step S305: and the target service server uploads the scheduling scheme to the registration center.

Referring to fig. 4, another flowchart of a task scheduling method provided by the embodiment of the present invention is shown, where the task scheduling method is applicable to a target service server, and the task scheduling method includes the following steps:

step S401: and judging whether the registration center is available or not when the task to be executed starts to be executed each time. If the registration center is available, executing step S402, if the registration center is not available, executing the executer, and processing the task to be executed according to the historical scheduling mode pre-cached in the target service server.

Step S402: and judging whether to reformulate the scheduling scheme of the task to be executed. If the scheduling scheme of the task to be executed is re-established, step S403 is executed, and if the scheduling scheme of the task to be executed is not re-established, step S404 is executed.

Step S403: and initiating a main node election process to the registration center, and determining whether the main node election process is finished. If the master node election process is finished, step S404 is executed, and if the master node election process is not finished, the sleep is performed for 100ms and the step of determining whether the master node election process is finished is returned to.

Step S404: it is determined whether the target traffic server is the master node. If the target service server is not the master node, step S405 is executed, and if the target service server is the master node, step S406 is executed.

Step S405: and judging whether the scheduling scheme of the task to be executed is in process. And if the scheduling scheme of the task to be executed is being made, sleeping for 100ms and returning to execute the step S405, and if the scheduling scheme of the task to be executed is made and a trigger signal sent by the main node is received, executing the executive to execute the task to be executed.

Step S406: the task assignment tag is written to the registry.

Step S407: and completing task allocation of the tasks to be executed.

In the process of implementing step S407, task allocation for completing the task to be executed is mainly divided into two cases: one situation is that if the scheduling scheme does not need to be re-established and the target service server is the master node, the task allocation of the tasks to be executed is directly completed according to the original scheduling scheme. If the scheduling scheme needs to be re-established and the target service server is the master node, a new scheduling scheme of the tasks to be executed is established, and task allocation of the tasks to be executed is completed according to the new scheduling scheme.

Step S408: and judging whether each slave node executes the task to be executed according to the IP of the slave node specified by the scheduling scheme or the new scheduling scheme. And if the to-be-executed task is being executed, ending the process, and if the to-be-executed task is not being executed, triggering each slave node to execute the executer to execute the to-be-executed task.

It is understood that the IP of each slave node can be obtained from the registry, and whether each slave node is executing the task to be executed is inquired from the storage layer according to the IP.

It should be noted that the contents of the steps in fig. 4 are only for illustration.

Fig. 5 shows a flowchart of a process for a registration center to initiate a master node election process according to an embodiment of the present invention, where the process for a target service server to race for a master node involved in step S104 in fig. 1 includes the following steps:

step S501: and initiating a master node election process to the registration center, and determining whether a task distribution mark and task global information of a task to be executed exist in the registration center. If the registry has the task allocation flag and/or the task global information, step S502 is executed, and if the registry does not have the task allocation flag and the task global information, step S503 is executed.

In the process of implementing step S501 specifically, after the target service server initiates the master node election process to the registry, it is determined whether task allocation flags of tasks to be executed and registered task global information set by other service servers already exist in the registry. If the registry already has the task allocation flag and/or the task global information of the task to be executed, step S502 is executed, and if the registry does not have the task allocation flag and the task global information of the task to be executed, step S503 is executed.

Step S502: server information of the target service server is registered with the registry so that the target service server becomes a slave node.

In the process of implementing step S502 specifically, if the registry already has the task allocation flag and/or the task global information of the task to be executed, the target service server registers the server information of the target service server with the registry, and persists the server information of the target service server in the registry, so that the target service server becomes a slave node.

Step S503: and setting a task distribution mark and registering task global information to the registration center, and registering server information of the target service server to the registration center so that the target service server becomes a main node.

In the process of implementing step S503 specifically, if the registration center does not have the task allocation flag and the task global information of the task to be executed, the target service server sets the task allocation flag of the task to be executed to the registration center and registers the task global information of the task to be executed, registers the server information of the target service server to the registration center, persists the task allocation flag of the task to be executed in the registration center, and persists the task global information of the task to be executed and the server information of the target service server in the registration center, so that the target service server becomes the master node.

To better explain how a target service server competes for master nodes with other service servers in the service server cluster (or how master nodes are competed among multiple service servers), another flow diagram for initiating a master node election process with a registration center is provided in fig. 6.

It should be noted that, in fig. 6, two service servers (service server a and service server B) are taken as an example to explain that if the master node is elected by an election, and the election result is that the service server a becomes the master node, the following steps S601-S611 are steps of initiating the master node election process from the service server a to the registration center, and the following steps S612-S622 are steps of initiating the master node election process from the service server B to the registration center.

It should be noted that the start sequence of the service server a, the registration center and the service server B shown in fig. 6 is as follows: the registration center is started firstly, then the service server A is started, the service server B is started finally, finally the service server A becomes a main node, the service server A firstly sets a task allocation mark of a task to be executed and registers task global information of the task to be executed in the registration center, and the service server B learns that the task allocation mark of the task to be executed and the task global information of the task to be executed exist according to information fed back by the registration center, so that the service server B is a slave node.

Fig. 6 provides another flow diagram of a process for initiating a master node election process with a registry, comprising the steps of:

step S601: the service server a listens to the registration center.

Step S602: the service server A initiates a main node election process to the registration center.

Step S603: the registration center elects the main node.

Step S604: the traffic server a becomes the master node.

Step S605: and the service server A sets a task allocation mark of the task to be executed and sends the task allocation mark to the registration center.

Step S606: the registry persists task assignment tags for tasks to be performed.

Step S607: the service server A registers the task global information of the task to be executed.

Step S608: and the registry persists the task global information of the task to be executed.

Step S609: the service server a registers the server information of the service server a.

Step S610: the registry persists the server information of the service server a.

Step S611: and the service server A waits for the task to be executed to run.

Step S612: the service server B listens to the registration center.

Step S613: and the service server B initiates a main node election process to the registration center.

Step S614: the registration center elects the main node.

Step S615: the traffic server B becomes a slave node.

Step S616: and the service server B sets a task allocation mark of the task to be executed and sends the task allocation mark to the registration center.

Step S617: and the registry determines that the task allocation mark of the task to be executed already exists and does not persist the task allocation mark of the task to be executed sent by the service server B.

Step S618: and the service server B registers the task global information of the task to be executed.

Step S619: and the registration center determines that the task global information of the task to be executed already exists and does not persist the task global information of the task to be executed registered by the service server B.

Step S620: the service server B registers the server information of the service server B.

Step S621: the registry persists the server information of the service server B.

Step S622: and the service server B waits for the task to be executed to run.

In the embodiment of the invention, the target service server monitors the registration center, and initiates the master node election process to the registration center when the task to be executed runs each time, and after the master node election process is finished, the target service server becomes a master node or a slave node. If the target service server is the master node, a new scheduling scheme is formulated according to all available slave nodes in the registration center, the task to be executed is executed, and all the slave nodes specified by the new scheduling scheme are triggered to execute the task to be executed. And if the target service server is a slave node, receiving a trigger signal sent by the master node and executing the task to be executed. The dependence on the registration center is reduced, and the stability and the usability of the task scheduling system are improved.

Corresponding to the task scheduling method provided by the embodiment of the present invention, referring to fig. 7, an embodiment of the present invention further provides a structural block diagram of a task scheduling device, where the task scheduling device is applicable to a target service server, the target service server is any service server in a service server cluster, and the task scheduling device includes: a first judgment unit 701, a first processing unit 702, a second judgment unit 703, and a second processing unit 704;

a first determining unit 701, configured to determine whether the registry is available each time a pre-configured task to be executed starts to be executed.

In a specific implementation, the first determining unit 701 is specifically configured to: inquiring the health index of a registration center when a pre-configured task to be executed starts to be executed each time; and judging whether the registration center is available according to the health index.

The first processing unit 702 is configured to, if the registry is unavailable, process the to-be-executed task according to a historical scheduling manner cached in the target service server in advance, where the historical scheduling manner is a manner of processing the to-be-executed task in a previous scheduling period.

In a specific implementation, the first processing unit 702 is specifically configured to: if the registration center is unavailable, determining whether the target service server executes the task to be executed in the last scheduling period according to a historical scheduling mode cached in the target service server in advance; if the target service server does not execute the task to be executed in the last scheduling period, the task to be executed is not executed; and if the target service server executes the task to be executed in the last scheduling period, executing the task to be executed.

The second determining unit 703 is configured to determine whether to reformulate a scheduling scheme of the task to be executed based on states of all the service servers in the service server cluster if the registration center is available, where the scheduling scheme is used to designate a service server in the service server cluster that needs to execute the task to be executed.

In a specific implementation, the second determining unit 703 is specifically configured to: if the registration center is available, determining whether to increase or decrease the service servers in the service server cluster based on the states of all the service servers in the service server cluster; if a service server is newly added or reduced in the service server cluster, determining to reformulate a scheduling scheme of the task to be executed; and if the service servers are not newly added or reduced in the service server cluster, determining not to reformulate the scheduling scheme.

A second processing unit 704, configured to execute the following steps if the scheduling scheme is not re-established; if the scheduling scheme is re-established, initiating a master node election process to the registration center, competing for master nodes with other service servers in the service server cluster, and executing the following steps;

the method comprises the following steps:

judging whether the target service server is a main node or not;

if the target service server is the master node and the scheduling scheme is not re-formulated, executing the task to be executed, and triggering all slave nodes specified by the scheduling scheme to execute the task to be executed, wherein each available slave node corresponds to any service server in the service server cluster;

if the target service server is the master node and the scheduling scheme is re-formulated, formulating a new scheduling scheme according to all available slave nodes in the registration center, executing the task to be executed, and triggering all slave nodes specified by the new scheduling scheme to execute the task to be executed;

and if the target service server is a slave node, receiving a trigger signal sent by the master node and executing the task to be executed.

In a specific implementation, the second processing unit 704 is specifically configured to: if the target service server is the master node and the scheduling scheme is re-formulated, capturing the IP of all available slave nodes in the registration center, and formulating a new scheduling scheme according to a preset algorithm; and executing the task to be executed, and sending a trigger signal to the slave node according to the IP of the slave node aiming at each slave node specified by the new scheduling scheme so as to enable the slave node to execute the task to be executed.

Preferably, in conjunction with the content shown in fig. 7, the second processing unit 704 for initiating the master node election process to the registry comprises: the system comprises a determining module, a first registration module and a second registration module, wherein the execution principle of each module is as follows:

and the determining module is used for initiating a main node election process to the registration center if the scheduling scheme is re-formulated, and determining whether a task distribution mark and task global information of the task to be executed exist in the registration center.

And the first registration module is used for registering the server information of the target service server to the registration center if the registration center has the task distribution mark and/or the task global information, so that the target service server becomes a slave node.

And the second registration module is used for setting the task distribution mark and registering the task global information to the registration center and registering the server information of the target service server to the registration center if the task distribution mark and the task global information do not exist in the registration center, so that the target service server becomes a main node.

Preferably, in combination with the content shown in fig. 7, the task scheduling device further includes:

the first uploading unit is used for uploading the execution information of the task to be executed to a preset database.

and the second uploading unit is used for uploading the new scheduling scheme to the registration center.

In summary, embodiments of the present invention provide a task scheduling method and apparatus, and in the embodiments of the present invention, when a task to be executed starts to be executed each time, whether a registration center is available is monitored. And when the registration center is unavailable, processing the tasks to be executed by utilizing a pre-cached historical scheduling mode. And when the registration center is available, judging whether the scheduling scheme of the task to be executed needs to be re-formulated. And if the master node does not need to re-formulate the scheduling scheme, executing the task to be executed and triggering other slave nodes to execute the task to be executed. If the scheduling scheme needs to be re-formulated, a master node election process is initiated to the registration center. And if the master node becomes the master node after the master node election process is initiated, formulating a new scheduling scheme, executing the tasks to be executed and triggering all slave nodes appointed by the new scheduling scheme to execute the tasks to be executed. And if the scheduling scheme does not need to be re-formulated and the master node is a slave node, or if the master node becomes a slave node after initiating the master node election process, receiving a trigger signal sent by the master node, and executing the task to be executed. The dependence on the registration center is reduced, and the stability and the usability of the task scheduling system are improved.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A task scheduling method is characterized in that the method is suitable for a target service server, the target service server is any service server in a service server cluster, and the method comprises the following steps:

if the scheduling scheme is not re-established, executing the following steps;

the following steps include:

judging whether the target service server is a main node or not;

2. The method according to claim 1, wherein if the registry is available, determining whether to reformulate the scheduling scheme for the task to be executed based on the status of all the service servers in the service server cluster, comprises:

3. The method of claim 1, wherein initiating a master node election process to the registration center to election master nodes for other service servers in the service server cluster if the scheduling scheme is reformulated, comprises:

4. The method of claim 1, wherein if the target service server is a master node and the scheduling scheme is re-established, establishing a new scheduling scheme according to all available slave nodes in the registry, executing the task to be executed, and triggering all slave nodes specified by the new scheduling scheme to execute the task to be executed, comprises:

the task to be executed is executed, and the task to be executed is executed,

5. The method of claim 1, wherein determining whether a registry is available each time a pre-configured task to be performed begins execution comprises:

6. The method of claim 1, wherein if the registry is not available, processing the task to be executed according to a historical scheduling manner cached in the target service server in advance comprises:

7. The method of claim 1, further comprising:

8. The method of claim 1, wherein after formulating a new scheduling scheme based on all available slave nodes in the registry, the method further comprises:

and uploading the new scheduling scheme to the registration center.

9. A task scheduling apparatus, wherein the apparatus is adapted to a target service server, and the target service server is any service server in a service server cluster, the apparatus includes:

the following steps include:

judging whether the target service server is a main node or not;

10. The apparatus according to claim 9, wherein the second determining unit is specifically configured to: if the registration center is available, determining whether to increase or decrease the service servers in the service server cluster based on the states of all the service servers in the service server cluster; if a service server is newly added or reduced in the service server cluster, determining to reformulate the scheduling scheme of the task to be executed; and if the service servers are not newly added or reduced in the service server cluster, determining not to reformulate the scheduling scheme.