CN115629853A - Task scheduling method and device - Google Patents
Task scheduling method and device Download PDFInfo
- Publication number
- CN115629853A CN115629853A CN202211202501.9A CN202211202501A CN115629853A CN 115629853 A CN115629853 A CN 115629853A CN 202211202501 A CN202211202501 A CN 202211202501A CN 115629853 A CN115629853 A CN 115629853A
- Authority
- CN
- China
- Prior art keywords
- execution
- scheduled
- scheduling
- task
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multi Processors (AREA)
Abstract
The invention discloses a method and a device for task scheduling, and relates to the technical field of computers. One embodiment of the method comprises: determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; determining a plurality of available first execution servers corresponding to the service instances according to the connection information; dividing the plurality of first execution servers into a plurality of execution areas so that each execution area includes at least one first execution server; allocating one or more tasks to be scheduled, which respectively correspond to each service instance, to corresponding target execution areas by using a scheduling server; and scheduling one or more first execution servers corresponding to each target execution area, and respectively executing the tasks to be scheduled corresponding to the target servers. The implementation method can enable the first execution server in each execution area to independently execute the corresponding task to be scheduled, and effectively improves the task execution efficiency under the conditions of large task amount and large data amount.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a task scheduling method and device.
Background
In order to ensure the accuracy of execution, a single execution server is usually used for executing tasks in the conventional task scheduling, and although a plurality of execution servers deployed by a cluster can be set, a database lock is also used in the execution process to ensure that only the execution server which has caught the lock is always executing in the execution process. Under the conditions of overlarge task amount and overlarge data amount, the execution efficiency of a single execution server is limited, the requirements of users cannot be met, and the condition of task accumulation is easily caused.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for task scheduling, in which a plurality of first execution servers are divided into a plurality of execution areas, and a task to be scheduled is allocated to a corresponding execution area, so that the first execution servers in each execution area can independently execute the corresponding task to be scheduled, and when a large amount of tasks and a large amount of data are encountered, the execution efficiency of the task is effectively improved. Meanwhile, under the condition that the first servers in the multiple execution areas execute together, the accuracy of task execution can still be ensured, and the situation of task repetition or omission can not occur.
To achieve the above object, according to an aspect of an embodiment of the present invention, a method for task scheduling is provided.
The task scheduling method of the embodiment of the invention comprises the following steps: determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; determining a plurality of available first execution servers corresponding to the service instances according to the connection information; dividing a plurality of the first execution servers into a plurality of execution areas so that each execution area includes at least one of the first execution servers; distributing one or more tasks to be scheduled, which respectively correspond to the service instances, to corresponding target execution areas by using a scheduling server; and scheduling one or more first execution servers corresponding to the target execution areas, and respectively executing tasks to be scheduled corresponding to the target servers.
Optionally, the method further comprises: monitoring the connection information of a plurality of second execution servers corresponding to the service instances in real time; and determining a second execution server corresponding to the connection information indicating normal connection as the first execution server.
Optionally, the allocating, by the scheduling server, one or more to-be-scheduled tasks respectively corresponding to the service instances to corresponding target execution areas includes: and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the main key fields corresponding to the tasks to be scheduled respectively and the quantity of the execution areas.
Optionally, when the scheduling task corresponds to multiple primary key fields, determining, according to the primary key fields and the number of execution areas corresponding to the tasks to be scheduled, target execution areas corresponding to one or more scheduling tasks, respectively includes: and splicing the field values corresponding to the plurality of main key fields, and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the splicing result and the number of the execution areas.
Optionally, the determining, according to the splicing result and the number of the execution areas, target execution areas corresponding to one or more tasks to be scheduled respectively includes: performing hash calculation on the primary key field to obtain a hash value; the hash value is subjected to remainder according to the number of the execution areas; and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the residue-taking result.
Optionally, the allocating, by the scheduling server, one or more to-be-scheduled tasks respectively corresponding to the respective project instances to corresponding target execution areas includes: determining the server utilization rate of each execution area; and according to the utilization rate of the server and a load balancing mode, distributing one or more tasks to be scheduled respectively corresponding to the project instances to corresponding target execution areas.
Optionally, the scheduling one or more first execution servers corresponding to the target execution areas, respectively executing tasks to be scheduled corresponding to the target servers, includes: scheduling the first execution server according to the execution time of the task to be scheduled, so as to execute the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled; the execution time and the execution method are configured when the task to be scheduled is created.
Optionally, the scheduling the first execution server according to the execution time of the task to be scheduled, so as to execute the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled, including: creating a plurality of scheduling queues and distributing a plurality of tasks to be scheduled to each scheduling queue; and respectively acquiring a first scheduling task in each scheduling queue through multithreading, and scheduling the first execution server according to the execution time corresponding to the first scheduling task so as to execute the task to be scheduled corresponding to the target server according to the execution method.
Optionally, in a case that there are multiple scheduling servers, the allocating, by the scheduling server, one or more to-be-scheduled tasks respectively corresponding to the respective project instances to corresponding target execution regions, further includes: and determining a target scheduling server from the plurality of scheduling servers by using the distributed lock, so that the target scheduling server allocates one or more to-be-scheduled tasks respectively corresponding to the project instances to corresponding target execution areas.
Optionally, the method further comprises: recording the execution state of each scheduling task; when the execution state indicates that the execution fails, rescheduling the first execution server corresponding to the target execution area according to a preset period, and executing the task to be scheduled; and/or replacing the first execution servers corresponding to other target execution areas to execute the tasks to be scheduled; and/or skipping the task to be scheduled and determining the task to be scheduled as an abnormal task.
Optionally, the method further comprises: receiving a query request, wherein the query request indicates an item instance code to be queried; determining a corresponding item example to be queried and one or more scheduling tasks to be queried corresponding to the item example according to the item example code; acquiring execution information corresponding to each scheduling task to be queried; and responding the query request according to each scheduling task to be queried and the execution information respectively corresponding to each scheduling task to be queried.
To achieve the above object, according to still another aspect of the embodiments of the present invention, an apparatus for task scheduling is provided.
The task scheduling device of the embodiment of the invention comprises: the system comprises a determining module, a scheduling module and a scheduling module, wherein the determining module is used for determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; determining a plurality of available first execution servers corresponding to the service instances according to the connection information; a dividing module, configured to divide the plurality of first execution servers into a plurality of execution areas, so that each execution area includes at least one first execution server; the distribution module is used for distributing one or more tasks to be scheduled, which respectively correspond to the service instances, to corresponding target execution areas by utilizing the scheduling server; and the scheduling module is used for scheduling one or more first execution servers corresponding to the target execution areas and respectively executing tasks to be scheduled corresponding to the target servers.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an electronic device for task scheduling.
An electronic device for task scheduling according to an embodiment of the present invention includes: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the task scheduling method of the embodiment of the invention.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a computer-readable storage medium.
A computer-readable storage medium of an embodiment of the present invention stores thereon a computer program that, when executed by a processor, implements a method of task scheduling of an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: the plurality of first execution servers are divided into the plurality of execution areas, and the tasks to be scheduled are distributed to the corresponding execution areas, so that the first execution servers in the execution areas can independently execute the corresponding tasks to be scheduled, and the task execution efficiency is effectively improved under the conditions of large task amount and large data amount. Meanwhile, under the condition that the first servers in the multiple execution areas execute together, the accuracy of task execution can still be ensured, and the situation of task repetition or omission can not occur.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a flow diagram illustrating a method of task scheduling according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating the determination of a first execution server according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a process of determining target execution areas corresponding to one or more tasks to be scheduled, respectively, according to an embodiment of the present invention;
FIG. 4 is a schematic flowchart of another method for determining target execution areas corresponding to one or more tasks to be scheduled, respectively, according to an embodiment of the present invention;
FIG. 5 is a schematic flowchart illustrating another method for determining target execution areas corresponding to one or more tasks to be scheduled, respectively, according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating scheduling of tasks to be scheduled according to an embodiment of the present invention;
FIG. 7 is a flow diagram for a scheduling failure case according to an embodiment of the present invention;
FIG. 8 is a flowchart illustrating querying a task to be scheduled, according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of the main modules of an apparatus for task scheduling according to an embodiment of the present invention;
FIG. 10 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 11 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention and the technical features of the embodiments may be combined with each other without conflict.
Fig. 1 is a schematic diagram of the main steps of a method of task scheduling according to an embodiment of the present invention.
As shown in fig. 1, the method for task scheduling according to the embodiment of the present invention mainly includes the following steps:
step S101: determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance;
step S102: determining a plurality of available first execution servers corresponding to the service instances according to the connection information;
step S103: dividing the plurality of first execution servers into a plurality of execution areas so that each execution area includes at least one first execution server;
step S104: allocating one or more tasks to be scheduled, which respectively correspond to each service instance, to corresponding target execution areas by using a scheduling server;
step S105: and scheduling one or more first execution servers corresponding to each target execution area, and respectively executing the tasks to be scheduled corresponding to the target servers.
The service instance may be understood as different services or different projects corresponding to each task to be scheduled, and the connection information may include: the server address of the connection, and the operating conditions of the server. In general, before step S101, service instances need to be registered, and multiple tasks to be scheduled corresponding to the same service instance are registered in the same directory, so as to facilitate execution of subsequent tasks. And each service instance corresponds to the server address of the registration connection, and the second execution server corresponding to each service instance can be obtained according to the registered server address. In an alternative embodiment, the determining the first execution server from the plurality of second execution servers according to the connection state in the connection information, specifically as shown in fig. 2, includes:
step S201: monitoring the connection information of a plurality of second execution servers corresponding to the service instances in real time;
step S202: and determining the second execution server corresponding to the connection information indicating that the connection is normal as the first execution server.
For the process of dividing the plurality of first execution servers into a plurality of execution areas in step S103, in an alternative embodiment, each first execution server may be used as an execution area; the number of execution zones may also be determined based on the number of service instances, where each execution zone corresponds to a different service instance.
In a further optional embodiment, the number of the first execution servers in each execution area may be the same or different, that is, the first execution servers may be divided equally, or may be divided according to the number of the tasks to be scheduled of the service instance corresponding to each execution area. For example, there are 13 available first execution servers, and there are 1300 tasks to be scheduled, where the tasks to be scheduled may be divided into 3 service instances: the service 1, the service 2 and the service 3 correspond to different task numbers under each service instance, that is, the service 1 corresponds to 100 tasks to be scheduled, the service 2 corresponds to 200 tasks to be scheduled, and the service 3 corresponds to 1000 tasks to be scheduled. Then the 13 available first execution servers need to be divided into 3 execution zones corresponding to the service instance: the system comprises an execution area A, an execution area B and an execution area C, wherein the execution area A is used for executing 100 tasks to be scheduled corresponding to the service 1, the execution area B is used for executing 200 tasks to be scheduled corresponding to the service 2, and the execution area C is used for executing 1000 tasks to be scheduled corresponding to the service 3. For the number of first actuators in each execution region, the ratio of the number of first actuators in each execution region may be as follows, 100:200:1000, 1 first execution server is set in the execution area a, 2 first execution servers are set in the execution area B, and 10 first execution servers are set in the execution area C.
For step S103, in an alternative embodiment, the target execution areas corresponding to one or more tasks to be scheduled may be determined according to the number of the execution areas and the main key fields corresponding to the tasks to be scheduled.
Wherein, the primary key field may include: in a further optional embodiment, field values corresponding to a plurality of main key fields may be spliced, and target execution areas corresponding to one or more tasks to be scheduled respectively are determined according to a splicing result and the number of execution areas.
In a further alternative embodiment, as shown in fig. 3, the method specifically includes:
step S301: performing hash calculation on the splicing result to obtain a hash value;
step S302: taking the remainder of the hash value according to the number of the execution areas;
step S303: and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the residue-taking result.
The hash calculation is carried out on the splicing result, and the residue is taken from the hash value according to the number of the execution areas, so that the tasks to be scheduled can be evenly distributed to the execution areas, and the condition that some execution areas are idle and some execution areas have tasks accumulated is avoided.
In addition to the foregoing allocation method, in an optional embodiment, the splicing result may also be converted into a binary string, and according to the similarity between the binary strings, the task to be scheduled corresponding to the binary string whose similarity satisfies the preset condition is allocated to the same execution area. The similarity between binary strings can be determined by cosine similarity, matrix similarity and string edit distance similarity.
In addition, the splicing result may also be clustered, and one or more target execution regions respectively corresponding to the tasks to be scheduled are determined according to the clustering result, as shown in fig. 4, the method includes:
step S401: clustering the splicing result by adopting a DBSCAN algorithm to obtain a clustering result;
step S402: and according to the clustering result, distributing the tasks to be scheduled corresponding to the splicing result belonging to the same clustering cluster to the same execution area, and taking the distributed execution area as a target execution area corresponding to the tasks to be scheduled.
In another optional embodiment, a process of allocating, by using a scheduling server, one or more tasks to be scheduled respectively corresponding to the item instances to corresponding target execution areas may further include, as shown in fig. 5:
step S501: determining the server utilization rate of each execution area;
step S502: and according to the utilization rate of the server and a load balancing mode, distributing one or more tasks to be scheduled respectively corresponding to the project instances to corresponding target execution areas.
By the aid of the methods, the corresponding target execution areas can be respectively determined for the tasks to be scheduled, the different tasks to be scheduled can be executed synchronously by the aid of the target execution areas, and processing efficiency of the tasks can be effectively improved.
It should be noted that, in the embodiment of the present invention, a scheduling server may also perform cluster deployment, but it needs to ensure that only one scheduling server performs scheduling at the same time point, so in an optional embodiment, in the case that there are multiple scheduling servers, the method further includes: and determining a target scheduling server from the scheduling servers by using the distributed lock, so that the target scheduling server allocates one or more to-be-scheduled tasks respectively corresponding to the project instances to the corresponding target execution areas.
For step S105, in an alternative embodiment, the method may include: scheduling the first execution server according to the execution time of the task to be scheduled so as to execute the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled; the execution time and the execution method are configured when the task to be scheduled is created. In a further optional embodiment, after creating the task to be scheduled, before step S101, the method may further include: and judging whether the task quantity of the task to be scheduled is larger than a preset threshold value, and if so, executing the step S101 to the step S105. When the task amount is smaller, the partition function does not need to be started, and a single execution server is determined to be used as the first execution server through the database lock. Therefore, when unnecessary partitioning is performed, energy consumption is saved, and when the backlog risk exists in the task to be processed, reasonable partitioning is performed, so that the overall task processing efficiency is improved.
In a further optional embodiment, the scheduling the first execution server according to the execution time of the task to be scheduled, so as to execute the process of the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled, as shown in fig. 6, includes:
step S601: creating a plurality of scheduling queues and distributing a plurality of tasks to be scheduled to each scheduling queue;
step S602: and respectively acquiring a first scheduling task in each scheduling queue through multithreading, and scheduling the first execution server according to the execution time corresponding to the first scheduling task so as to execute the task to be scheduled corresponding to the target server according to the execution method.
By constructing a plurality of scheduling queues, the asynchronous processing process can be realized, so that the tasks to be scheduled are processed asynchronously through the plurality of scheduling queues and are processed orderly according to the arrangement sequence in each scheduling queue, and the processing efficiency is further improved.
In an alternative embodiment, in the process of scheduling the task to be scheduled, a scheduling failure may occur, and for the scheduling failure, as shown in fig. 7, the scheduling failure includes:
step S701: recording the execution state of each task to be scheduled;
step S702: under the condition that the execution state indicates that the execution fails, rescheduling the first execution server corresponding to the target execution area according to a preset period, and executing the task to be scheduled; and/or replacing the first execution servers corresponding to other target execution areas to execute the tasks to be scheduled; and/or skipping the task to be scheduled and determining the task to be scheduled as an abnormal task.
By recording the execution state of each task to be scheduled, a user can know the processing condition of each task to be scheduled conveniently, and whether the task needs to be executed again is judged according to the task grade and the like, so that the processing efficiency of the task is further improved. In some application scenarios, there are some tasks that are less important, and for these tasks, there is no need to repeat execution all the time, or the first server resource occupying other target execution area executes again, and the task can be skipped directly. In some scenarios, a logical relationship of front and back execution exists between tasks to be scheduled, and if the task a to be scheduled fails to execute, the task B to be scheduled may be affected to execute.
After recording the execution state of the task to be scheduled, the user may conveniently perform query, in an optional embodiment, as shown in fig. 8, the method further includes:
step S801: receiving a query request, wherein the query request indicates the item instance code to be queried;
step S802: determining a corresponding item example to be queried and one or more scheduling tasks to be queried corresponding to the item example according to the item example code;
step S803: acquiring execution information corresponding to each scheduling task to be queried;
step S804: and responding the query request according to each scheduling task to be queried and the execution information respectively corresponding to each scheduling task to be queried.
Wherein the execution information may include: execution state, execution time, execution node and the like, so that the inquirer can obtain more comprehensive inquiry results and process subsequent data conveniently.
According to the task scheduling method, the plurality of first execution servers are divided into the plurality of execution areas, and the tasks to be scheduled are distributed to the corresponding execution areas, so that the first execution servers in the execution areas can independently execute the corresponding tasks to be scheduled, and the task execution efficiency is effectively improved under the conditions of large task amount and large data amount. Meanwhile, under the condition that the first servers in the multiple execution areas execute together, the accuracy of task execution can still be ensured, and the situation of task repetition or omission can not occur.
Fig. 9 is a schematic diagram of main blocks of an apparatus for task scheduling according to an embodiment of the present invention.
As shown in fig. 9, an apparatus 900 for task scheduling according to an embodiment of the present invention includes:
a determining module 901, configured to determine a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; determining a plurality of available first execution servers corresponding to the service instances according to the connection information;
a dividing module 902, configured to divide the plurality of first execution servers into a plurality of execution areas, so that each of the execution areas includes at least one of the first execution servers;
an allocating module 903, configured to allocate, by using a scheduling server, one or more to-be-scheduled tasks respectively corresponding to the service instances to corresponding target execution areas;
and a scheduling module 904, configured to schedule one or more first execution servers corresponding to the target execution regions, and respectively execute the tasks to be scheduled corresponding to the target servers.
In an alternative embodiment of the present invention, the apparatus 900 further comprises: the monitoring module is used for monitoring the connection information of the second execution servers corresponding to the service instances in real time; and determining a second execution server corresponding to the connection information indicating normal connection as the first execution server.
In an optional embodiment of the present invention, the allocating module 903 is further configured to determine, according to the number of the execution areas and the primary key fields corresponding to the tasks to be scheduled, target execution areas corresponding to one or more tasks to be scheduled.
In an optional embodiment of the present invention, the allocating module 903 is further configured to, in a case that the scheduling task corresponds to multiple primary key fields, splice field values corresponding to the multiple primary key fields, and determine, according to a splicing result and the number of the execution areas, target execution areas corresponding to one or more tasks to be scheduled, respectively.
In an optional embodiment of the present invention, the allocating module 903 is further configured to perform hash calculation on the splicing result to obtain a hash value; the hash value is subjected to remainder according to the number of the execution areas; and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the residue-taking result.
In an optional embodiment of the present invention, the allocating module 903 is further configured to determine a server utilization rate of each execution area; and according to the utilization rate of the server and a load balancing mode, distributing one or more tasks to be scheduled respectively corresponding to the project instances to corresponding target execution areas.
In an optional embodiment of the present invention, the scheduling module 904 is further configured to schedule the first execution server according to the execution time of the task to be scheduled, so as to execute the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled; the execution time and the execution method are configured when the task to be scheduled is created.
In an optional embodiment of the present invention, the scheduling module 904 is further configured to create a plurality of scheduling queues, and allocate a plurality of tasks to be scheduled to each of the scheduling queues; and respectively acquiring a first scheduling task in each scheduling queue through multithreading, and scheduling the first execution server according to the execution time corresponding to the first scheduling task so as to execute the task to be scheduled corresponding to the target server according to the execution method.
In an optional embodiment of the present invention, the allocating module 903 is further configured to, in a case that there are multiple scheduling servers, determine, by using a distributed lock, a target scheduling server from the multiple scheduling servers, so that the target scheduling server allocates one or more to-be-scheduled tasks respectively corresponding to the respective project instances to corresponding target execution areas.
In an optional embodiment of the present invention, the apparatus 900 further includes a query module, configured to receive a query request, where the query request indicates an item instance code to be queried; determining a corresponding item example to be queried and one or more scheduling tasks to be queried corresponding to the item example according to the item example code; acquiring execution information corresponding to each scheduling task to be inquired; and responding the query request according to each scheduling task to be queried and the execution information respectively corresponding to each scheduling task to be queried.
According to the task scheduling device provided by the embodiment of the invention, the plurality of first execution servers are divided into the plurality of execution areas, and the tasks to be scheduled are distributed to the corresponding execution areas, so that the first execution servers in each execution area can independently execute the corresponding tasks to be scheduled, and the task execution efficiency is effectively improved under the conditions of large task amount and large data amount. Meanwhile, under the condition that the first servers in the multiple execution areas execute together, the accuracy of task execution can still be ensured, and the situation of task repetition or omission can not occur.
Fig. 10 shows an exemplary system architecture 1000 of a device to which the method of task scheduling or task scheduling of an embodiment of the present invention may be applied.
As shown in fig. 10, the system architecture 1000 may include terminal devices 1001, 1002, 1003, a network 1004, a scheduling server 1005, and an execution server 1006. The network 1004 is used to provide a medium for communication links between the terminal devices 1001, 1002, 1003 and the scheduling server 1005, and between the scheduling server 1005 and the execution server 1006. Network 1004 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user can interact with a server 1005 via a network 1004 using terminal devices 1001, 1002, 1003 to receive or transmit data or the like. Various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like, may be installed on the terminal devices 1001, 1002, and 1003.
The terminal devices 1001, 1002, 1003 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 1005 may be a server that provides various services, such as a background management server that provides support for a query request transmitted by a user using the terminal apparatuses 1001, 1002, 1003. The backend management server may analyze and otherwise process the received data such as the query request, and feed back a processing result (e.g., execution information) to the terminal device.
It should be understood that the number of terminal devices, networks, and servers in fig. 10 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 11, shown is a block diagram of a computer system 1100 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 11, the computer system 1100 includes a Central Processing Unit (CPU) 1101 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. In the RAM 1103, various programs and data necessary for the operation of the system 1100 are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other by a bus 1104. An input/output (I/O) first interface 1105 is also connected to bus 1104.
The following components are connected to the I/O first interface 1105: an input portion 1106 including a keyboard, mouse, and the like; an output portion 1107 including a signal output unit such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 1108 including a hard disk and the like; and a communication section 1109 including a network first interface card such as a LAN card, a modem, or the like. The communication section 1109 performs communication processing via a network such as the internet. The driver 1110 is also connected to the I/O first interface 1105 as needed. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is mounted into the storage section 1108 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 1109 and/or installed from the removable medium 1111. The above-described functions defined in the system of the present invention are executed when the computer program is executed by a Central Processing Unit (CPU) 1101.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a determination module, a partitioning module, an assignment module, and a scheduling module. The names of the modules do not form a limitation on the modules themselves under certain conditions, for example, the determining module may also be described as "determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; and determining a plurality of available modules of the first execution server corresponding to the service instance according to the connection information.
As another aspect, the present invention also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; determining a plurality of available first execution servers corresponding to the service instances according to the connection information; dividing a plurality of the first execution servers into a plurality of execution areas so that each of the execution areas includes at least one of the first execution servers; distributing one or more tasks to be scheduled, which respectively correspond to the service instances, to corresponding target execution areas by using a scheduling server; and scheduling one or more first execution servers corresponding to the target execution areas, and respectively executing tasks to be scheduled corresponding to the target servers.
According to the technical scheme of the embodiment of the invention, the plurality of first execution servers are divided into the plurality of execution areas, and the tasks to be scheduled are distributed to the corresponding execution areas, so that the first execution servers in each execution area can independently execute the corresponding tasks to be scheduled, and the task execution efficiency is effectively improved under the conditions of large task amount and large data amount. Meanwhile, under the condition that the first servers in the multiple execution areas execute together, the accuracy of task execution can still be ensured, and the situation of task repetition or omission can not occur.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (14)
1. A method of task scheduling, comprising:
determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance;
determining a plurality of available first execution servers corresponding to the service instances according to the connection information;
dividing a plurality of the first execution servers into a plurality of execution areas so that each of the execution areas includes at least one of the first execution servers;
distributing one or more tasks to be scheduled, which respectively correspond to the service instances, to corresponding target execution areas by using a scheduling server;
and scheduling one or more first execution servers corresponding to the target execution areas, and respectively executing tasks to be scheduled corresponding to the target servers.
2. The method of claim 1, further comprising:
monitoring the connection information of a plurality of second execution servers corresponding to the service instances in real time;
and determining a second execution server corresponding to the connection information indicating that the connection is normal as the first execution server.
3. The method according to claim 1, wherein the allocating, by the scheduling server, one or more tasks to be scheduled respectively corresponding to the service instances to corresponding target execution areas comprises:
and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the main key fields corresponding to the tasks to be scheduled respectively and the quantity of the execution areas.
4. The method according to claim 3, wherein, in a case that the scheduling task corresponds to a plurality of primary key fields, the determining, according to the primary key field corresponding to each of the tasks to be scheduled and the number of the execution areas, a target execution area corresponding to each of one or more scheduling tasks includes:
and splicing the field values corresponding to the plurality of main key fields, and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the splicing result and the number of the execution areas.
5. The method according to claim 4, wherein the splicing field values corresponding to the plurality of main key fields and determining target execution areas corresponding to one or more tasks to be scheduled respectively according to the splicing result and the number of the execution areas comprises:
performing hash calculation on the splicing result to obtain a hash value;
the hash value is subjected to remainder according to the number of the execution areas;
and determining one or more target execution areas corresponding to the tasks to be scheduled respectively according to the residue-taking result.
6. The method according to claim 1, wherein the allocating, by the scheduling server, one or more tasks to be scheduled respectively corresponding to the respective project instances to the corresponding target execution areas comprises:
determining the server utilization rate of each execution area;
and according to the utilization rate of the server and a load balancing mode, distributing one or more tasks to be scheduled respectively corresponding to the project instances to corresponding target execution areas.
7. The method according to claim 1, wherein the scheduling one or more first execution servers corresponding to the target execution areas to respectively execute the tasks to be scheduled corresponding to the target servers comprises:
scheduling the first execution server according to the execution time of the task to be scheduled, so as to execute the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled; the execution time and the execution method are configured when the task to be scheduled is created.
8. The method according to claim 7, wherein the scheduling the first execution server according to the execution time of the task to be scheduled, so as to execute the task to be scheduled corresponding to the target server according to the execution method of the task to be scheduled, includes:
creating a plurality of scheduling queues and distributing a plurality of tasks to be scheduled to each scheduling queue;
and respectively acquiring a first scheduling task in each scheduling queue through multithreading, and scheduling the first execution server according to the execution time corresponding to the first scheduling task so as to execute the task to be scheduled corresponding to the target server according to the execution method.
9. The method according to claim 1, wherein in a case that there are a plurality of scheduling servers, the utilizing scheduling server allocates one or more tasks to be scheduled respectively corresponding to the respective project instances to corresponding target execution areas, further comprising:
and determining a target scheduling server from the plurality of scheduling servers by using the distributed lock, so that the target scheduling server allocates one or more to-be-scheduled tasks respectively corresponding to the project instances to corresponding target execution areas.
10. The method of claim 1, further comprising:
recording the execution state of each task to be scheduled;
in the case where the execution status is to indicate an execution failure,
rescheduling the first execution server corresponding to the target execution area according to a preset period, and executing the task to be scheduled; and/or replacing the first execution servers corresponding to other target execution areas to execute the tasks to be scheduled; and/or skipping the task to be scheduled, and determining the task to be scheduled as an abnormal task.
11. The method of claim 1, further comprising:
receiving a query request, wherein the query request indicates an item instance code to be queried;
determining a corresponding item example to be queried and one or more scheduling tasks to be queried corresponding to the item example according to the item example code;
acquiring execution information corresponding to each scheduling task to be inquired;
and responding the query request according to each scheduling task to be queried and the execution information respectively corresponding to each scheduling task to be queried.
12. An apparatus for task scheduling, comprising:
the system comprises a determining module, a scheduling module and a scheduling module, wherein the determining module is used for determining a service instance corresponding to a task to be scheduled and connection information corresponding to the service instance; determining a plurality of available first execution servers corresponding to the service instances according to the connection information;
a dividing module, configured to divide the plurality of first execution servers into a plurality of execution areas, so that each execution area includes at least one first execution server;
the distribution module is used for distributing one or more tasks to be scheduled, which respectively correspond to the service instances, to corresponding target execution areas by utilizing the scheduling server;
and the scheduling module is used for scheduling one or more first execution servers corresponding to the target execution areas and respectively executing tasks to be scheduled corresponding to the target servers.
13. An electronic device for task scheduling, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-11.
14. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211202501.9A CN115629853A (en) | 2022-09-29 | 2022-09-29 | Task scheduling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211202501.9A CN115629853A (en) | 2022-09-29 | 2022-09-29 | Task scheduling method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115629853A true CN115629853A (en) | 2023-01-20 |
Family
ID=84903979
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211202501.9A Pending CN115629853A (en) | 2022-09-29 | 2022-09-29 | Task scheduling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115629853A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115834714A (en) * | 2023-02-09 | 2023-03-21 | 中国证券登记结算有限责任公司 | Cross-platform task scheduling method, server and system |
-
2022
- 2022-09-29 CN CN202211202501.9A patent/CN115629853A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115834714A (en) * | 2023-02-09 | 2023-03-21 | 中国证券登记结算有限责任公司 | Cross-platform task scheduling method, server and system |
| CN115834714B (en) * | 2023-02-09 | 2023-06-16 | 中国证券登记结算有限责任公司 | Cross-platform task scheduling method, server and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109408205B (en) | Task scheduling method and device based on hadoop cluster | |
| US10108458B2 (en) | System and method for scheduling jobs in distributed datacenters | |
| US9489231B2 (en) | Selecting provisioning targets for new virtual machine instances | |
| CN108572875B (en) | Resource allocation method, device and system | |
| US8869149B2 (en) | Concurrency identification for processing of multistage workflows | |
| CN112114950A (en) | Task scheduling method and device and cluster management system | |
| CN110166507B (en) | Multi-resource scheduling method and device | |
| US20140201371A1 (en) | Balancing the allocation of virtual machines in cloud systems | |
| CN109614227A (en) | Task resource concocting method, device, electronic equipment and computer-readable medium | |
| US11418583B2 (en) | Transaction process management by dynamic transaction aggregation | |
| CN109428926B (en) | Method and device for scheduling task nodes | |
| CN116166395A (en) | Task scheduling method, device, medium and electronic equipment | |
| CN115629853A (en) | Task scheduling method and device | |
| CN114116173A (en) | Method, device and system for dynamically adjusting task allocation | |
| US8316367B2 (en) | System and method for optimizing batch resource allocation | |
| US9577869B2 (en) | Collaborative method and system to balance workload distribution | |
| CN112559179A (en) | Job processing method and device | |
| CN113760488A (en) | Method, device, equipment and computer readable medium for scheduling task | |
| CN116881003A (en) | Resource allocation method, device, service equipment and storage medium | |
| CN113051245A (en) | Method, device and system for migrating data | |
| CN115170026A (en) | Task processing method and device | |
| CN108683608B (en) | Method and device for distributing flow | |
| CN114237902A (en) | Service deployment method and device, electronic equipment and computer readable medium | |
| CN114169733A (en) | Resource allocation method and device | |
| GB2504812A (en) | Load balancing in a SAP (RTM) system for processors allocated to data intervals based on system load |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |