CN112988340A - Task scheduling method, device and system - Google Patents
Task scheduling method, device and system Download PDFInfo
- Publication number
- CN112988340A CN112988340A CN201911307958.4A CN201911307958A CN112988340A CN 112988340 A CN112988340 A CN 112988340A CN 201911307958 A CN201911307958 A CN 201911307958A CN 112988340 A CN112988340 A CN 112988340A
- Authority
- CN
- China
- Prior art keywords
- target
- instance
- task
- scheduling
- pipeline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multi Processors (AREA)
Abstract
The invention provides a task scheduling method, a device and a system, which select a target pipeline example for executing a target service; analyzing the target pipeline example, and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example; and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance. The invention can select the target pipeline instance for executing the target service according to the target service, thereby obtaining the target task instance forming the target pipeline instance from the target pipeline instance, and scheduling the target task instance to the corresponding target running resource for running according to the scheduling mode corresponding to the target task instance, thereby realizing the unified scheduling of different task instances and improving the task scheduling efficiency.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a task scheduling method, device and system.
Background
In the technical field of distributed computers, tasks are basic working units completed by computers, and are one or more instruction sequences processed by a control program, the tasks are divided into a plurality of different types according to different execution modes of the instruction sequences, the tasks of different types are split by different computing frames along with the development of technologies such as distributed computing and cloud computing, each type of task independently corresponds to one scheduling mode, and the scheduling modes of each type of task have isolation, namely, a user needs to switch among the scheduling modes of different types of tasks in the process of scheduling the tasks of different types, so that the task scheduling efficiency is seriously influenced.
Therefore, how to provide a unified task scheduling mode and improve the task scheduling efficiency becomes a technical problem to be solved at present.
Disclosure of Invention
In view of this, the present invention provides a method, an apparatus, and a system for task scheduling to provide a unified task scheduling manner and improve task scheduling efficiency.
In order to achieve the purpose, the invention provides the following technical scheme:
a method of task scheduling, the method comprising:
selecting a target pipeline example for executing the target service;
analyzing the target pipeline example, and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example;
and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance.
Preferably, in the case that the target service includes a plurality of services, the selecting the target pipeline instance for executing the target service includes:
selecting a target pipeline example corresponding to each target service from the preset corresponding relation between the service and the pipeline example;
and according to the priority sequence of each target service, performing priority arrangement on each target pipeline instance to obtain a target pipeline instance queue, wherein the target pipeline instance queue comprises a plurality of target pipeline instances.
Preferably, the analyzing the target pipeline instance, and the obtaining the target task instance, the scheduling manner corresponding to the target task instance, and the target running resource corresponding to the target task instance from the target pipeline instance includes:
and analyzing each target pipeline instance in each target pipeline instance queue in sequence according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, and acquiring the target task instance contained in each target pipeline instance, the scheduling mode corresponding to the target task instance and the target running resource corresponding to the target task instance.
Preferably, when a plurality of target task instances are obtained from one target pipeline instance, the scheduling the target task instances to corresponding target running resources according to the scheduling manner corresponding to the target task instances includes:
allocating each target task instance to a task execution interface according to at least one of a running mode corresponding to each target task instance, a dependency relationship among different target task instances and a priority order of different target task instances;
and at the task execution interface, allocating the target task instances to the corresponding target running resources for running.
Preferably, before scheduling the target task instance to the corresponding target running resource for running according to the scheduling manner corresponding to the target task instance, the method further includes:
and analyzing the target pipeline example to obtain the scheduling mode of the target pipeline example.
Preferably, the scheduling the target task instance to the corresponding target running resource according to the scheduling manner corresponding to the target task instance comprises:
and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target pipeline instance and the scheduling mode corresponding to the target task instance.
A task scheduling apparatus, the apparatus comprising:
a target pipeline instance selection unit for selecting a target pipeline instance for executing the target service;
the analysis unit is used for analyzing the target pipeline example and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example;
and the task scheduling unit is used for scheduling the target task instances to the corresponding target running resources to run according to the scheduling modes corresponding to the target task instances.
Preferably, in a case where the target service includes a plurality of services, the target pipeline instance selecting unit includes:
the target pipeline instance selecting subunit is used for selecting a target pipeline instance corresponding to each target service from the preset corresponding relationship between the services and the pipeline instances;
and the arranging unit is used for arranging the priority of each target pipeline instance according to the priority sequence of each target service to obtain a target pipeline instance queue, and the target pipeline instance queue comprises a plurality of target pipeline instances.
Preferably, the analysis unit includes:
and the analysis subunit is used for sequentially analyzing each target pipeline instance in each target pipeline instance queue according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, and acquiring the target task instance, the scheduling mode corresponding to the target task instance and the target running resource corresponding to the target task instance from each target pipeline instance.
A task scheduling system, the system comprising:
a processor and a memory;
the processor is used for calling and executing the program stored in the memory;
the memory is configured to store the program, the program at least to:
the task scheduling method as described above is performed.
According to the technical scheme, compared with the prior art, the task scheduling method, the task scheduling device and the task scheduling system are characterized in that a target pipeline example for executing target service is selected; analyzing the target pipeline example, and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example; and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance. The embodiment of the invention can select the target production line instance for executing the target service according to the target service, thereby obtaining the target task instance forming the target production line instance from the target production line instance, and scheduling the target task instance to the corresponding target running resource for running according to the scheduling mode corresponding to the target task instance, thereby realizing the unified scheduling of different task instances and improving the task scheduling efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a task scheduling method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another task scheduling method according to an embodiment of the present invention;
fig. 3 is a flowchart of another task scheduling method according to an embodiment of the present invention;
fig. 4 is a block diagram of a task scheduling device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the technical field of distributed computers, tasks are basic working units finished by computers, are one or more instruction sequences processed by a control program, are abstracted from the application of various computing forms from the viewpoint of a set of distributed operating systems, and have diversity in computing application types borne in the task field. From the field division of the interactive mode, tasks are divided into the following types:
on-line computing task type: the method comprises the steps of web human-computer interaction online service tasks with slightly low time delay requirements and high throughput requirements, and the Internet of things field object-object interaction online control tasks with high time delay requirements and throughput requirements;
calculating task types in batches: the method is distributed in the fields with low requirements on time delay and high requirements on throughput, such as tasks of various analysis types, batch statistical analysis reports and the like;
DAG computation task type: the application-oriented task computing has a computing scene of flow dependence, and the tasks have low time delay requirements and extremely high throughput requirements, such as monthly computing tasks in the telecommunication industry, deep learning tasks in the AI field and other types of tasks;
computational model task type of flow: the requirement on time delay is high, and the requirement on throughput is also considered.
Meanwhile, the different task type scheduling modes are different, and the following classifications are provided:
single batch computation task: time-driven scheduling modes, such as various timing tasks; data is imported into a driving scheduling mode, such as various temporary and periodic data uploading to trigger calculation tasks;
DAG computing tasks: the method comprises the following steps of (1) scheduling state dependence among tasks, and scheduling types such as node task presentation time and data drive in a DAG ring;
and (3) streaming computing tasks which depend on scheduling among task data streams, and after the tasks are scheduled and run, data flow and are computed among the tasks in real time.
With the development of technologies such as distributed computing and cloud computing, various computing frameworks are introduced for tasks of different forms and scheduling types of task computing, and these computing frameworks provide corresponding task computing models and scheduling capabilities for certain business type fields, such as MapReduce, spark, flex and various owned computing frameworks. That is to say, at present, different types of tasks are split by different computing frames, each type of task individually corresponds to one scheduling mode, and the scheduling modes of each type of task have isolation, that is, a user needs to switch between the scheduling modes of different types of tasks during the process of scheduling the different types of tasks, which seriously affects the efficiency of task scheduling.
In order to solve the technical problem, the embodiment of the invention discloses a task scheduling method, which comprises the steps of selecting a target pipeline example for executing a target service; analyzing the target pipeline example, and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example; and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance. The embodiment of the invention can select the target production line instance for executing the target service according to the target service, thereby obtaining the target task instance forming the target production line instance from the target production line instance, and scheduling the target task instance to the corresponding target running resource for running according to the scheduling mode corresponding to the target task instance, thereby realizing the unified scheduling of different task instances and improving the task scheduling efficiency.
The following describes a task scheduling method provided in an embodiment of the present invention, and fig. 1 is a flowchart of the task scheduling method provided in the embodiment of the present invention, and referring to fig. 1, the method may include:
s100, selecting a target pipeline example for executing the target service;
it should be noted that, the target pipeline instance is used to execute the target service, and different target services may correspond to different target pipeline instances or correspond to the same target pipeline instance; one target service may correspond to one target pipeline instance, or may correspond to multiple target pipeline instances, and embodiments of the present invention are not limited in particular.
Therefore, the number of target pipeline instances for executing the target service selected in the embodiment of the present invention may be one or multiple.
The target pipeline example is formed by target task examples, at least one target task forms a target pipeline, and the target pipeline is a pipeline of the target task.
Step S110, analyzing the target pipeline instance, and acquiring a target task instance, a scheduling mode corresponding to the target task instance and a target running resource corresponding to the target task instance from the target pipeline instance;
it should be noted that the target pipeline instance in the embodiment of the present invention includes not only the target task instance, but also a scheduling manner corresponding to the target task instance, a target running resource corresponding to the target task instance, and attribute information of the target pipeline instance, and the like, and the embodiment of the present invention is not limited specifically.
Optionally, the same task instance may have different scheduling manners and different target running resources in different pipeline instances, and different task instances may also have the same scheduling manner and the same target running resources, which is not specifically limited in the embodiment of the present invention.
In the embodiment of the present invention, attribute information of a target pipeline instance is stored in a target pipeline instance model, where the attribute information of the target pipeline instance stores a unique identifier (such as a number or a name) of a target pipeline, related information of a target task instance included in the target pipeline (such as a correspondence between a pipeline and a task in one target pipeline instance, at least one target task instance included in one pipeline), a scheduling manner of the target pipeline instance (such as a start execution time, an end execution time, a run period of the target pipeline instance triggered periodically, an entry pipeline trigger manner, and the like of the target pipeline instance), a performance description of the target pipeline instance, an operator corresponding to the target pipeline instance, creation time and modification time of the target pipeline instance, and the like.
Included in the target task instance is information related to the target task: the target task processing method comprises the following steps of unique identification (such as a number or a name) of a target task, the type of the target task (such as a transaction processing task type, an intensive batch computation type, an AI task type and the like), target task grouping information, a target task attributive tenant code, a target task version, a target task processing type, the starting execution time of the target task, the ending execution time of the target task and implementation class related information definition.
The scheduling mode corresponding to the target task instance comprises the following steps: the method comprises the steps of identifying attribute identification (such as a number or a name) of a target task instance scheduling mode, identifying uniqueness (such as a number or a name) of a target task, a fault-tolerant mode, scheduling time of the target task instance, an operator corresponding to the target task instance scheduling mode, creating time and modifying time of the target task instance scheduling mode, scheduling types (such as time period scheduling, state trigger scheduling, manual trigger scheduling and the like), corresponding scheduling periods, task running modes (resident, periodic, real-time starting and the like), task fragmentation setting (splitting distributed node unit number), dependency relationship among different target task instances, priority sequences of different target task instances and the like.
The dependencies between different target task instances are, for example: the method comprises the following steps that a state dependency relationship of a front task and a rear task (a task is finished and b task operation is triggered according to the state of successful operation of a), data transmission dependency between the front task and the rear task (namely the situation that the a task processes a single stroke of data and then flows to a b task to process a pipeline), the front task and the rear task trigger dependency (for example, the former task triggers a certain action event, such as initiating one account checking and one report calculation, and the subsequent task receives the event and then processes), and the mode dependency of the subsequent calculation task is triggered by importing a batch of files on an interface, namely, the task processing depends on external manual operation triggering relationship.
The priority sequence of different target task instances is the execution priority sequence of different target tasks. The priority order of the target task instances determines which target task is dispatched and scheduled first; the task instance marked with the task emergency definition can be preferentially processed by a queue insertion mechanism during task distribution, and is preferentially distributed and scheduled; the task instance marked with the task isolation priority definition supports the task bottom layer to distribute and dispatch to the isolated target running resources, and ensures that the task bottom layer is not influenced by the processing of other tasks.
The target running resource corresponding to the target task instance comprises the related attribute information of the target running resource, and the method comprises the following steps: the unique identification (such as a number or a name) of the target running resource, the resource type (such as cpu, gpu, storage, network and the like) of the target running resource, the unit of the target running resource, the quota of the target running resource and the like.
The corresponding relation between the target pipeline instance and the target task instance comprises the following steps: the method comprises the steps of identifying the uniqueness of the corresponding relation between a target pipeline instance and a target task instance, identifying the uniqueness of the target pipeline instance, identifying the uniqueness of the target task instance, correlating the target pipeline instance with the task before and after, identifying a preposed task, and the like.
The corresponding relation between the target task instance and the target running resource comprises the following steps: the corresponding relation uniqueness identification of the target task instance and the target running resource, the target pipeline instance uniqueness identification, the target task instance uniqueness identification, the target running resource uniqueness identification and the like.
And step S120, scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance.
Under the condition that a plurality of target task instances are obtained from one target pipeline instance, the scheduling the target task instances to corresponding target running resources according to the scheduling mode corresponding to the target task instances comprises:
allocating each target task instance to a task execution interface according to at least one of a running mode corresponding to each target task instance, a dependency relationship among different target task instances and a priority order of different target task instances; and at the task execution interface, allocating the target task instances to the corresponding target running resources for running.
And the resource scheduler issues the target task instance to a specific resource domain (such as a virtual machine cluster, a kubernets, a mess container cluster and the like) for scheduling according to a scheduling mode corresponding to the target task instance from the analyzed target task instance.
And the task scheduling engine is uniformly butted with the resource layer adaptive container and the scheduling interface of the virtual machine, and distributes and schedules the target task instance entering the running sequence to run. Whatever type of resource domain, the task scheduling engine is responsible for adapting to the provided resource scheduling API, and the priority level and other control of the resource domain which are not related to the specific open source technology stack are realized at the task scheduling layer.
The scheduling of the target task instance to the corresponding target running resource may specifically include container scheduling or virtual machine scheduling, which is not specifically limited in the embodiment of the present invention.
The embodiment of the invention selects the target production line example for executing the target service; analyzing the target pipeline example, and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example; and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance. The embodiment of the invention can select the target production line instance for executing the target service according to the target service, thereby obtaining the target task instance forming the target production line instance from the target production line instance, and scheduling the target task instance to the corresponding target running resource for running according to the scheduling mode corresponding to the target task instance, thereby realizing the unified scheduling of different task instances and improving the task scheduling efficiency. And the scheduling of the target pipeline instance and the scheduling of the target task instance are decoupled, so that the flexibility of a scheduling mode is realized.
Another task scheduling method provided in the embodiment of the present invention is described below, where multiple target services exist in the embodiment of the present invention, fig. 2 is a flowchart of the task scheduling method provided in the embodiment of the present invention, and referring to fig. 2, the method may include:
s200, selecting a target pipeline example corresponding to each target service from the preset corresponding relation between the services and the pipeline examples;
each target service may correspond to one target pipeline instance, and may also correspond to multiple target pipeline instances, which is not specifically limited in the embodiments of the present invention.
Step S210, according to the priority sequence of each target service, performing priority arrangement on each target pipeline instance to obtain a target pipeline instance queue, wherein the target pipeline instance queue comprises a plurality of target pipeline instances;
different target services have different priority orders, and the priority order may be determined according to the execution order of the target services, or according to the characteristics of the target services.
If one target service corresponds to a plurality of target pipeline instances, the target service corresponding to the plurality of target pipeline instances can be sequenced according to the priority order of the target pipeline instances.
Step S220, sequentially analyzing each target pipeline instance in each target pipeline instance queue according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, and acquiring target task instances, scheduling modes corresponding to the target task instances and target running resources corresponding to the target task instances from each target pipeline instance;
according to the analysis sequence of the embodiment of the invention, the target pipeline instance queue with high priority is analyzed first and then the target pipeline instance with high priority in the target pipeline instance queue is analyzed by referring to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue.
And step S230, scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance.
The embodiment of the invention selects a target assembly line example corresponding to each target service from the preset corresponding relation between the service and the assembly line example, and arranges the priority of each target assembly line example according to the priority sequence of each target service to obtain a target assembly line example queue, wherein the target assembly line example queue comprises a plurality of target assembly line examples; according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, analyzing each target pipeline instance in each target pipeline instance queue in sequence, acquiring a target task instance contained in each target pipeline instance, a scheduling mode corresponding to the target task instance and a target running resource corresponding to the target task instance from each target pipeline instance, and scheduling the target task instance to the corresponding target running resource for running according to the scheduling mode corresponding to the target task instance. Therefore, under the condition that a plurality of target services exist, different task instances corresponding to the plurality of target services can be uniformly scheduled, and the task scheduling efficiency is improved.
Another task scheduling method provided in the embodiment of the present invention is described below, and fig. 3 is a flowchart of the task scheduling method provided in the embodiment of the present invention, and referring to fig. 3, the method may include:
step S300, selecting a target pipeline example for executing the target service;
step S310, analyzing the target pipeline instance to obtain a scheduling mode of the target pipeline instance;
because the target pipeline instance also has different scheduling modes, the scheduling mode of the target pipeline instance is obtained by analyzing the target pipeline instance.
Step S320, analyzing the target pipeline instance, and acquiring a target task instance, a scheduling mode corresponding to the target task instance and a target running resource corresponding to the target task instance from the target pipeline instance;
it should be noted that, in the embodiment of the present invention, the step S310 and the step S320 are not performed in sequence, and the step S310 may be performed first, and then the step S320 may be performed; step S320 may be executed first, and then step S310 may be executed, or step S310 and step S320 may be executed simultaneously, which is not limited in the embodiment of the present invention.
And step S330, scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target pipeline instance and the scheduling mode corresponding to the target task instance.
In the task scheduling process, the target task instance is scheduled to the corresponding target running resource to run by referring to the scheduling mode corresponding to the target production line instance and the scheduling mode corresponding to the target task instance. The method can enrich the scheduling modes of the tasks, realize the transverse expansion of the scheduling modes, enhance the flexibility and expansibility of the task scheduling modes and improve the task scheduling efficiency.
In the following, the task scheduling device provided in the embodiment of the present invention is introduced, and the task scheduling device described below may be referred to in correspondence with the task scheduling method described above.
Fig. 4 is a block diagram of a task scheduling apparatus according to an embodiment of the present invention, and referring to fig. 4, the task scheduling apparatus may include:
a target pipeline instance selection unit 400 for selecting a target pipeline instance for executing a target service;
the analysis unit 410 is configured to analyze the target pipeline instance, and obtain a target task instance, a scheduling manner corresponding to the target task instance, and a target running resource corresponding to the target task instance from the target pipeline instance;
and the task scheduling unit 420 is configured to schedule the target task instance to a corresponding target running resource for running according to the scheduling manner corresponding to the target task instance.
In the case that the target traffic includes a plurality of traffic, the target pipeline instance selection unit includes:
the target pipeline instance selecting subunit is used for selecting a target pipeline instance corresponding to each target service from the preset corresponding relationship between the services and the pipeline instances;
and the arranging unit is used for arranging the priority of each target pipeline instance according to the priority sequence of each target service to obtain a target pipeline instance queue, and the target pipeline instance queue comprises a plurality of target pipeline instances.
The analysis unit includes:
and the analysis subunit is used for sequentially analyzing each target pipeline instance in each target pipeline instance queue according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, and acquiring the target task instance, the scheduling mode corresponding to the target task instance and the target running resource corresponding to the target task instance from each target pipeline instance.
In the case of obtaining multiple target task instances from one target pipeline instance, the task scheduling unit includes:
the first allocation unit is used for allocating each target task instance to the task execution interface according to at least one of the running mode corresponding to each target task instance, the dependency relationship among different target task instances and the priority order of different target task instances;
and the second allocation unit is used for allocating the target task instances to the corresponding target running resources to run on the task execution interface.
The task scheduling apparatus further includes:
and the target pipeline instance scheduling mode obtaining unit is used for analyzing the target pipeline instance and obtaining the scheduling mode of the target pipeline instance.
The task scheduling unit includes:
and the task scheduling subunit is used for scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target pipeline instance and the scheduling mode corresponding to the target task instance.
The embodiment of the invention also discloses a task scheduling system, which comprises:
a processor and a memory;
the processor is used for calling and executing the program stored in the memory;
the memory is configured to store the program, the program at least to:
the task scheduling method as described above is performed.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for task scheduling, the method comprising:
selecting a target pipeline example for executing the target service;
analyzing the target pipeline example, and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example;
and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target task instance.
2. The method of claim 1, wherein in the case that the target service includes a plurality of services, the selecting the target pipeline instance for executing the target service comprises:
selecting a target pipeline example corresponding to each target service from the preset corresponding relation between the service and the pipeline example;
and according to the priority sequence of each target service, performing priority arrangement on each target pipeline instance to obtain a target pipeline instance queue, wherein the target pipeline instance queue comprises a plurality of target pipeline instances.
3. The method of claim 1, wherein the parsing the target pipeline instance, and the obtaining the target task instance, the scheduling manner corresponding to the target task instance, and the target running resource corresponding to the target task instance from the target pipeline instance comprises:
and analyzing each target pipeline instance in each target pipeline instance queue in sequence according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, and acquiring the target task instance contained in each target pipeline instance, the scheduling mode corresponding to the target task instance and the target running resource corresponding to the target task instance.
4. The method according to claim 1, wherein, in a case that a plurality of target task instances are obtained from one target pipeline instance, said scheduling the target task instances to corresponding target running resources according to the scheduling manner corresponding to the target task instances comprises:
allocating each target task instance to a task execution interface according to at least one of a running mode corresponding to each target task instance, a dependency relationship among different target task instances and a priority order of different target task instances;
and at the task execution interface, allocating the target task instances to the corresponding target running resources for running.
5. The method according to claim 1, wherein before scheduling the target task instance to the corresponding target running resource for running according to the scheduling manner corresponding to the target task instance, further comprising:
and analyzing the target pipeline example to obtain the scheduling mode of the target pipeline example.
6. The method according to claim 5, wherein said scheduling the target task instance to the corresponding target running resource for running according to the scheduling manner corresponding to the target task instance comprises:
and scheduling the target task instance to the corresponding target running resource to run according to the scheduling mode corresponding to the target pipeline instance and the scheduling mode corresponding to the target task instance.
7. A task scheduling apparatus, characterized in that the apparatus comprises:
a target pipeline instance selection unit for selecting a target pipeline instance for executing the target service;
the analysis unit is used for analyzing the target pipeline example and acquiring a target task example, a scheduling mode corresponding to the target task example and a target running resource corresponding to the target task example from the target pipeline example;
and the task scheduling unit is used for scheduling the target task instances to the corresponding target running resources to run according to the scheduling modes corresponding to the target task instances.
8. The apparatus of claim 7, wherein in the case that the target service includes a plurality of services, the target pipeline instance selection unit comprises:
the target pipeline instance selecting subunit is used for selecting a target pipeline instance corresponding to each target service from the preset corresponding relationship between the services and the pipeline instances;
and the arranging unit is used for arranging the priority of each target pipeline instance according to the priority sequence of each target service to obtain a target pipeline instance queue, and the target pipeline instance queue comprises a plurality of target pipeline instances.
9. The apparatus of claim 7, wherein the parsing unit comprises:
and the analysis subunit is used for sequentially analyzing each target pipeline instance in each target pipeline instance queue according to the priority sequence of the target pipeline instance queue and the priority sequence of each target pipeline instance in the target pipeline instance queue, and acquiring the target task instance, the scheduling mode corresponding to the target task instance and the target running resource corresponding to the target task instance from each target pipeline instance.
10. A task scheduling system, the system comprising:
a processor and a memory;
the processor is used for calling and executing the program stored in the memory;
the memory is configured to store the program, the program at least to:
performing a task scheduling method as claimed in any one of the preceding claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911307958.4A CN112988340A (en) | 2019-12-18 | 2019-12-18 | Task scheduling method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911307958.4A CN112988340A (en) | 2019-12-18 | 2019-12-18 | Task scheduling method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112988340A true CN112988340A (en) | 2021-06-18 |
Family
ID=76343853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911307958.4A Pending CN112988340A (en) | 2019-12-18 | 2019-12-18 | Task scheduling method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988340A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114489867A (en) * | 2022-04-19 | 2022-05-13 | 浙江大华技术股份有限公司 | Algorithm module scheduling method, algorithm module scheduling device and readable storage medium |
| CN116483580A (en) * | 2022-09-29 | 2023-07-25 | 陕西震旦纪信息技术有限公司 | System and method for scheduling server computing resources based on Kubernetes |
-
2019
- 2019-12-18 CN CN201911307958.4A patent/CN112988340A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114489867A (en) * | 2022-04-19 | 2022-05-13 | 浙江大华技术股份有限公司 | Algorithm module scheduling method, algorithm module scheduling device and readable storage medium |
| CN116483580A (en) * | 2022-09-29 | 2023-07-25 | 陕西震旦纪信息技术有限公司 | System and method for scheduling server computing resources based on Kubernetes |
| CN116483580B (en) * | 2022-09-29 | 2024-05-28 | 陕西震旦纪信息技术有限公司 | System and method for scheduling server computing resources based on Kubernetes |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110162413B (en) | Event-driven method and device | |
| CN112416585B (en) | Deep learning-oriented GPU resource management and intelligent scheduling method | |
| US9798830B2 (en) | Stream data multiprocessing method | |
| US20130339972A1 (en) | Determining an allocation of resources to a program having concurrent jobs | |
| CN111625331A (en) | Task scheduling method, device, platform, server and storage medium | |
| US20130268941A1 (en) | Determining an allocation of resources to assign to jobs of a program | |
| Imai et al. | Accurate resource prediction for hybrid IaaS clouds using workload-tailored elastic compute units | |
| CN110825522A (en) | Spark parameter self-adaptive optimization method and system | |
| CN112988340A (en) | Task scheduling method, device and system | |
| US20230004440A1 (en) | Allocating of computing resources for applications | |
| Lin et al. | Modeling and simulation of spark streaming | |
| Agrawal et al. | Scheduling parallelizable jobs online to minimize the maximum flow time | |
| Zuk et al. | Scheduling methods to reduce response latency of function as a service | |
| Shah et al. | Dynamic multilevel hybrid scheduling algorithms for grid computing | |
| CN109669767B (en) | Task packaging and scheduling method and system for multi-type context dependence | |
| CN110750362A (en) | Method and apparatus for analyzing biological information, and storage medium | |
| CN111625352A (en) | Scheduling method, device and storage medium | |
| CN111767125A (en) | Task execution method and device, electronic equipment and storage medium | |
| Grigoras et al. | Elastic management of reconfigurable accelerators | |
| CN115129481B (en) | Computing resource allocation method and device and electronic equipment | |
| CN107391262B (en) | Job scheduling method and device | |
| CN114546424A (en) | Application release method, device and equipment | |
| Le Hai et al. | A working time deadline-based backfilling scheduling solution | |
| CN113225269A (en) | Container-based workflow scheduling method, device and system and storage medium | |
| CN113515355A (en) | Resource scheduling method, device, server and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |