CN110888721A - Task scheduling method and related device - Google Patents
Task scheduling method and related device Download PDFInfo
- Publication number
- CN110888721A CN110888721A CN201910977724.4A CN201910977724A CN110888721A CN 110888721 A CN110888721 A CN 110888721A CN 201910977724 A CN201910977724 A CN 201910977724A CN 110888721 A CN110888721 A CN 110888721A
- Authority
- CN
- China
- Prior art keywords
- task
- tasks
- user
- running
- queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a task scheduling method and a related device. The method is applied to a task scheduling system, the task scheduling system comprises a task scheduling platform and a task distributing component, and the method comprises the following steps: acquiring a task flow submitted by a user; splitting the task stream through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the plurality of tasks, and generating a resubmissable task queue according to the submission triggering conditions of the plurality of tasks; submitting tasks in the resumable task queue to the task distribution component; and distributing the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task operation result. The technical scheme of the embodiment of the application is beneficial to simplifying the process of establishing the task and improving the task scheduling efficiency.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and a related apparatus for task scheduling.
Background
Oozie is an open source framework based on a workflow engine, is tightly combined with a Hadoop ecosystem, and can provide task scheduling and coordination for Hadoop MapReduce and Pig Job.
However, the workflow in the Oozie needs to define the execution sequence by using an xml file, the definition is too complex, when a user needs to establish a plurality of associated tasks, the execution sequence of the tasks needs to be written into the xml file, then the task is submitted by using the Oozie, and the Oozie can schedule the task flow.
Disclosure of Invention
The embodiment of the application provides a task scheduling method and a related device, which are beneficial to simplifying the process of establishing tasks and improving the task scheduling efficiency.
A first aspect of the present application provides a method for task scheduling, where the method is applied to a task scheduling system, where the task scheduling system includes a task scheduling platform and a task distributing component, and the method includes:
acquiring a task flow submitted by a user;
splitting the task stream through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the plurality of tasks, and generating a resubmissable task queue according to the submission triggering conditions of the plurality of tasks;
submitting tasks in the resumable task queue to the task distribution component;
and distributing the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task operation result.
The second aspect of the present application provides a device for task scheduling, where the device is applied to a task scheduling system, the task scheduling system includes a task scheduling platform and a task distributing component, and the device includes:
the acquisition unit is used for acquiring the task flow submitted by the user;
the scheduling unit is used for splitting the task flow through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the tasks, and generating a task queue capable of being submitted according to the submission triggering conditions of the tasks;
a submission unit, configured to submit the tasks in the resumable task queue to the task distribution component;
and the distribution unit is used for distributing the tasks in the resumable task queue to the target distributed cluster through the task distribution component and acquiring the task operation result.
A third aspect of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps of the method of any of the first aspects of the present application.
A fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon for execution by a processor to perform some or all of the steps described in any of the methods of the first aspect of the present application.
The task scheduling system comprises a task scheduling platform and a task distributing component, obtains a task stream submitted by a user, splits the task stream through the task scheduling platform to obtain a plurality of tasks, determines submission triggering conditions of the plurality of tasks, generates a submittable task queue according to the submission triggering conditions of the plurality of tasks, submits the tasks in the submittable task queue to the task distributing component, and distributes the tasks in the submittable task queue to a target distributed cluster through the task distributing component to obtain a task running result. In the task scheduling process, after a user submits a task stream, only the triggering condition for submitting the task needs to be configured on the task scheduling platform side, the task is split and scheduled on the task scheduling platform side, the task distributing component distributes the task, the complex xml file configuration on the bottom layer is not needed when the task is established, the task establishing process is simple, the task distributing component side does not need to split and schedule the task, the pressure of the task distributing component can be reduced, and the task scheduling efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic diagram of a task scheduling system according to an embodiment of the present application;
fig. 2 is a flowchart of a method for task scheduling according to an embodiment of the present application;
fig. 3 is a flowchart of a task scheduling platform according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a directed acyclic graph according to an embodiment of the present disclosure;
FIG. 5 is a diagram illustrating a method for defining a directed acyclic graph according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of an apparatus for task scheduling according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device in a hardware operating environment according to an embodiment of the present application.
Detailed Description
The task scheduling method and the related device provided by the embodiment of the application are beneficial to simplifying the process of establishing the task and improving the task scheduling efficiency.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
The following describes embodiments of the present application in detail.
Referring first to fig. 1, fig. 1 is a schematic diagram of a task scheduling system 100 according to an embodiment of the present application. The task scheduling system 100 comprises a task scheduling platform 101, a task distribution component 102, a user management system 103, a virtual user management system 104 and a plurality of distributed clusters 105;
the task scheduling system 100 provided in this embodiment of the application is a web application, and includes a client portion and a background portion, where the client portion is configured to display background task information on a browser and receive setting information of a user, specifically, when the user logs in the task scheduling system 100, the user opens the browser and enters an interactive interface of the task scheduling system 100, and can display the background task information on the interactive interface and receive the setting information of the user, where the task information may be, for example, a running result of a task, and the setting information of the user may be, for example, a task established by the user.
The task scheduling platform 101 is connected to a user management system 103 and a virtual user management system 104 in an interfacing manner, wherein the user management system 103 is used for login authentication of a user, the virtual user management system 104 is used for obtaining virtual user permissions associated with real-name users, specifically, when the user logs in the task scheduling system 100, a browser is opened, a user name and a password of the user are input, the user name and the password are sent to the user management system 103 for authentication, after the authentication is passed, the virtual user permissions of the user are obtained from the virtual user management system 104, and the virtual user permissions can achieve data permission isolation and resource isolation of task submission.
As a Web application, the task scheduling system 100 provided in this embodiment of the present application runs in a Tomcat container, where the Tomcat container is an open source Web application server, belongs to a lightweight application server, and provides simple foreground data logic processing, task generation and distribution, database operation, and a remote Hadoop cluster interaction function. The whole application is managed through a Spring framework, a display layer adopts Spring MVC, and a persistence layer adopts Mybatis.
The task distributing component 102 is used for submitting tasks, the task scheduling platform 101 splits and analyzes the tasks of the user, and then transmits the tasks to the task distributing component 102 for distributing and running, all task dependencies and task triggers are all in charge of the task scheduling platform 101 side, so that the pressure of the task distributing component 102 can be reduced, the task distributing component 102 runs as a task distributor, submits the tasks to a distributed cluster, and obtains a running result, specifically, the user sets the tasks on the task scheduling platform 101, the tasks set by the user are complex, and the task scheduling platform 101 splits and analyzes the tasks set by the user to obtain the tasks which can be identified by the task distributing component 102, namely, the task scheduling platform 101 splits the original workflow tasks into small workflow jobs.
Moreover, the task scheduling system 100 provided in the embodiment of the present application further supports a cross-cluster scheduling function, that is, a user may select to submit tasks to different distributed clusters in the multiple distributed clusters 105, so as to alleviate performance bottleneck of a single cluster resource and improve service quality.
As can be seen, according to the task scheduling system 100 provided in this embodiment of the present application, the task scheduling platform 101 obtains a task stream submitted by a user, splits the task stream through the task scheduling platform 101 to obtain a plurality of tasks, determines submission trigger conditions of the plurality of tasks, generates a submittable task queue according to the submission trigger conditions of the plurality of tasks, submits the tasks in the submittable task queue to the task distributing component 102, and distributes the tasks in the submittable task queue to a target distributed cluster in the plurality of distributed clusters 105 through the task distributing component 102 to obtain a task running result. Thus, in the task scheduling process, after a user submits a task flow, only the triggering condition for submitting the task needs to be configured on the task scheduling platform 101 side, the task is split and scheduled on the task scheduling platform 101 side, the task distribution component 102 distributes the task, the complex xml file configuration at the bottom layer is not needed when the task is established, the task establishing process is simple, the use threshold of the user is reduced, the task distribution component 102 side does not need to split and schedule the task, the pressure of the task distribution component 102 can be reduced, the task scheduling efficiency is improved, the task scheduling system 100 further comprises user login authentication and user authority acquisition, the authority is isolated, the data security is improved, meanwhile, the task scheduling system 100 further supports a cross-cluster scheduling function, namely, the user can select different distributed clusters in the plurality of distributed clusters 105 to submit the task, the performance bottleneck of single cluster resources is relieved, and the service quality is improved.
Referring to fig. 2, fig. 2 is a flowchart of a method for task scheduling according to an embodiment of the present disclosure, where the method for task scheduling according to the embodiment is applied to a task scheduling system, and the task scheduling system includes a task scheduling platform and a task distributing component. As shown in fig. 2, a method for task scheduling provided in this embodiment may include:
201. and acquiring a task flow submitted by a user.
When a user logs in a task scheduling system, a browser is opened, the user enters an interactive interface of the task scheduling system, background task information can be displayed on the interactive interface and setting information of the user can be received, the task information can be a task running result for example, the setting information of the user can be a task established by the user for example, and after the user establishes the task on the interactive interface, a task scheduling platform can obtain a task flow submitted by the user.
In one possible example, the task scheduling platform interfaces with the user management system and the virtual user management system, and before acquiring the task flow submitted by the user, further includes: receiving a login request of a user, wherein the login request carries identity information of the user; sending an authentication message carrying identity information to a user management system, wherein the authentication message is used for indicating the user management system to authenticate the identity information; and if the authentication is passed, acquiring the virtual user authority of the user from the virtual user management system according to the identity information.
Specifically, the user management system is used for login authentication of a user, the virtual user management system is used for obtaining virtual user permission associated with a real-name user, when the user logs in the task scheduling system, the browser is opened, the user name and the password of the user are input, the user name and the password are sent to the user management system for authentication, after the user passes the authentication, the virtual user permission of the user is obtained from the virtual user management system, and the virtual user permission can achieve data permission isolation and resource isolation submitted by a task.
Therefore, the task scheduling system also comprises user login authentication and user authority acquisition, so that the authority aspect is isolated, and the data security is improved.
202. Splitting the task flow through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the plurality of tasks, and generating a resubmissable task queue according to the submission triggering conditions of the plurality of tasks.
After the task flow submitted by the user is obtained, the task flow is complex, and the task scheduling platform needs to split and analyze the task flow to obtain tasks which can be identified by a plurality of task distribution components, namely, the task scheduling platform decomposes an original workflow task into small workflow jobs, wherein each workflow job has only one action therein.
In one possible example, the submission trigger condition comprises a timing trigger or a dependent trigger, and the task scheduling platform determining the submission trigger condition for the plurality of tasks comprises: when a first task in the multiple tasks has no dependency relationship or the first task is in top-level dependency in the dependency relationship, determining the time for submitting the first task, wherein the dependency relationship comprises the dependency relationship between a child task and a parent task, the child task can be executed only after the parent task is completed, and the task in top-level dependency has no parent task; when the first task has a dependency relationship and the first task is not in a top-level dependency, a parent task of the first task is determined.
Specifically, the submission triggering condition of the task includes timing triggering or dependency triggering, that is, time reaches a requirement or all dependent parent tasks are completely run, wherein for the task with a dependency relationship, a plurality of tasks are deployed in a Directed Acyclic Graph (DAG) mode, a node in the DAG represents the task itself, and an edge represents the dependency relationship, so as to maintain the dependency relationship between the plurality of tasks, that is, there is a dependency relationship between a child task and a parent task, the child task needs to wait for the parent task to be completed before being executed, and the dependency relationship between the plurality of tasks is specified by a user when the task is established. For a single task without dependency or a task with top-level dependency in the dependency, a user can set the submission time of the task, and for a task with dependency and not with top-level dependency, the user can set all parent tasks of the task, and if the task is previously set with the submission time, the submission time previously set for the task will be invalid once the dependency is set. Furthermore, for a task having a dependency relationship, when one task depends on a plurality of tasks, it is necessary to set a dependency period of the multi-dependent task, that is, a trigger frequency, and the dependency period may be set such that the multi-dependent task is scheduled once every day, or every week, or every month, for example.
In one possible example, a directed acyclic graph DAG can be defined by using a two-dimensional array of N × N, the content of the array can be only 0 or 1, where 0 represents no edge, 1 represents an edge, and the content on the diagonal line must be 0 because the node cannot depend on itself, for example, task a points to task B to understand that the content at the intersection of the row where task a is located and the column where task B is located is 1. The number of 1's stored in the two-dimensional array is the number of DAG's edges. A row of an array may be understood as meaning that the task points to other tasks and the columns of the array are exactly the opposite. Assuming that the contents of a column are all 0, i.e., the in-degree of the task is 0, i.e., the task is not currently dependent, it can be placed in the resumable task queue (provided that the runtime condition has been reached). Assuming that a certain task, for example, task a is in a final state after running and completing, the dependency relationship between task a and other tasks needs to be eliminated, and it is only necessary to set all the contents of the row where task a is located in the DAG to 0.
In the task scheduling process, after a user submits a task stream, only the triggering condition for submitting the task needs to be configured on the task scheduling platform side, the task is split and scheduled on the task scheduling platform side, the complex xml file configuration on the bottom layer is not needed when the task is established, the task establishing process is simple, and the use threshold of the user is reduced.
In one possible example, generating a resubmitteable task queue according to task trigger conditions for a plurality of tasks includes: when the first task has no dependency relationship or is in top-level dependency in the dependency relationship, acquiring the current time, and when the current time reaches the time for submitting the first task, putting the first task into a task queue capable of being submitted; when the first task has a dependency relationship and the first task is not in top-level dependency, obtaining the running result of the parent task of the first task, and when the running results of the parent task of the first task are all running completely, placing the first task into a resumable task queue.
Specifically, the submission triggering condition of the task includes timing triggering or dependency triggering, that is, time reaches a requirement or all dependent parent tasks finish running, for a single task without a dependency relationship or a task which is dependent on the top level in the dependency relationship, the submission triggering condition of the task is timing triggering, when the current time reaches the requirement, submission can be triggered, for a task which has a dependency relationship and is not dependent on the top level, the submission triggering condition of the task is dependency triggering, and when all parent tasks finish running, submission can be triggered.
203. Submitting the tasks in the resubmible task queue to the task distribution component.
In one possible example, the task scheduling platform submitting the tasks in the resumable task queue to the task distribution component comprises: inquiring the submittable task queue according to a preset period, and acquiring a second task from the submittable task queue; generating an xml file according to the second task; the xml file is submitted to the task distribution component.
Specifically, the task scheduling platform queries the submittable task queue at regular time, all tasks stored in the submittable task queue are triggered to be submitted, the task scheduling platform acquires the tasks from the submittable task queue, and the task distributing component supports the task of distributing the xml file format, so that the task scheduling platform needs to generate the tasks of the xml file format according to the acquired tasks, then submits the tasks of the xml file format to the task distributing component, and the task distributing component performs subsequent distribution operation.
204. And distributing the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task operation result.
The task distribution component is used for submitting tasks, all task dependencies and task triggers are all responsible for the task scheduling platform side, so that the pressure of the task distribution component can be reduced, the task distribution component operates as a task distributor, the tasks are submitted to the target distributed cluster, and an operation result is obtained. In addition, in the process of determining the target distributed cluster, the task scheduling method provided in this embodiment further supports a cross-cluster scheduling function, that is, a user may select to submit tasks to different distributed clusters in the multiple distributed clusters, which alleviates the performance bottleneck of a single cluster resource and improves the service quality.
In one possible example, the task scheduling platform obtaining the task execution result comprises: sending a running result acquisition request carrying a task identifier of a second task to the task distributing component, wherein the running result acquisition request is used for indicating the task distributing component to acquire a running result of the second task from the target distributed cluster; receiving a response which is sent by the task distribution component and carries the running result of the second task; storing the running result of the second task in a database; and when the running result of the second task is unfinished running, sending the running result acquisition request to the task distributing component again, and updating the running result of the second task stored in the database until the running result of the second task is finished running.
Specifically, after a task is distributed to a target distributed cluster by a task distribution component, the task runs in the target distributed cluster, a task scheduling platform sends a running result obtaining request carrying a task identifier of the task to the task distribution component, the task distribution component obtains the running result of the task from the target distributed cluster, and feeds back the running result of the task to the task scheduling platform, the task scheduling platform stores the obtained running result of the task in a database for subsequent checking, and when the operation result of the task is unfinished operation, the task scheduling platform can send a request for many times, inquire the operation result of the task, update the original operation result stored in the database according to the newly acquired operation result, so as to be convenient for subsequent checking of the latest running result until the task is run and completed, and the running result of the task stored in the database is run and completed.
The task scheduling platform can store the acquired running result of the task in the database, so that when a subsequent user needs to check the running result, the running result can be acquired from the database and presented to the user, and when the running result of the task is unfinished running, the task scheduling platform can send a request for many times, inquire the running result of the task, and update the original running result stored in the database according to the newly acquired running result, so that when the subsequent user needs to check the running result, the latest running result can be acquired and presented to the user, and the task scheduling platform can be used for mastering the running condition of the task in real time.
In one possible example, after obtaining the task execution result, the task scheduling platform further includes: when receiving a running result viewing request carrying a task identifier of a second task, acquiring a running result of the second task from a database; generating a visual interface for displaying the running result of the second task; and when the running result of the second task is that the running is completed, updating the resubmissable task queue according to the running result of the second task and the submission triggering conditions of the plurality of tasks.
Specifically, the task scheduling system of the method application provided by this embodiment is a web application, and includes a client portion and a background portion, where the client portion is configured to display background task information on a browser and receive setting information of a user, and when the user logs in the task scheduling system, the user opens the browser and enters an interactive interface of the task scheduling system, and can display the background task information and receive the setting information of the user on the interactive interface, where the task information may be, for example, a running result of a task, and the setting information of the user may be, for example, a task established by the user. When a user needs to check the running result of a certain task, the user can click and check through an interactive interface, the task scheduling platform receives a running result checking request carrying a task identifier of the task, obtains the running result from the database, and then generates a visual interface for displaying the running result of the task on the interactive interface, wherein the visual interface comprises a regenerated interface or a popup window generated on the original interface. And if the obtained running result of the task is that the running is completed, the submittable task queue needs to be updated according to the running result of the task and the submission triggering conditions of the plurality of tasks, for example, the task a is a parent task of the task B and the task C, and both the task B and the task C have only one parent task of the task a, so that when the running of the task a is completed, both the task B and the task C can be triggered to submit, that is, the task B and the task C need to be put into the submittable task queue, and the submittable task queue is updated.
Therefore, when the running result of the task is running, the submittable task queue is updated according to the running result of the task and the submission triggering conditions of the plurality of tasks, so that other submittable tasks can be ensured to be put into the submittable task queue in time and distributed by the task distribution component, and the effectiveness of task scheduling is ensured.
It can be seen that, according to the task scheduling method provided by this embodiment, the task scheduling system includes a task scheduling platform and a task distribution component, obtains a task stream submitted by a user, splits the task stream through the task scheduling platform to obtain a plurality of tasks, determines submission trigger conditions of the plurality of tasks, generates a resumable task queue according to the submission trigger conditions of the plurality of tasks, submits the tasks in the resumable task queue to the task distribution component, and distributes the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task running result. In the task scheduling process, after a user submits a task stream, only the triggering condition for submitting the task needs to be configured on the task scheduling platform side, the task is split and scheduled on the task scheduling platform side, the task distributing component distributes the task, the complex xml file configuration on the bottom layer is not needed when the task is established, the task establishing process is simple, the task distributing component side does not need to split and schedule the task, the pressure of the task distributing component can be reduced, and the task scheduling efficiency is improved.
Referring to fig. 3, fig. 3 is a workflow diagram of a task scheduling platform according to an embodiment of the present application, where the task scheduling platform includes a database, a task instantiation module, a DAG dependency establishment module, a timing scheduling module, and a synchronization module, and after a user logs in a task scheduling system, a template and a task are set through an interactive interface of a browser, where the template is a compressed package into which an instruction script of the task is compressed, on one hand, the recorded task is written into the database of the task scheduling platform, and on the other hand, the set template is decompressed to obtain the instruction script of the task, and the instruction script may be, for example, a shell script or a python script, and then the instruction script of the task is uploaded to a distributed file system (HDFS) of a distributed cluster, specifically, when the task distribution component distributes the task to the distributed cluster, the method distributes tasks to resource managers (YARNs) in a distributed cluster to run, and a specific running instruction script needs to be obtained from an HDFS, wherein a task distribution component can be an Oozie component. And after the uploading is finished, waiting for the task scheduling platform side to perform task scheduling. As shown in fig. 3, a workflow of a task scheduling platform provided in an embodiment of the present application may include:
Specifically, the specific time may be, for example, at morning zero every day, a new task of the user is taken out from the database, and the new task of the user is instantiated. The data base stores the date of the last instantiation of the task, so that the task scheduling platform can know which tasks are needed to be instantiated only by comparing the current time with the date of the last instantiation. When the special time is morning zero every day, the task scheduling system only instantiates the tasks once every morning zero, so that the newly-built tasks can take effect the next day.
The database stores task instance id and running state, the database is restarted at zero time every day, the restart process instantiates tasks created today, the running state of yesterday task instances is judged, the running state is taken out from the database to be unfinished tasks and is instantiated in a memory, if the task instances are submitted to the task distribution component yesterday, the task scheduling platform continues to synchronize the running state of the task instances from the task distribution component, and if the task instances are not submitted to the task distribution component, the task scheduling platform submits the task instances to the task distribution component until a submission triggering condition is met.
And 2, after the task instance is obtained, the DAG dependency building module builds a DAG dependency relationship on the task instance.
Specifically, a plurality of tasks are provided in a workflow, for example, a Hadoop MapReduce, a Hadoop peg jobb, and the like, all the tasks are deployed in a Directed Acyclic Graph (DAG) mode, nodes in the DAG are regarded as tasks themselves, and edges are regarded as dependency relationships, so that the dependency relationships between the tasks are maintained. In short, a subtask needs to wait for a parent task to complete before execution, the subtask has a dependency relationship with the parent task, the top-level dependent task has no parent task, and the relationships are specified by a user when establishing a task.
And 3, generating a task queue capable of being submitted according to the judgment condition for triggering the submission of the task.
The task submission triggering has two judgment conditions, namely, the time reaches the requirement and all the dependent parent tasks finish running, and a task submission queue is generated according to the two conditions.
Specifically, the submission trigger of the task includes a timing trigger or a dependency trigger, that is, the time reaches the requirement or all the parent tasks that depend on the task are run completely, wherein for a single task without a dependency relationship or a task that is dependent on the top level in the dependency relationship, the user can set the submission time of the task, for a task that has a dependency relationship and is not dependent on the top level, the user can set all the parent tasks of the task, and if the submission time is set before the task, the submission time set before the task will fail once the dependency relationship is set. Furthermore, for a task having a dependency relationship, when one task depends on a plurality of tasks, it is necessary to set a dependency period of the multi-dependent task, that is, a trigger frequency, and the dependency period may be set such that the multi-dependent task is scheduled once every day, or every week, or every month, for example.
For a single task without dependency or a task which is dependent on the top layer in the dependency, the submission triggering condition of the task is triggered at fixed time, and the submission can be triggered when the current time reaches the requirement.
And 4, the timing scheduling module regularly inquires the task queue capable of being submitted to acquire the task.
And 5, generating an xml file according to the acquired task, submitting the xml file to the task distribution component for processing, and interacting the task distribution component with the distributed cluster.
And 6, putting the tasks submitted to the task distribution component into a running task queue.
And 7, the synchronization module acquires the tasks needing the synchronization state from the running task queue.
And 8, the synchronization module initiates a request to the task distribution component to inquire the running result of the task.
Wherein the synchronization module may send the request to the task distribution component multiple times until the task runs are completed.
And 9, writing the obtained task operation result into a database by the synchronization module.
Therefore, the user can operate on the interactive interface of the task scheduling system, interact with the updated database and inquire the running result of the task.
And step 10, updating DAG dependency when the task is completed, and putting the task which is triggered to be submitted into a task queue which can be submitted.
It can be seen that, with the workflow of the task scheduling platform provided by this embodiment, in the task scheduling process, after a user submits a task, only the trigger condition for submitting the task needs to be configured on the task scheduling platform side, the task is scheduled on the task scheduling platform side, and the complex xml file configuration on the bottom layer is not needed when the task is established, which simplifies the process of establishing the task.
Referring to fig. 4, fig. 4 is a schematic diagram of a directed acyclic graph according to an embodiment of the present disclosure. As shown in fig. 4, the directed acyclic graph includes a task a, a task B, a task C, a task D, and a task E, where the task a is in top-level dependency, the child task is a task D, the task B is also in top-level dependency, the child task includes a task C and a task E, the parent task of the task C is a task B, the child task is a task E, the parent task of the task D is a task a, the child task is a task E, the parent task of the task E includes a task B, a task C, and a task D, there is no child task, an edge in the directed acyclic graph indicates that there is a dependency relationship, and the direction of the edge indicates a direction of dependency, for example, for the task a and the task D, the direction of the edge indicates that the task a points to the task D, and indicates that the task a is a parent task of the task D, and the task D can be run only after.
Referring to fig. 5, fig. 5 is a schematic diagram defining a directed acyclic graph according to an embodiment of the present application, and corresponds to fig. 4. As shown in fig. 5, a directed acyclic graph DAG can be defined by using a two-dimensional array of N × N, the content of the array can only be 0 or 1, where 0 represents no edge, 1 represents an edge, and the content on the diagonal line is necessarily 0, because the node cannot depend on itself, for example, task B points to task C, so that it can be understood that the content at the boundary between the row where task B is located and the column where task C is located is 1. The number of 1's stored in the two-dimensional array is the number of DAG's edges. A row of an array may be understood as meaning that the task points to other tasks and the columns of the array are exactly the opposite. Assuming that the contents of a column are all 0, i.e., the in-degree of the task is 0, i.e., the task is not currently dependent, it can be placed in the resumable task queue (provided that the runtime condition has been reached). Assuming that a certain task, for example, task B is in a final state after running and completing, the dependency relationship between task B and other tasks needs to be eliminated, and it is only necessary to set all the contents of the row where task B is located in the DAG to 0.
Referring to fig. 6, fig. 6 is a schematic diagram of an apparatus for task scheduling according to an embodiment of the present application, where the apparatus is applied to a task scheduling system, and the task scheduling system includes a task scheduling platform and a task distribution component. As shown in fig. 6, an apparatus for task scheduling provided in an embodiment of the present application may include:
an obtaining unit 601, configured to obtain a task stream submitted by a user;
a scheduling unit 602, configured to split the task stream through the task scheduling platform to obtain multiple tasks, determine submission triggering conditions of the multiple tasks, and generate a resubmissable task queue according to the submission triggering conditions of the multiple tasks;
a submitting unit 603, configured to submit the tasks in the resumable task queue to the task distribution component;
the distributing unit 604 is configured to distribute the tasks in the resumable task queue to the target distributed cluster through the task distributing component, and obtain a task running result.
For specific implementation of the device for task scheduling in the present application, reference may be made to each embodiment of the method for task scheduling, which is not described herein again.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device in a hardware operating environment according to an embodiment of the present application. As shown in fig. 7, an electronic device of a hardware operating environment according to an embodiment of the present application may include:
a processor 701, such as a CPU.
The memory 702, which may optionally be a high speed RAM memory, may also be a stable memory, such as a disk memory.
A communication interface 703 for implementing connection communication between the processor 701 and the memory 702.
Those skilled in the art will appreciate that the configuration of the electronic device shown in fig. 7 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 7, the memory 702 may include an operating system, a network communication module, and a program for task scheduling. An operating system is a program that manages and controls the hardware and software resources of an electronic device, a program that supports task scheduling, and the execution of other software or programs. The network communication module is used to enable communication between the components within the memory 702, as well as with other hardware and software in the electronic device.
In the electronic device shown in fig. 7, the processor 701 is configured to execute a program for scheduling tasks stored in the memory 702, and implement the following steps:
acquiring a task flow submitted by a user;
splitting the task stream through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the plurality of tasks, and generating a resubmissable task queue according to the submission triggering conditions of the plurality of tasks;
submitting tasks in the resumable task queue to the task distribution component;
and distributing the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task operation result.
For specific implementation of the electronic device of the present application, reference may be made to various embodiments of the task scheduling method, which are not described herein again.
Another embodiment of the present application provides a computer-readable storage medium storing a computer program for execution by a processor to perform the steps of:
acquiring a task flow submitted by a user;
splitting the task stream through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the plurality of tasks, and generating a resubmissable task queue according to the submission triggering conditions of the plurality of tasks;
submitting tasks in the resumable task queue to the task distribution component;
and distributing the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task operation result.
For specific implementation of the computer-readable storage medium of the present application, reference may be made to various embodiments of the task scheduling method, which are not described herein again.
It is also noted that while for simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present application is not limited by the order of acts, as some steps may, in accordance with the present application, occur in other orders and concurrently. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application. In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A method for task scheduling is applied to a task scheduling system, wherein the task scheduling system comprises a task scheduling platform and a task distributing component, and the method comprises the following steps:
acquiring a task flow submitted by a user;
splitting the task stream through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the plurality of tasks, and generating a resubmissable task queue according to the submission triggering conditions of the plurality of tasks;
submitting tasks in the resumable task queue to the task distribution component;
and distributing the tasks in the resumable task queue to a target distributed cluster through the task distribution component to obtain a task operation result.
2. The method of claim 1, wherein the task scheduling platform interfaces with a user management system and a virtual user management system, and wherein prior to the obtaining the flow of tasks submitted by the user, the method further comprises:
receiving a login request of the user, wherein the login request carries identity information of the user;
sending an authentication message carrying the identity information to the user management system, wherein the authentication message is used for indicating the user management system to authenticate the identity information;
and if the authentication is passed, acquiring the virtual user authority of the user from the virtual user management system according to the identity information.
3. The method of claim 1 or 2, wherein the submission trigger condition comprises a timing trigger or a dependent trigger, and wherein the determining the submission trigger conditions for the plurality of tasks comprises:
when a first task in the plurality of tasks has no dependency relationship or the first task is in top-level dependency in the dependency relationship, determining the time for submitting the first task, wherein the dependency relationship comprises the dependency relationship between a sub task and a parent task, the sub task can be executed only after the parent task is completed, and the task in the top-level dependency has no parent task;
determining a parent task of the first task when the first task has a dependency relationship and the first task is not in a top-level dependency.
4. The method of claim 3, wherein generating a resubmitted task queue according to a submission trigger condition of the plurality of tasks comprises:
when the first task has no dependency relationship or is in top-level dependency relationship in the dependency relationship, obtaining current time, and when the current time reaches the time for submitting the first task, putting the first task into the resumable task queue;
when the first task has a dependency relationship and the first task is not in top-level dependency, obtaining the running result of the parent task of the first task, and when the running results of the parent task of the first task are all running completely, putting the first task into the resumable task queue.
5. The method of claim 1, wherein the submitting the tasks in the resumable task queue to the task distribution component comprises:
inquiring the submittable task queue according to a preset period, and acquiring a second task from the submittable task queue;
generating a task in an xml file format according to the second task;
and submitting the tasks in the xml file format to the task distribution component.
6. The method of claim 5, wherein the obtaining task execution results comprises:
sending a running result obtaining request carrying a task identifier of the second task to the task distributing component, wherein the running result obtaining request is used for indicating the task distributing component to obtain a running result of the second task from the target distributed cluster;
receiving a response which is sent by the task distribution component and carries the running result of the second task;
storing the running result of the second task in a database;
and when the running result of the second task is unfinished running, sending the running result acquisition request to the task distribution component again, and updating the running result of the second task stored in the database until the running result of the second task is finished running.
7. The method of claim 6, wherein after obtaining the task execution results, the method further comprises:
when receiving a running result viewing request carrying a task identifier of the second task, obtaining a running result of the second task from the database;
generating a visual interface for displaying the running result of the second task;
and when the running result of the second task is that the running is finished, updating the resumable task queue according to the running result of the second task and the submission triggering conditions of the plurality of tasks.
8. An apparatus for task scheduling, the apparatus being applied to a task scheduling system, the task scheduling system including a task scheduling platform and a task distributing component, the apparatus comprising:
the acquisition unit is used for acquiring the task flow submitted by the user;
the scheduling unit is used for splitting the task flow through the task scheduling platform to obtain a plurality of tasks, determining submission triggering conditions of the tasks, and generating a task queue capable of being submitted according to the submission triggering conditions of the tasks;
a submission unit, configured to submit the tasks in the resumable task queue to the task distribution component;
and the distribution unit is used for distributing the tasks in the resumable task queue to the target distributed cluster through the task distribution component and acquiring the task operation result.
9. An electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910977724.4A CN110888721A (en) | 2019-10-15 | 2019-10-15 | Task scheduling method and related device |
| PCT/CN2020/118350 WO2021073414A1 (en) | 2019-10-15 | 2020-09-28 | Task scheduling method and related apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910977724.4A CN110888721A (en) | 2019-10-15 | 2019-10-15 | Task scheduling method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110888721A true CN110888721A (en) | 2020-03-17 |
Family
ID=69746188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910977724.4A Pending CN110888721A (en) | 2019-10-15 | 2019-10-15 | Task scheduling method and related device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110888721A (en) |
| WO (1) | WO2021073414A1 (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111475272A (en) * | 2020-04-07 | 2020-07-31 | 四川虹美智能科技有限公司 | Method and device for controlling Java Web application timing task and task scheduling platform |
| CN111553545A (en) * | 2020-07-13 | 2020-08-18 | 四川新网银行股份有限公司 | Method for optimizing batch task scheduling dependence |
| CN111782373A (en) * | 2020-07-01 | 2020-10-16 | 上海乾臻信息科技有限公司 | Job scheduling method and device |
| CN111782389A (en) * | 2020-06-22 | 2020-10-16 | 中科边缘智慧信息科技（苏州）有限公司 | Task scheduling system and method under mobile edge information service network |
| CN111930487A (en) * | 2020-08-28 | 2020-11-13 | 北京百度网讯科技有限公司 | Job flow scheduling method and device, electronic equipment and storage medium |
| CN112114944A (en) * | 2020-09-04 | 2020-12-22 | 武汉旷视金智科技有限公司 | Task scheduling method and device, task scheduling platform and computer storage medium |
| CN112114950A (en) * | 2020-09-21 | 2020-12-22 | 中国建设银行股份有限公司 | Task scheduling method and device and cluster management system |
| CN112114956A (en) * | 2020-09-29 | 2020-12-22 | 中国银行股份有限公司 | Task scheduling method, device and system |
| CN112327719A (en) * | 2020-11-19 | 2021-02-05 | 山东高速信息集团有限公司 | Method and system for realizing platform unified application interface library in intelligent park |
| CN112559156A (en) * | 2020-12-23 | 2021-03-26 | 百果园技术(新加坡)有限公司 | Multi-dependency task grouping management method, device, equipment and storage medium |
| WO2021073414A1 (en) * | 2019-10-15 | 2021-04-22 | 平安科技（深圳）有限公司 | Task scheduling method and related apparatus |
| CN112732898A (en) * | 2020-12-30 | 2021-04-30 | 平安科技（深圳）有限公司 | Document abstract generation method and device, computer equipment and storage medium |
| CN112905321A (en) * | 2021-02-07 | 2021-06-04 | 北京红山信息科技研究院有限公司 | Event response type task triggering method and device, electronic equipment and storage medium |
| CN112910703A (en) * | 2021-02-01 | 2021-06-04 | 中金云金融（北京）大数据科技股份有限公司 | Offline task management platform |
| CN112925619A (en) * | 2021-02-24 | 2021-06-08 | 深圳依时货拉拉科技有限公司 | Big data real-time computing method and platform |
| CN113268318A (en) * | 2021-04-07 | 2021-08-17 | 北京思特奇信息技术股份有限公司 | Task scheduling method and distributed system |
| CN113326117A (en) * | 2021-07-15 | 2021-08-31 | 中国电子科技集团公司第十五研究所 | Task scheduling method, device and equipment |
| CN113468212A (en) * | 2021-07-21 | 2021-10-01 | 华青融天(北京)软件股份有限公司 | Event execution method and device and electronic equipment |
| CN113821320A (en) * | 2021-08-28 | 2021-12-21 | 云智联网络科技(北京)有限公司 | Distributed data warehouse workflow scheduling method |
| CN114039962A (en) * | 2021-11-04 | 2022-02-11 | 北京知道创宇信息技术股份有限公司 | Node selection method, system, electronic device and storage medium |
| WO2022261867A1 (en) * | 2021-06-16 | 2022-12-22 | 华为技术有限公司 | Task scheduling method and apparatus |
| CN115840631A (en) * | 2023-01-04 | 2023-03-24 | 中科金瑞(北京)大数据科技有限公司 | RAFT-based high-availability distributed task scheduling method and equipment |
| WO2023108654A1 (en) * | 2021-12-17 | 2023-06-22 | 华为技术有限公司 | Task scheduling method, apparatus and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8402466B2 (en) * | 2009-02-17 | 2013-03-19 | International Business Machines Corporation | Practical contention-free distributed weighted fair-share scheduler |
| CN102262751A (en) * | 2010-05-31 | 2011-11-30 | ***通信集团贵州有限公司 | Method and system for acquiring service application based on SOA (service-oriented architecture) |
| CN102831012A (en) * | 2011-06-16 | 2012-12-19 | 日立(中国)研究开发有限公司 | Task scheduling device and task scheduling method in multimode distributive system |
| CN104462370A (en) * | 2014-12-09 | 2015-03-25 | 北京百度网讯科技有限公司 | Distributed task scheduling system and method |
| CN105824697A (en) * | 2016-03-23 | 2016-08-03 | 浪潮通信信息***有限公司 | Distributed multi-stage scheduling method based on queues |
| CN110888721A (en) * | 2019-10-15 | 2020-03-17 | 平安科技（深圳）有限公司 | Task scheduling method and related device |
-
2019
- 2019-10-15 CN CN201910977724.4A patent/CN110888721A/en active Pending
-
2020
- 2020-09-28 WO PCT/CN2020/118350 patent/WO2021073414A1/en active Application Filing
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021073414A1 (en) * | 2019-10-15 | 2021-04-22 | 平安科技（深圳）有限公司 | Task scheduling method and related apparatus |
| CN111475272A (en) * | 2020-04-07 | 2020-07-31 | 四川虹美智能科技有限公司 | Method and device for controlling Java Web application timing task and task scheduling platform |
| CN111782389A (en) * | 2020-06-22 | 2020-10-16 | 中科边缘智慧信息科技（苏州）有限公司 | Task scheduling system and method under mobile edge information service network |
| CN111782373A (en) * | 2020-07-01 | 2020-10-16 | 上海乾臻信息科技有限公司 | Job scheduling method and device |
| CN111782373B (en) * | 2020-07-01 | 2023-03-07 | 上海乾臻信息科技有限公司 | Job scheduling method and device |
| CN111553545A (en) * | 2020-07-13 | 2020-08-18 | 四川新网银行股份有限公司 | Method for optimizing batch task scheduling dependence |
| CN111553545B (en) * | 2020-07-13 | 2020-10-27 | 四川新网银行股份有限公司 | Method for optimizing batch task scheduling dependence |
| CN111930487A (en) * | 2020-08-28 | 2020-11-13 | 北京百度网讯科技有限公司 | Job flow scheduling method and device, electronic equipment and storage medium |
| CN111930487B (en) * | 2020-08-28 | 2024-05-24 | 北京百度网讯科技有限公司 | Job stream scheduling method and device, electronic equipment and storage medium |
| CN112114944A (en) * | 2020-09-04 | 2020-12-22 | 武汉旷视金智科技有限公司 | Task scheduling method and device, task scheduling platform and computer storage medium |
| CN112114950A (en) * | 2020-09-21 | 2020-12-22 | 中国建设银行股份有限公司 | Task scheduling method and device and cluster management system |
| CN112114956A (en) * | 2020-09-29 | 2020-12-22 | 中国银行股份有限公司 | Task scheduling method, device and system |
| CN112327719A (en) * | 2020-11-19 | 2021-02-05 | 山东高速信息集团有限公司 | Method and system for realizing platform unified application interface library in intelligent park |
| CN112559156A (en) * | 2020-12-23 | 2021-03-26 | 百果园技术(新加坡)有限公司 | Multi-dependency task grouping management method, device, equipment and storage medium |
| CN112732898A (en) * | 2020-12-30 | 2021-04-30 | 平安科技（深圳）有限公司 | Document abstract generation method and device, computer equipment and storage medium |
| CN112910703A (en) * | 2021-02-01 | 2021-06-04 | 中金云金融（北京）大数据科技股份有限公司 | Offline task management platform |
| CN112905321A (en) * | 2021-02-07 | 2021-06-04 | 北京红山信息科技研究院有限公司 | Event response type task triggering method and device, electronic equipment and storage medium |
| CN112925619A (en) * | 2021-02-24 | 2021-06-08 | 深圳依时货拉拉科技有限公司 | Big data real-time computing method and platform |
| CN113268318A (en) * | 2021-04-07 | 2021-08-17 | 北京思特奇信息技术股份有限公司 | Task scheduling method and distributed system |
| WO2022261867A1 (en) * | 2021-06-16 | 2022-12-22 | 华为技术有限公司 | Task scheduling method and apparatus |
| CN113326117A (en) * | 2021-07-15 | 2021-08-31 | 中国电子科技集团公司第十五研究所 | Task scheduling method, device and equipment |
| CN113468212A (en) * | 2021-07-21 | 2021-10-01 | 华青融天(北京)软件股份有限公司 | Event execution method and device and electronic equipment |
| CN113821320A (en) * | 2021-08-28 | 2021-12-21 | 云智联网络科技(北京)有限公司 | Distributed data warehouse workflow scheduling method |
| CN114039962A (en) * | 2021-11-04 | 2022-02-11 | 北京知道创宇信息技术股份有限公司 | Node selection method, system, electronic device and storage medium |
| WO2023108654A1 (en) * | 2021-12-17 | 2023-06-22 | 华为技术有限公司 | Task scheduling method, apparatus and system |
| CN115840631A (en) * | 2023-01-04 | 2023-03-24 | 中科金瑞(北京)大数据科技有限公司 | RAFT-based high-availability distributed task scheduling method and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021073414A1 (en) | 2021-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110888721A (en) | Task scheduling method and related device | |
| US9661071B2 (en) | Apparatus, systems and methods for deployment and management of distributed computing systems and applications | |
| CN106844198B (en) | Distributed dispatching automation test platform and method | |
| US20200133666A1 (en) | Application lifecycle management system | |
| CN112104723B (en) | Multi-cluster data processing system and method | |
| CN108304260B (en) | Virtualization operation scheduling system based on high-performance cloud computing and implementation method thereof | |
| CN110083455B (en) | Graph calculation processing method, graph calculation processing device, graph calculation processing medium and electronic equipment | |
| CN111290854A (en) | Task management method, device and system, computer storage medium and electronic equipment | |
| US20150095917A1 (en) | Distributed uima cluster computing (ducc) facility | |
| CN111212116A (en) | High-performance computing cluster creating method and system based on container cloud | |
| CN113569987A (en) | Model training method and device | |
| CN104536899A (en) | Software deploying and maintaining method based on intelligent cluster | |
| CN112051993A (en) | State machine template generation and task processing method, device, medium and equipment | |
| US10110508B2 (en) | Queued workload service in a multi tenant environment | |
| CN108170417B (en) | Method and device for integrating high-performance job scheduling framework in MESOS cluster | |
| CN113377493A (en) | Container cloud simulation system and design method thereof | |
| KR20220088333A (en) | Dynamic cloud deployment of robotic process automation (rpa) robots | |
| CN111435315A (en) | Method, apparatus, device and computer readable medium for allocating resources | |
| US20200310828A1 (en) | Method, function manager and arrangement for handling function calls | |
| Werner et al. | HARDLESS: A generalized serverless compute architecture for hardware processing accelerators | |
| US20190205110A1 (en) | Compilation optimization via dynamic server cloning | |
| CN106648871B (en) | Resource management method and system | |
| US20210149709A1 (en) | Method and apparatus for processing transaction | |
| CN113326098B (en) | Cloud management platform supporting KVM virtualization and container virtualization | |
| CN109471709A (en) | The dispatching method of flow tasks based on Apache Oozie frame processing big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |