CN112269641B - Scheduling method, scheduling device, electronic equipment and storage medium - Google Patents
Scheduling method, scheduling device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112269641B CN112269641B CN202011299381.XA CN202011299381A CN112269641B CN 112269641 B CN112269641 B CN 112269641B CN 202011299381 A CN202011299381 A CN 202011299381A CN 112269641 B CN112269641 B CN 112269641B
- Authority
- CN
- China
- Prior art keywords
- task
- node
- scheduled
- processing area
- scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Abstract
The application relates to a scheduling method, a scheduling device, electronic equipment and a storage medium. Acquiring newly added tasks to be scheduled in the cluster, if it is determined that candidate scheduling nodes meeting the resource quantity required by the tasks to be scheduled exist in computing nodes of the cluster, and the tasks to be scheduled are online tasks, judging whether nodes belonging to an online task processing area exist in the candidate scheduling nodes, and if so, selecting a target scheduling node from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. For the online task to be scheduled to the online task processing area preferentially, when the resources of the online task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the situation that the online task fails to be scheduled due to the increase of scheduled tasks can be avoided, the online task can be guaranteed to be processed in time, and further, the service quality of the online task is guaranteed.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a scheduling method, an apparatus, an electronic device, and a storage medium.
Background
The container technology is a current hot technology, an underlying architecture based on K8S (Kubernetes) cluster design is popular, and software deployment is easier. Typically, some internet online application is deployed on the K8S, typically in the form of a controller model, deviyment. The request amount of the online application is generally non-constant, such as a higher request amount in the daytime, a lower request amount in the evening, or an abrupt increase in the request amount when an active day is encountered. How to allocate resources for such non-constant requesting applications is a problem with traditional physical/virtual machine based deployment architectures as an allodynia point. If the resources are allocated according to the maximum request amount, when the request amount is reduced, great resource waste exists, and when the request amount is increased, the resources are allocated according to a lower standard, and the problem that the capacity expansion is not timely, and the service quality of the online application is affected exists.
Under the Deployment architecture based on K8S, K8S provides a transverse automatic capacity expansion and contraction technology (Horizontal Pod Autoscaler, HPA) technology, and can automatically expand and contract copies according to the memory overhead of the minimum deployable unit (Pod) calculated by K8S under the condition of creation and management under the condition of Deployment or other custom indexes (such as request quantity), if the CPU (central processing unit) utilization rate is greater than a certain threshold (such as 70%), capacity expansion is performed, and if the CPU utilization rate is lower than a certain threshold (30%). However, in an actual scenario, the HPA technology of the K8S does not consider the actual resource allocation situation of the current cluster, and when the cluster resources are insufficient, a conventional scheduling node screening method, such as a load balancing algorithm and a minimum consumption node algorithm, is adopted, so that the problem of Pod scheduling failure waiting occurs with a high probability, and the service quality of online application is affected.
Disclosure of Invention
In view of the above, the embodiments of the present application at least provide a scheduling method, apparatus, electronic device, and storage medium, which can ensure that an online task can be processed in time, and further ensure the quality of service for the online task.
The application mainly comprises the following aspects:
in a first aspect, an embodiment of the present application provides a scheduling method, where the scheduling method includes:
acquiring newly added tasks to be scheduled in the cluster;
if it is determined that at least one candidate scheduling node with residual resource quantity meeting the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node;
if the target scheduling node exists, selecting the target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from nodes belonging to a mixed task processing area in the at least one candidate scheduling node;
and dispatching the task to be dispatched to the target dispatching node.
In a possible implementation manner, the selecting a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node includes:
Counting the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
if at least two nodes exist, selecting the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm;
and if one node exists, taking the node as the target scheduling node.
In one possible implementation manner, if there is no node belonging to the online task processing area in the at least one candidate scheduling node, the scheduling method further includes:
and prompting that the number of the nodes in the online task processing area is insufficient.
In a possible implementation manner, after the target scheduling node is selected from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, the scheduling method further includes:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is larger than or equal to a first preset proportion;
if yes, the fact that the number of the nodes in the mixed task processing area is insufficient is indicated.
In a possible implementation manner, if it is determined that at least one candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled exists in the computing nodes of the cluster, and the task to be scheduled is an offline task, the scheduling method further includes:
Calculating a second resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
In one possible implementation manner, if the second resource occupancy is less than or equal to a second preset proportion, and there is no node belonging to the mixed task processing zone in the at least one candidate scheduling node, the scheduling method further includes:
and refusing to process the task to be scheduled.
In a possible implementation manner, the determining the target scheduling node based on the node belonging to the mixed task processing zone in the at least one candidate scheduling node includes:
calculating the resource residual rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting a node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a possible implementation manner, if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling method further includes:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
In one possible implementation manner, the scheduling method further includes:
and if the time of the task to be scheduled in the mixed task processing area is greater than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, the task to be scheduled is scheduled to the target node of the online task processing area.
In a second aspect, an embodiment of the present application further provides a scheduling apparatus, where the scheduling apparatus includes:
the acquisition module is used for acquiring newly added tasks to be scheduled in the cluster;
the judging module is used for judging whether a node belonging to an online task processing area exists in at least one candidate scheduling node if the fact that the residual resource quantity meets the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task is determined;
The selecting module is used for selecting a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node if the target scheduling node exists; otherwise, selecting the target scheduling node from nodes belonging to a mixed task processing area in the at least one candidate scheduling node;
and the scheduling module is used for scheduling the task to be scheduled to the target scheduling node.
In one possible implementation manner, the selecting module includes:
a statistics unit, configured to count the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
a selecting unit, configured to select, if at least two nodes exist, the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm;
and the determining unit is used for taking the node as the target scheduling node if the node exists.
In one possible implementation manner, the scheduling apparatus further includes:
and the first prompting module is used for prompting that the number of the nodes in the online task processing area is insufficient.
In a possible implementation manner, the scheduling device further comprises a second prompting module; the second prompting module is used for:
Calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is larger than or equal to a first preset proportion;
if yes, the fact that the number of the nodes in the mixed task processing area is insufficient is indicated.
In a possible implementation manner, if it is determined that at least one candidate scheduling node whose remaining resource quantity satisfies the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster, and the task to be scheduled is an offline task, the scheduling device further includes a determining module; the determining module is used for:
calculating a second resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
In one possible implementation manner, if the second resource occupancy is less than or equal to a second preset proportion, and there is no node belonging to the mixed task processing zone in the at least one candidate scheduling node, the determining module is further configured to:
And refusing to process the task to be scheduled.
In a possible implementation manner, the determining module is configured to determine the target scheduling node according to the following steps:
calculating the resource residual rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting a node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a possible implementation manner, if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling device further includes a deletion module; the deleting module is used for:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
In a possible implementation manner, the scheduling module is further configured to:
and if the time of the task to be scheduled in the mixed task processing area is greater than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, the task to be scheduled is scheduled to the target node of the online task processing area.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory in communication via the bus when the electronic device is running, the machine readable instructions when executed by the processor performing the steps of the scheduling method as described in the first aspect or any possible implementation of the first aspect.
In a fourth aspect, the present embodiment further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor performs the steps of the scheduling method described in the first aspect or any possible implementation manner of the first aspect.
According to the scheduling method, the device, the electronic equipment and the storage medium, different task processing areas are preset, so that on-line tasks are scheduled to the on-line task processing areas preferentially, when resources of the on-line task processing areas are insufficient, the tasks to be scheduled can be scheduled to the mixed task processing areas, compared with the conventional scheduling node screening method, such as a load balancing algorithm and a minimum consumption node algorithm, when the cluster resources are insufficient in the prior art, the problem that Pod scheduling failure waiting occurs at a high probability, and further service quality of on-line application can be affected.
Further, in the scheduling method provided by the embodiment of the application, if at least one candidate scheduling node does not have a node belonging to the online task processing area, the insufficient number of nodes in the online task processing area is prompted. Here, when the online task processing area does not meet the requirement of processing the node of the task to be scheduled, the node of the online task processing area is insufficient, namely, the resources of the sub-line task processing area are at insufficient risk, and the operation and maintenance personnel can be reminded to add entity computers into the cluster in advance by sending a prompt message to the operation and maintenance personnel, so that the service requirement is ensured.
Further, in the scheduling method provided by the embodiment of the application, if the time of the task to be scheduled in the mixed task processing area is greater than or equal to the preset time length and the target node meeting the resource quantity required by the task to be scheduled is in the online task processing area, the task to be scheduled is scheduled to the target node in the online task processing area. Here, the application designs a rescheduling function, which can regularly track the condition of the online task of the mixed task processing area and whether the current online task processing area has enough resources, if the online task processing area has enough resources, rescheduling occurs, and the online task of the mixed task processing area meeting the conditions is rescheduled to the online task processing area for processing, so that the processing efficiency of the scheduled task can be improved, and the capability of the mixed task processing area for processing the online task is ensured.
In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a scheduling method according to an embodiment of the present application;
FIG. 2 shows one of the functional block diagrams of a scheduling apparatus according to an embodiment of the present application;
FIG. 3 is a functional block diagram illustrating selected blocks of FIG. 2;
FIG. 4 is a second functional block diagram of a scheduling apparatus according to an embodiment of the present application;
fig. 5 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for the purpose of illustration and description only and are not intended to limit the scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present application. It should be appreciated that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.
In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art based on embodiments of the application without making any inventive effort, fall within the scope of the application.
In order to enable a person skilled in the art to use the present disclosure, in connection with a specific application scenario "how to schedule tasks to be scheduled in a cluster", the following implementation is given, and the general principles defined herein may be applied to other embodiments and application scenarios by a person skilled in the art without departing from the spirit and scope of the present disclosure.
The method, the device, the electronic equipment or the computer readable storage medium can be applied to any scene needing to be scheduled, the embodiment of the application does not limit the specific application scene, and any scheme using the scheduling method and the device provided by the embodiment of the application is within the protection scope of the application.
It should be noted that, before the present application proposes the present application, the actual resource allocation situation of the current cluster is not considered by the K8S HPA technology in the related scheme, and when the cluster resources are insufficient, a conventional scheduling node screening method, such as a load balancing algorithm and a minimum consumption node algorithm, is adopted, so that the problem of Pod scheduling failure waiting occurs with a high probability, and the service quality of online application is further affected.
Aiming at the problems, the embodiment of the application firstly acquires the newly added task to be scheduled in the cluster, if the candidate scheduling node meeting the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster, and the task to be scheduled is an online task, judging whether the node belonging to the online task processing area exists in the candidate scheduling node, and if the node exists, selecting the target scheduling node from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. For the online task to be scheduled to the online task processing area preferentially, when the resources of the online task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the situation that the online task fails to be scheduled due to the increase of scheduled tasks can be avoided, the online task can be guaranteed to be processed in time, and further, the service quality of the online task is guaranteed.
It should be noted that, K8S is an open source application for managing containerization on multiple hosts in the cloud platform; pod is the smallest deployable unit capable of creating and managing K8S calculation, and consists of one or more containers, is also the basic granularity of K8S scheduling, and can apply for a certain amount of resources, such as 2-core CPU and 4G memory, and is a scheduling task for each Pod; deployment is a resource type of K8S, including 0 to multiple copies (Pod), multiple copies (Pod) being equivalent in position; node is a server, also called Node, in the K8S cluster; the Cluster is a K8S Cluster and consists of a control node and a plurality of computing nodes; HPA refers to transverse automatic expansion and contraction technology, and can automatically expand or reduce the copy number of the Deployment according to the occupation condition of CPU, MEM or other user-defined indexes (such as request quantity) of Pod, for example, the CPU occupation rate exceeds 70% of the automatic expansion copy number, and the CPU occupation rate is smaller than the residual 30% of the automatic reduction copy number; the Zone is a set of a plurality of computing nodes, and a plurality of computing nodes in the K8S cluster can be divided into a certain Zone; if the current Pod is scheduled to run on the node, the node is called a scheduling node; if all the computing node residual resources of the current cluster resources can not meet the resource requirements of the Pod to be scheduled, the Pod can enter a Pending state (waiting state), the Pod of Pending and all the current cluster resources meet the Pod scheduling can be detected at regular time in the subsequent flow by K8S scheduling, if yes, scheduling is carried out, and if not, the Pending is continued; online applications, i.e., online tasks, are applications that directly provide online services to users, requiring long-time operation, such as online games, media websites, etc.; the offline task is a timing task, an AI training task and a big data offline computing task which are not used for directly providing services for users in general and are finished after a period of operation.
In order to facilitate understanding of the present application, the following detailed description of the technical solution provided by the present application is provided in connection with specific embodiments.
Fig. 1 is a flowchart of a scheduling method according to an embodiment of the present application. As shown in fig. 1, the scheduling method provided by the embodiment of the application includes the following steps:
s101: and acquiring newly added tasks to be scheduled in the cluster.
In specific implementation, when a newly added task to be scheduled in the cluster is detected, the newly added task to be scheduled is acquired, and the amount of resources required by the task to be scheduled is determined.
The cluster comprises a control node and a plurality of computing nodes, wherein the control node is responsible for distributing scheduling tasks to the computing nodes, and the computing nodes are used for processing the scheduling tasks and providing computing resources for the scheduling tasks.
It should be noted that, in general, the request amount for an online application (online task) is not constant, such as a higher request amount in the daytime and a lower request amount in the evening, or the request amount is increased suddenly when an active day, such as twenty-one, is encountered. When the request volume for an online application is increased, the HPA technique may be employed to automatically increase the scheduled tasks (online tasks) that handle the request depending on how much volume is requested.
Preferably, the main body for executing the scheduling method provided by the application can be a control node in the cluster.
S102: if it is determined that at least one candidate scheduling node with the residual resource quantity meeting the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node.
In specific implementation, for any newly added task to be scheduled, determining the amount of resources required by the task to be scheduled, and further judging whether at least one candidate scheduling node with the residual amount of resources meeting the amount of resources required by the task to be scheduled exists in computing nodes of the cluster, namely, preselecting the computing nodes in the cluster to obtain the candidate scheduling node meeting the amount of resources required by the task to be scheduled, if so, judging the task type of the task to be scheduled, and if so, judging whether nodes belonging to an online task processing area exist in the candidate scheduling nodes meeting the amount of resources required by the task to be scheduled.
It should be noted that, in the present application, the computing nodes in the cluster are divided into different task processing areas in advance, where there are two task processing areas, namely, an online task processing area and a mixed task processing area, where the computing nodes in the online task processing area are only responsible for processing online tasks, and the computing nodes in the mixed task processing area may be responsible for processing online tasks and offline tasks, and considering that the online tasks need to be processed in time, and the offline tasks may be processed in a delayed manner, for this reason, the priority of the online tasks is set to be higher than the priority of the offline tasks. Generally, the amount of resources in the on-line task processing section is larger than that in the mixed task processing section, so that the on-line task can be sufficiently processed, and the on-line task processing section may preferably have a resource ratio of more than 80%.
Here, one computing node may process a plurality of scheduling tasks at the same time, and the scheduling tasks may be allocated to only one computing node for processing.
In an example, the cluster has three computing nodes, namely a computing node A, a computing node B and a computing node C, wherein the total amount of CPU of the computing node A is 20 cores, the current residual amount of CPU is 10 cores, the total amount of CPU of the computing node B is 20 cores, the current residual amount of CPU is 5 cores, the total amount of CPU of the computing node C is 20 cores, and the current residual amount of CPU is 3 cores. The amount of resources required by the task to be scheduled is CPU6 core, the remaining CPU 5 core of the computing node B and the remaining CPU3 core of the computing node C do not meet the amount of resources required by the task to be scheduled, and only the remaining CPU10 core of the computing node A meets the amount of resources required by the task to be scheduled CPU6 core, the computing node A is selected from the computing node A, the computing node B and the computing node C as candidate scheduling nodes.
Here, after determining in step S102 whether the at least one candidate scheduling node has a node belonging to the online task processing area, determining that the at least one candidate scheduling node does not have a node belonging to the online task processing area, which indicates that the online task processing area cannot process a task to be scheduled for an online task, and that resources of the online task processing area are insufficient, at this time, it is necessary to prompt an operator to increase an entity computer, that is, prompt that the number of nodes in the online task processing area is insufficient, so that the operator increases computing nodes in the online task processing area, thereby avoiding a situation that a request cannot be processed in time due to insufficient resources of the online task processing area.
Further, the task to be scheduled may be an online task or an offline task, if it is determined that at least one candidate scheduling node whose residual resource quantity meets the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster, and the task to be scheduled is an offline task, the method further includes the following steps:
calculating a second resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area; judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion; if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node; if not, refusing to process the task to be scheduled.
In a specific implementation, if it is determined that the task to be scheduled is an offline task, the online task processing area can only process the online task, and the mixed task processing area can also process the offline task, so that the task to be scheduled can only be scheduled to the mixed task processing area, but it is now required to determine whether a node belonging to the mixed task processing area exists among candidate scheduling nodes meeting the resource amount required by the task to be scheduled, and meanwhile it is also required to determine whether the resource amount required by the offline task of the mixed task processing area has reached a threshold value, if the threshold value is reached, because the priority of the online task is higher than that of the offline task, the task to be scheduled cannot be processed no matter whether the computing node meeting the requirement exists in the mixed task processing area, so that the mixed task processing area has a space for processing the online task, specifically, if the second resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area is determined to be smaller than or equal to a second preset proportion, selecting a target scheduling node from nodes belonging to the mixed task processing area among the candidate scheduling nodes, if the second resource occupancy rate is determined to be smaller than or equal to the second preset proportion, and if the second resource occupancy rate is determined to be greater than the second preset proportion, rejecting to process the mixed task to be scheduled to be processed by the second preset resource occupancy rate, and then waiting for the offline task to be processed by the second preset proportion to be smaller than the second preset proportion.
Here, the second preset ratio may be set according to an actual resource amount of the mixed task processing zone and a resource amount required for the off-line task, and the second preset ratio is preferably set to be greater than 50%, wherein a sum of the first preset ratio and the second preset ratio is 1.
Further, if the second resource occupancy rate is less than or equal to a second preset proportion, and no node belonging to the mixed task processing area exists in the at least one candidate scheduling node, the method further includes the following steps: and refusing to process the task to be scheduled.
In a specific implementation, since the online task processing area can only process online tasks, the mixed task processing area can process online tasks as well as offline tasks, if the second resource occupancy rate is smaller than or equal to a second preset proportion, but no node belonging to the mixed task processing area meeting the resources required by the tasks to be scheduled exists in the candidate scheduling nodes, at this time, the task to be scheduled can be made to process a waiting state, the task to be scheduled is refused to process at present, and when the node belonging to the mixed task processing area meeting the resources required by the task to be scheduled exists, the task to be scheduled is processed.
In this case, if the task to be scheduled is an online task, because the priority of the online task is higher than that of the offline task, preemptive scheduling is started, and the scheduled task with the lowest priority in the mixed task processing area is paused or deleted, so that the candidate scheduled node is screened for the task to be scheduled again. In addition, among the computing nodes of the cluster, there is no candidate scheduling node with the residual resource quantity meeting the resource quantity required by the task to be scheduled, which indicates that the resource of the cluster is insufficient, if the task to be scheduled is an offline task, the processing can be carried out after waiting for the resource meeting the requirement because the priority of the offline task is lower.
Here, after the task to be scheduled, which is an online task, is scheduled to a mixed task processing area, there may be a case that the task to be scheduled is not processed in time, and during this period, there may be a node meeting the processing of the task to be scheduled in the online task processing area, and the task to be scheduled may be rescheduled to a target node meeting the resource requirement in the online task processing area, that is, if the time of the task to be scheduled in the mixed task processing area is greater than or equal to a preset duration, and there is a target node meeting the resource quantity required by the task to be scheduled in the online task processing area, the task to be scheduled is scheduled to the target node in the online task processing area.
It should be noted that, the capacity expansion sequence of the Pod copies (scheduling tasks) in the K8s cluster is strongly related to the capacity reduction sequence, for example, there is one copy P1 originally, there are P1 and P2 after expansion to two copies, and then capacity reduction to one copy is performed, so that after a series of capacity expansion/capacity reduction scheduling in the K8s cluster, there may be sufficient resources in the online task processing area due to the residual P1, but the scheduling tasks of the online tasks still stay in the mixed task processing area for a long time, and if this happens in a large amount, the mixed task processing area will be greatly piled up with the Pod of the online tasks, so that the mixed task processing area loses the guarantee function. Therefore, the application designs a rescheduling function, which can regularly track the condition of online tasks in the online task processing area and whether the current online task processing area has enough resources, and rescheduling occurs if the online task processing area has enough resources, and rescheduling online tasks in the mixed task processing area meeting the conditions to the online task processing area.
S103: if the target scheduling node exists, selecting the target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from nodes belonging to the mixed task processing area in the at least one candidate scheduling node.
In a specific implementation, among computing nodes of the cluster, candidate scheduling nodes meeting the resource amount required by the task to be scheduled are available, the task to be scheduled is determined to be an online task, further, whether nodes belonging to an online task processing area exist in the candidate scheduling nodes meeting the resource amount required by the task to be scheduled is judged, if yes, the task to be scheduled is preferably scheduled to the nodes in the online task processing area, if no, the node meeting the resource amount required by the task to be scheduled is not found in the nodes in the online task processing area, and at the moment, a target scheduling node can be selected from the nodes belonging to a mixed task processing area in the candidate scheduling nodes meeting the resource amount required by the task to be scheduled to process the task to be scheduled.
Here, in step S102, selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node includes:
Counting the number of nodes belonging to the online task processing area in the at least one candidate scheduling node; if at least two nodes exist, selecting the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm; and if one node exists, taking the node as the target scheduling node.
In the step, after determining that the task to be scheduled is an online task, preferably scheduling the task to be scheduled on a node of an online task processing area, firstly judging whether a node belonging to the online task processing area exists in at least one candidate scheduling node meeting the resource required by the task to be scheduled, if so, selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node, firstly counting the number of the nodes belonging to the online task processing area in the at least one candidate scheduling node, and if only one node meeting the requirement, directly determining the node as the target scheduling node; if more than two nodes meeting the requirements exist, the target scheduling node can be screened out from at least two nodes meeting the requirements according to a preset scheduling node screening algorithm.
It should be noted that, the preset scheduling node screening algorithm is a conventional scheduling node screening method, for example, a load balancing algorithm (bandwidth_resource_allocation), a minimum consumption node algorithm (drain_request), a node tendency matching algorithm (node_preference_average_points), a selector_spreading algorithm (selector_spreading), and the like, where the load balancing algorithm is a node that selects a node with the most balanced utilization rate of each resource from a node list, and the minimum consumption node algorithm is a node with the least consumption (the CPU (total capacity-sum (used) is estimated by 10/total capacity) according to an idle ratio).
It should be noted that, when the request amount for online application is suddenly increased, the HPA technology may be adopted to automatically increase the scheduling task for processing the request according to the amount of the request amount, so as to avoid the situation that the newly increased scheduling task cannot be processed in time due to the increase of the scheduling task, a reasonable scheduling scheme needs to be provided, and the newly increased scheduling task is processed under the situation that the number of computing nodes in the cluster is unchanged as much as possible. In this way, the cluster nodes are divided into two task processing areas, one is an online task processing area, and the other is a mixed task processing area, wherein the online task processing area is only responsible for processing online tasks, and the mixed task processing area is only used for processing offline tasks under the condition that the resources of the online task processing area are sufficient, and can process online tasks besides offline tasks under the condition that the resources of the online task processing area are insufficient, so that the online tasks can be timely processed, the situation that scheduling failure occurs to the online tasks due to the increase of scheduling tasks can be avoided, and further, the service quality of the online tasks is guaranteed.
Further, in step S103, after selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, the method further includes the following steps: calculating a first resource occupancy rate of an online task in the mixed task processing area; judging whether the first resource occupancy rate is larger than or equal to a first preset proportion; if yes, the fact that the number of the nodes in the mixed task processing area is insufficient is indicated.
In a specific implementation, if the task to be scheduled is an online task and there is no node belonging to the online task processing area in the candidate scheduling nodes, it is indicated that there is no node satisfying the amount of resources required by the task to be scheduled in the nodes in the online task processing area, and at this time, a target scheduling node may be selected from the nodes belonging to the mixed task processing area in the candidate scheduling nodes satisfying the amount of resources required by the task to be scheduled to process the task to be scheduled. After the tasks to be scheduled, which are online tasks, are scheduled to the target scheduling nodes in the mixed task processing area, the proportion of resources corresponding to the online tasks in the mixed task processing area to the total resources in the mixed task processing area, namely, the first resource occupancy rate, is calculated, whether the first resource occupancy rate is larger than or equal to a first preset proportion is judged, if the first resource occupancy rate is larger than the first preset proportion, more online tasks are scheduled to the mixed task processing area, and because the mixed task processing area is mainly used for processing offline tasks, the situation that the resources in the mixed task processing area are insufficient is described, and the insufficient number of nodes in the mixed task processing area needs to be prompted to an operation and maintenance personnel is needed, so that the operation and maintenance personnel can timely add a computer of an entity to increase the number of calculation nodes in the mixed task processing area. According to the application, when the first resource occupancy rate of the online tasks in the mixed task processing area is larger than or equal to the first preset proportion, insufficient number of nodes in the mixed task processing area is prompted to operation and maintenance personnel, and at the moment, the mixed task processing area can also process scheduling tasks, so that the time for adding computer equipment to the operation and maintenance personnel is reserved, and before the resources of the mixed task processing area are completely used, the computing nodes are additionally arranged, so that the situation that the requests cannot be timely processed due to insufficient resources of the offline task processing area is avoided.
Here, the first preset ratio may be set according to the actual amount of resources of the mixed task processing section and the amount of resources required for the off-line task, and the first preset ratio is preferably set to be less than 50%.
It should be noted that when the resources of the on-line task processing area are insufficient or the resources of the mixed task processing area reach a certain on-line state, an alarm is triggered to remind cluster operation and maintenance personnel to increase server nodes in advance, so that the cluster can process the request from the user in time.
Further, the step S103 of determining the target scheduling node based on the node belonging to the mixed task processing zone in the at least one candidate scheduling node includes the steps of:
calculating the resource residual rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node; and selecting a node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a specific implementation, for a task to be scheduled, which is an offline task, a node with a small amount of residual resources meeting the requirement of the task to be scheduled is selected as a target scheduling node to process the task to be scheduled, specifically, the resource residual rate corresponding to each node belonging to a mixed task processing area in candidate scheduling nodes is calculated, and a node with the minimum resource residual rate is selected from nodes belonging to the mixed task processing area in the candidate scheduling nodes to be determined as the target scheduling node. In this way, the mixed task processing section can be provided with computing nodes to process online tasks.
Here, if the nodes only include cpu and memory resources, the calculation formula of the resource remaining rate of each node is as follows:
Score＝(UsedCpu/cpuNodeCap acity+UsedMemory/MemoryNode Capacity)/2
the CPU NodeCapacity is the total amount of the node CPU, and usedCpu is the amount of the applied CPU; the memory NodeCapacity is the total amount of node memory, and the usedmemory is the applied memory amount.
S104: and dispatching the task to be dispatched to the target dispatching node.
In a specific implementation, after determining a target scheduling node for processing the task to be scheduled, the task to be scheduled is scheduled to the target scheduling node, that is, the task to be scheduled is allocated to the target scheduling node.
It should be noted that, by partitioning the cluster (on-line task processing area and mixed task processing area) in advance and defining the priority of the scheduled task (high on-line task priority and low off-line task priority) and matching with the self-defined target scheduling node screening method, the application can effectively avoid the problem of HPA capacity expansion failure (scheduling task increase), and simultaneously improves the utilization of resources in the mixed task processing area through the mixed part of the on-line task processing area and the mixed task processing area, and can timely supplement the cluster nodes through an alarm mechanism.
In the embodiment of the application, newly added tasks to be scheduled in a cluster are acquired, if it is determined that candidate scheduling nodes meeting the resource quantity required by the tasks to be scheduled exist in computing nodes of the cluster, the tasks to be scheduled are online tasks, whether the nodes belonging to an online task processing area exist in the candidate scheduling nodes is judged, and if the nodes exist, a target scheduling node is selected from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. For the online task to be scheduled to the online task processing area preferentially, when the resources of the online task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the situation that the online task fails to be scheduled due to the increase of scheduled tasks can be avoided, the online task can be guaranteed to be processed in time, and further, the service quality of the online task is guaranteed.
Based on the same application conception, the embodiment of the present application further provides a scheduling device corresponding to the scheduling method provided in the foregoing embodiment, and since the principle of solving the problem by the device in the embodiment of the present application is similar to that of the scheduling method in the foregoing embodiment of the present application, the implementation of the device may refer to the implementation of the method, and the repetition is omitted.
FIG. 2 shows one of the functional block diagrams of a scheduling apparatus 200 according to an embodiment of the present application; FIG. 3 illustrates a functional block diagram of the selection module 230 of FIG. 2; fig. 4 shows a second functional block diagram of a scheduling apparatus 200 according to an embodiment of the application.
As shown in fig. 2, the scheduling apparatus 200 includes:
an obtaining module 210, configured to obtain a task to be scheduled newly added in the cluster;
a judging module 220, configured to judge whether a node belonging to an online task processing area exists in at least one candidate scheduling node if it is determined that the remaining resource quantity satisfies the resource quantity required by the task to be scheduled and the task to be scheduled is an online task in the computing nodes of the cluster;
a selecting module 230, configured to select, if any, a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from nodes belonging to a mixed task processing area in the at least one candidate scheduling node;
And the scheduling module 240 is configured to schedule the task to be scheduled to the target scheduling node.
In one possible implementation, as shown in fig. 3, the selecting module 230 includes:
a statistics unit 231 configured to count the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
a selecting unit 232, configured to select, if there are at least two nodes, the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm;
the determining unit 233 is configured to take a node as the target scheduling node if the node exists.
In one possible implementation, as shown in fig. 4, the scheduling apparatus 200 further includes:
a first prompting module 250, configured to prompt that the number of nodes in the online task processing area is insufficient.
In one possible implementation, as shown in fig. 4, the scheduling apparatus 200 further includes a second prompting module 260; the second prompting module 260 is configured to:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is larger than or equal to a first preset proportion;
if yes, the fact that the number of the nodes in the mixed task processing area is insufficient is indicated.
In a possible implementation manner, as shown in fig. 4, if it is determined that at least one candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled exists in the computing nodes of the cluster, and the task to be scheduled is an offline task, the scheduling apparatus 200 further includes a determining module 270; the determining module 270 is configured to:
calculating a second resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
In a possible implementation manner, as shown in fig. 4, if the second resource occupancy is less than or equal to a second preset proportion and there is no node belonging to the mixed task processing area in the at least one candidate scheduling node, the determining module 270 is further configured to:
and refusing to process the task to be scheduled.
In a possible implementation manner, as shown in fig. 4, the determining module 270 is configured to determine the target scheduling node according to the following steps:
Calculating the resource residual rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting a node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a possible implementation manner, as shown in fig. 4, if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling apparatus 200 further includes a deletion module 280; the deletion module 280 is configured to:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
In one possible implementation, as shown in fig. 4, the scheduling module 240 is further configured to:
and if the time of the task to be scheduled in the mixed task processing area is greater than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, the task to be scheduled is scheduled to the target node of the online task processing area.
Based on the same application concept, referring to fig. 5, a schematic structural diagram of an electronic device 500 according to an embodiment of the present application includes: a processor 510, a memory 520 and a bus 530, the memory 520 storing machine readable instructions executable by the processor 510, the processor 510 and the memory 520 communicating via the bus 530 when the electronic device 500 is running, the machine readable instructions being executable by the processor 510 to perform the steps of the scheduling method as described in any of the above embodiments.
In particular, the machine-readable instructions, when executed by the processor 510, may perform the following:
acquiring newly added tasks to be scheduled in the cluster;
if it is determined that at least one candidate scheduling node with residual resource quantity meeting the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node;
if the target scheduling node exists, selecting the target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from nodes belonging to a mixed task processing area in the at least one candidate scheduling node;
And dispatching the task to be dispatched to the target dispatching node.
In the embodiment of the application, newly added tasks to be scheduled in a cluster are acquired, if it is determined that candidate scheduling nodes meeting the resource quantity required by the tasks to be scheduled exist in computing nodes of the cluster, the tasks to be scheduled are online tasks, whether the nodes belonging to an online task processing area exist in the candidate scheduling nodes is judged, and if the nodes exist, a target scheduling node is selected from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. For the online task to be scheduled to the online task processing area preferentially, when the resources of the online task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the situation that the online task fails to be scheduled due to the increase of scheduled tasks can be avoided, the online task can be guaranteed to be processed in time, and further, the service quality of the online task is guaranteed.
Based on the same application concept, the embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of the scheduling method provided in the above embodiment are executed.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, when a computer program on the storage medium is run, the scheduling method can be executed, and for online tasks, the online tasks are scheduled to an online task processing area preferentially, and when resources of the online task processing area are insufficient, the tasks to be scheduled can be scheduled to a mixed task processing area, so that the situation that the online tasks are failed to be scheduled due to the increase of the scheduled tasks can be avoided, the online tasks can be guaranteed to be processed timely, and further, the service quality of the online tasks is guaranteed.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily appreciate variations or alternatives within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (12)
1. A scheduling method, the scheduling method comprising:
acquiring newly added tasks to be scheduled in the cluster;
determining at least one candidate scheduling node with residual resource quantity meeting the resource quantity required by the task to be scheduled in the computing nodes of the cluster; wherein the computing nodes of the cluster are divided into an online task processing area and a mixed task processing area;
if the task to be scheduled is determined to be an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node; if the target scheduling node exists, selecting the target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from nodes belonging to a mixed task processing area in the at least one candidate scheduling node; the resource amount of the online task processing area is larger than that of the mixed task processing area;
Scheduling the task to be scheduled to the target scheduling node;
if the task to be scheduled is determined to be an offline task, refusing to process the task to be scheduled when the second resource occupancy rate is determined to be larger than a second preset proportion; and the second resource occupancy rate is the resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area.
2. The scheduling method according to claim 1, wherein selecting a target scheduling node from the nodes belonging to the online task processing area among the at least one candidate scheduling node comprises:
counting the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
if at least two nodes exist, selecting the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm;
and if one node exists, taking the node as the target scheduling node.
3. The scheduling method of claim 1, wherein if there is no node belonging to the online task processing region among the at least one candidate scheduling node, the scheduling method further comprises:
And prompting that the number of the nodes in the online task processing area is insufficient.
4. The scheduling method according to claim 1, wherein after selecting the target scheduling node from among the nodes belonging to the mixed task processing zone among the at least one candidate scheduling node, the scheduling method further comprises:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is larger than or equal to a first preset proportion;
if yes, the fact that the number of the nodes in the mixed task processing area is insufficient is indicated.
5. The scheduling method according to claim 1, wherein after determining that the task to be scheduled is an offline task, the scheduling method further comprises:
calculating a second resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion;
if yes, determining the target scheduling node based on the node belonging to the mixed task processing area in the at least one candidate scheduling node, and scheduling the task to be scheduled to the target scheduling node.
6. The scheduling method of claim 5, wherein if the second resource occupancy is less than or equal to a second preset proportion and there is no node belonging to the mixed task processing zone in the at least one candidate scheduling node, the scheduling method further comprises:
and refusing to process the task to be scheduled.
7. The scheduling method of claim 5, wherein the determining the target scheduling node based on the node belonging to the mixed task processing zone among the at least one candidate scheduling node comprises:
calculating the resource residual rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting a node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
8. The scheduling method according to claim 1, wherein if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled among the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling method further comprises:
And deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
9. The scheduling method of claim 8, wherein the scheduling method further comprises:
and if the time of the task to be scheduled in the mixed task processing area is greater than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, the task to be scheduled is scheduled to the target node of the online task processing area.
10. A scheduling apparatus, characterized in that the scheduling apparatus comprises:
the acquisition module is used for acquiring newly added tasks to be scheduled in the cluster;
the judging module is used for determining at least one candidate scheduling node with the residual resource quantity meeting the resource quantity required by the task to be scheduled in the computing nodes of the cluster; wherein the computing nodes of the cluster are divided into an online task processing area and a mixed task processing area; if the task to be scheduled is determined to be an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node;
The selecting module is used for selecting a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node if the target scheduling node exists; otherwise, selecting the target scheduling node from nodes belonging to a mixed task processing area in the at least one candidate scheduling node; the resource amount of the online task processing area is larger than that of the mixed task processing area;
the scheduling module is used for scheduling the task to be scheduled to the target scheduling node;
the scheduling device further comprises a determining module; the determining module is used for: if the task to be scheduled is determined to be an offline task, refusing to process the task to be scheduled when the second resource occupancy rate is determined to be larger than a second preset proportion; and the second resource occupancy rate is the resource occupancy rate of the offline task containing the task to be scheduled in the mixed task processing area.
11. An electronic device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating via said bus when the electronic device is running, said machine readable instructions when executed by said processor performing the steps of the scheduling method according to any one of claims 1 to 9.
12. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the scheduling method according to any of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299381.XA CN112269641B (en) | 2020-11-18 | 2020-11-18 | Scheduling method, scheduling device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299381.XA CN112269641B (en) | 2020-11-18 | 2020-11-18 | Scheduling method, scheduling device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112269641A CN112269641A (en) | 2021-01-26 |
| CN112269641B true CN112269641B (en) | 2023-09-15 |
Family
ID=74340705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011299381.XA Active CN112269641B (en) | 2020-11-18 | 2020-11-18 | Scheduling method, scheduling device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269641B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113382077B (en) * | 2021-06-18 | 2023-05-23 | 广西电网有限责任公司 | Micro-service scheduling method, micro-service scheduling device, computer equipment and storage medium |
| CN113590317A (en) * | 2021-07-27 | 2021-11-02 | 杭州朗和科技有限公司 | Scheduling method, device, medium and computing equipment of offline service |
| CN113760549B (en) * | 2021-08-30 | 2024-03-15 | 聚好看科技股份有限公司 | Pod deployment method and device |
| CN114401263A (en) * | 2022-01-21 | 2022-04-26 | 中兴飞流信息科技有限公司 | Image ETL access elastic processing analysis method and system based on microservice implementation |
| CN114629960B (en) * | 2022-03-14 | 2023-09-19 | 抖音视界有限公司 | Resource scheduling method, device, system, equipment, medium and program product |
| CN114995997A (en) * | 2022-04-24 | 2022-09-02 | 阿里巴巴（中国）有限公司 | Task processing method |
| CN114979282B (en) * | 2022-07-28 | 2023-01-20 | 北京金山云网络技术有限公司 | Task scheduling method, device, storage medium and electronic equipment |
| CN115454680B (en) * | 2022-10-12 | 2023-07-07 | 中航信移动科技有限公司 | Application control system |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105843670A (en) * | 2016-03-22 | 2016-08-10 | 浙江大学 | Cloud platform based virtual cluster deployment and integration method |
| CN107621973A (en) * | 2016-07-13 | 2018-01-23 | 阿里巴巴集团控股有限公司 | A kind of method for scheduling task and device across cluster |
| CN107968810A (en) * | 2016-10-20 | 2018-04-27 | 阿里巴巴集团控股有限公司 | A kind of resource regulating method of server cluster, device and system |
| CN109597688A (en) * | 2018-11-23 | 2019-04-09 | 金色熊猫有限公司 | Online resource management method, device, storage medium and electronic equipment |
| CN110647394A (en) * | 2018-06-27 | 2020-01-03 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and equipment |
| CN110830391A (en) * | 2018-08-10 | 2020-02-21 | 阿里巴巴集团控股有限公司 | Resource allocation method and device and cluster system |
| CN111026553A (en) * | 2019-12-13 | 2020-04-17 | 深圳先进技术研究院 | Resource scheduling method for offline mixed part operation and server system |
| CN111399989A (en) * | 2020-04-10 | 2020-07-10 | 中国人民解放军国防科技大学 | Task preemption scheduling method and system for container cloud |
| CN111459678A (en) * | 2020-04-02 | 2020-07-28 | 上海极链网络科技有限公司 | Resource scheduling method and device, storage medium and electronic equipment |
| CN111506423A (en) * | 2020-04-09 | 2020-08-07 | 网易（杭州）网络有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111722928A (en) * | 2020-06-12 | 2020-09-29 | 北京字节跳动网络技术有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111736957A (en) * | 2020-06-29 | 2020-10-02 | 平安普惠企业管理有限公司 | Multi-type service mixed deployment method, device, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10310458B2 (en) * | 2015-12-22 | 2019-06-04 | Schneider Electric Software, Llc | Process optimization by grouping mixed integer nonlinear programming constraints |
-
2020
- 2020-11-18 CN CN202011299381.XA patent/CN112269641B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105843670A (en) * | 2016-03-22 | 2016-08-10 | 浙江大学 | Cloud platform based virtual cluster deployment and integration method |
| CN107621973A (en) * | 2016-07-13 | 2018-01-23 | 阿里巴巴集团控股有限公司 | A kind of method for scheduling task and device across cluster |
| CN107968810A (en) * | 2016-10-20 | 2018-04-27 | 阿里巴巴集团控股有限公司 | A kind of resource regulating method of server cluster, device and system |
| CN110647394A (en) * | 2018-06-27 | 2020-01-03 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and equipment |
| CN110830391A (en) * | 2018-08-10 | 2020-02-21 | 阿里巴巴集团控股有限公司 | Resource allocation method and device and cluster system |
| CN109597688A (en) * | 2018-11-23 | 2019-04-09 | 金色熊猫有限公司 | Online resource management method, device, storage medium and electronic equipment |
| CN111026553A (en) * | 2019-12-13 | 2020-04-17 | 深圳先进技术研究院 | Resource scheduling method for offline mixed part operation and server system |
| CN111459678A (en) * | 2020-04-02 | 2020-07-28 | 上海极链网络科技有限公司 | Resource scheduling method and device, storage medium and electronic equipment |
| CN111506423A (en) * | 2020-04-09 | 2020-08-07 | 网易（杭州）网络有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111399989A (en) * | 2020-04-10 | 2020-07-10 | 中国人民解放军国防科技大学 | Task preemption scheduling method and system for container cloud |
| CN111722928A (en) * | 2020-06-12 | 2020-09-29 | 北京字节跳动网络技术有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111736957A (en) * | 2020-06-29 | 2020-10-02 | 平安普惠企业管理有限公司 | Multi-type service mixed deployment method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112269641A (en) | 2021-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112269641B (en) | Scheduling method, scheduling device, electronic equipment and storage medium | |
| US9442763B2 (en) | Resource allocation method and resource management platform | |
| US20210149737A1 (en) | Method for fast scheduling for balanced resource allocation in distributed and collaborative container platform environment | |
| CN105159782B (en) | Based on the method and apparatus that cloud host is Order splitting resource | |
| CN110647394B (en) | Resource allocation method, device and equipment | |
| CN110515704B (en) | Resource scheduling method and device based on Kubernetes system | |
| CN111506423B (en) | Resource scheduling method and device, electronic equipment and storage medium | |
| CN110474966B (en) | Method for processing cloud platform resource fragments and related equipment | |
| CN111966500A (en) | Resource scheduling method and device, electronic equipment and storage medium | |
| US10884667B2 (en) | Storage controller and IO request processing method | |
| CN112783659B (en) | Resource allocation method and device, computer equipment and storage medium | |
| CN110244901B (en) | Task allocation method and device and distributed storage system | |
| CN112073532B (en) | Resource allocation method and device | |
| KR20230073315A (en) | Resource scheduling method and system, electronic device, and computer readable storage medium | |
| CN112860383A (en) | Cluster resource scheduling method, device, equipment and storage medium | |
| CN113110927A (en) | Task scheduling method and device, computer equipment and storage medium | |
| CN109522113B (en) | Memory management method and device | |
| CN116743755A (en) | Resource scheduling method, computer equipment and storage medium of customer service scene | |
| CN113986458A (en) | Container set scheduling method, device, equipment and storage medium | |
| CN113760549A (en) | Pod deployment method and device | |
| CN113672347A (en) | Container group scheduling method and device | |
| CN113992666A (en) | Cloud resource allocation method, system, device and storage medium | |
| CN115311123B (en) | Pixel stream GPU resource scheduling method and device | |
| CN115168057B (en) | Resource scheduling method and device based on k8s cluster | |
| CN109241053B (en) | Identification code allocation method, device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |