CN112269641A - Scheduling method, scheduling device, electronic equipment and storage medium - Google Patents
Scheduling method, scheduling device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112269641A CN112269641A CN202011299381.XA CN202011299381A CN112269641A CN 112269641 A CN112269641 A CN 112269641A CN 202011299381 A CN202011299381 A CN 202011299381A CN 112269641 A CN112269641 A CN 112269641A
- Authority
- CN
- China
- Prior art keywords
- task
- node
- scheduling
- processing area
- scheduled
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a scheduling method, a scheduling device, an electronic device and a storage medium. Acquiring a newly added task to be scheduled in a cluster, if determining that candidate scheduling nodes meeting the resource quantity required by the task to be scheduled exist in computing nodes of the cluster and the task to be scheduled is an online task, judging whether nodes belonging to an online task processing area exist in the candidate scheduling nodes or not, and if so, selecting a target scheduling node from the nodes of the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. And for the on-line task to be preferentially scheduled to the on-line task processing area, when the resources of the on-line task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the condition that the on-line task fails to be scheduled due to the increase of the scheduling task can be avoided, the on-line task can be ensured to be processed in time, and further, the service quality of the on-line task is ensured.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a scheduling method, an apparatus, an electronic device, and a storage medium.
Background
The container technology is a current popular technology, a bottom layer architecture based on K8S (Kubernetes) cluster design is popular, and software is easier to deploy. Typically, some internet online application is deployed on K8S, typically in the form of a controller model, Deployment. The request volume of the online application is generally not constant, such as the request volume is relatively high in the daytime and relatively low in the evening, or the request volume is suddenly increased when an activity day is met. How to allocate resources for such applications with non-constant requests is an abnormal pain point problem under the traditional deployment architecture based on physical machines/virtual machines. If the resources are allocated according to the maximum request amount, when the request amount is reduced, a great deal of resources are wasted, the resources are allocated according to a lower standard, and when the request amount is increased, the capacity expansion is not timely, so that the quality of service of online application is affected.
Under a Deployment architecture based on K8S, K8S provides a Horizontal automatic capacity expansion and reduction technology (HPA) technology, which can automatically scale copies according to the memory overhead of the minimum deployable unit (Pod) or other custom indicators (such as a request amount) calculated by creating and managing K8S under the Deployment environment, and expand the capacity if the CPU utilization rate is greater than a certain threshold (such as 70%), or reduce the capacity if the CPU utilization rate is lower than a certain threshold (such as 30%). However, in an actual scenario, the HPA technique of K8S does not consider the actual resource allocation situation of the current cluster, and when the cluster resources are insufficient, a traditional scheduling node screening method, such as a load balancing algorithm and a minimum consumption node algorithm, is adopted, there is a problem that Pod scheduling fails to wait with a high probability, and the service quality of online application is affected.
Disclosure of Invention
In view of this, embodiments of the present application at least provide a scheduling method, an apparatus, an electronic device, and a storage medium, which can ensure that an online task can be processed in time, and further ensure quality of service for the online task.
The application mainly comprises the following aspects:
in a first aspect, an embodiment of the present application provides a scheduling method, where the scheduling method includes:
acquiring a newly added task to be scheduled in a cluster;
if it is determined that at least one candidate scheduling node with the residual resource amount meeting the resource amount required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node;
if yes, selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node;
and scheduling the task to be scheduled to the target scheduling node.
In a possible implementation manner, the selecting a target scheduling node from among nodes belonging to the online task processing area in the at least one candidate scheduling node includes:
counting the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
if at least two nodes exist, selecting the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm;
and if one node exists, taking the node as the target scheduling node.
In a possible implementation manner, if there is no node belonging to the online task processing area in the at least one candidate scheduling node, the scheduling method further includes:
and prompting that the number of the nodes in the online task processing area is insufficient.
In a possible implementation manner, after the selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, the scheduling method further includes:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is greater than or equal to a first preset proportion or not;
and if so, prompting that the number of the nodes in the mixed task processing area is insufficient.
In a possible implementation manner, if it is determined that there is at least one candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an offline task, the scheduling method further includes:
calculating a second resource occupancy rate of an offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion or not;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
In a possible implementation manner, if the second resource occupancy is less than or equal to a second preset ratio and there is no node belonging to the mixed task processing area in the at least one candidate scheduling node, the scheduling method further includes:
and refusing to process the task to be scheduled.
In a possible embodiment, the determining the target scheduling node based on the node belonging to the mixed task processing area in the at least one candidate scheduling node includes:
calculating a resource surplus rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting the node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a possible implementation manner, if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling method further includes:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
In a possible implementation, the scheduling method further includes:
and if the time of the task to be scheduled in the mixed task processing area is longer than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, scheduling the task to be scheduled to the target node in the online task processing area.
In a second aspect, an embodiment of the present application further provides a scheduling apparatus, where the scheduling apparatus includes:
the acquisition module is used for acquiring a newly added task to be scheduled in the cluster;
the judging module is used for judging whether a node belonging to an online task processing area exists in at least one candidate scheduling node if the computing nodes of the cluster are determined that the residual resource amount meets the resource amount required by the task to be scheduled and the task to be scheduled is an online task;
a selecting module, configured to select a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node, if the target scheduling node exists; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node;
and the scheduling module is used for scheduling the task to be scheduled to the target scheduling node.
In a possible implementation, the selecting module includes:
a counting unit, configured to count the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
a selecting unit, configured to select the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm if there are at least two nodes;
and the determining unit is used for taking a node as the target scheduling node if the node exists.
In a possible implementation, the scheduling apparatus further includes:
and the first prompting module is used for prompting that the number of the nodes in the online task processing area is insufficient.
In a possible implementation, the scheduling apparatus further includes a second prompting module; the second prompting module is used for:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is greater than or equal to a first preset proportion or not;
and if so, prompting that the number of the nodes in the mixed task processing area is insufficient.
In a possible implementation manner, if it is determined that at least one candidate scheduling node exists in the computing nodes of the cluster, where the remaining resource amount satisfies the resource amount required by the task to be scheduled, and the task to be scheduled is an offline task, the scheduling apparatus further includes a determining module; the determination module is to:
calculating a second resource occupancy rate of an offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion or not;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
In a possible implementation manner, if the second resource occupancy is less than or equal to a second preset ratio and there is no node belonging to the mixed task processing area in the at least one candidate scheduling node, the determining module is further configured to:
and refusing to process the task to be scheduled.
In a possible implementation manner, the determining module is configured to determine the target scheduling node according to the following steps:
calculating a resource surplus rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting the node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a possible implementation manner, if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling apparatus further includes a deletion module; the deletion module is used for:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
In one possible embodiment, the scheduling module is further configured to:
and if the time of the task to be scheduled in the mixed task processing area is longer than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, scheduling the task to be scheduled to the target node in the online task processing area.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the memory communicate with each other through the bus, and the machine-readable instructions are executed by the processor to perform the steps of the scheduling method according to the first aspect or any one of the possible implementation manners of the first aspect.
In a fourth aspect, this application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the scheduling method described in the first aspect or any possible implementation manner of the first aspect.
According to the scheduling method, the scheduling device, the electronic equipment and the storage medium, different task processing areas are preset, the online task is preferentially scheduled to the online task processing area, when the resources of the online task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, compared with the prior art that when the cluster resources are insufficient, the traditional scheduling node screening method is adopted, for example, a load balancing algorithm and a minimum consumption node algorithm have a high probability of causing the problem of Pod scheduling failure waiting, and further the service quality of online application is influenced, the scheduling method, the device and the electronic equipment can avoid the situation that the scheduling failure of the online task is caused by the increase of the scheduling task, can ensure that the online task can be processed in time, and further guarantee the service quality of the online task.
Further, in the scheduling method provided in the embodiment of the present application, if there is no node belonging to the online task processing area in the at least one candidate scheduling node, it is prompted that the number of nodes in the online task processing area is insufficient. Here, when the online task processing area does not satisfy the node for processing the task to be scheduled, it indicates that the node in the online task processing area is insufficient, that is, there is a risk of insufficient resources in the sub-line task processing area, and by sending a prompt message to the operation and maintenance staff, the operation and maintenance staff can be reminded to add a physical computer to a cluster in advance, so as to guarantee service requirements.
Further, in the scheduling method provided in the embodiment of the present application, if the time of the task to be scheduled in the mixed task processing area is greater than or equal to the preset time and there is a target node satisfying the amount of resources required by the task to be scheduled in the online task processing area, the task to be scheduled is scheduled to the target node in the online task processing area. The method designs a rescheduling function, can regularly track the condition of the online tasks of the mixed task processing area, and can determine whether the current online task processing area has enough resources, if the online task processing area has enough resources, rescheduling can occur, and rescheduling the online tasks of the mixed task processing area meeting the conditions to the online task processing area for processing, so that the processing efficiency of scheduled tasks can be improved, and the capacity of the mixed task processing area for processing the online tasks can be guaranteed.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a flowchart illustrating a scheduling method provided in an embodiment of the present application;
fig. 2 is a functional block diagram of a scheduling apparatus according to an embodiment of the present application;
FIG. 3 shows a functional block diagram of selected blocks in FIG. 2;
fig. 4 shows a second functional block diagram of a scheduling apparatus according to an embodiment of the present application;
fig. 5 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and that steps without logical context may be performed in reverse order or concurrently. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
To enable those skilled in the art to use the present disclosure in conjunction with a specific application scenario "how to schedule tasks to be scheduled by adding scheduled tasks in a cluster", the following embodiments are given, and it will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and application scenarios without departing from the spirit and scope of the present disclosure.
The following method, apparatus, electronic device or computer-readable storage medium in the embodiments of the present application may be applied to any scenario that needs to be scheduled, and the embodiments of the present application do not limit specific application scenarios, and any scheme that uses the scheduling method and apparatus provided in the embodiments of the present application is within the protection scope of the present application.
It is worth noting that, before the present application is proposed, the HPA technique of K8S in the related scheme does not consider the actual resource allocation situation of the current cluster, and when the cluster resources are insufficient, a traditional scheduling node screening method, such as a load balancing algorithm and a minimum consumption node algorithm, is adopted, there is a great probability that a Pod scheduling failure waiting problem occurs, and further, the service quality of online application is affected.
Aiming at the problems, the embodiment of the application firstly obtains a task to be scheduled newly added in a cluster, if the candidate scheduling node meeting the resource quantity required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, whether a node belonging to an online task processing area exists in the candidate scheduling node is judged, and if the candidate scheduling node exists, a target scheduling node is selected from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. And for the on-line task to be preferentially scheduled to the on-line task processing area, when the resources of the on-line task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the condition that the on-line task fails to be scheduled due to the increase of the scheduling task can be avoided, the on-line task can be ensured to be processed in time, and further, the service quality of the on-line task is ensured.
It should be noted that K8S is an open source application for managing containerization on multiple hosts in a cloud platform; the Pod is a minimum deployable unit capable of creating and managing K8S calculation, is composed of one or more containers, is also a basic granularity of K8S scheduling, and can apply for a certain amount of resources, such as 2-core CPUs and 4G memories, and each Pod is a scheduling task for the application; the Deployment is a resource type of K8S, and comprises 0 to multiple copies (Pod), wherein the multiple copies (Pod) are equal in position; node is a server in the K8S cluster, also called Node; the Cluster is a K8S Cluster and consists of a control node and a plurality of computing nodes; HPA refers to a horizontal automatic capacity expansion and reduction technology, and can automatically expand or reduce the copy number of the Deployment according to the occupation condition of CPU, MEM or other self-defined indexes (such as request quantity) of Pod, for example, the CPU occupancy rate exceeds 70% to automatically expand the copy number, and the CPU occupancy rate is 30% less to automatically reduce the copy number; a Zone (area) is a set of a plurality of computing nodes, a plurality of computing nodes in a K8S cluster can be classified into a Zone, for the application, the computing nodes in the cluster are divided into an online task processing area and a mixed task processing area, and if a Pod is scheduled by a certain Zone, the Pod is only scheduled to the Node to which the Zone belongs; if the current Pod is scheduled to the node to run, the node is called a scheduling node; if all the computing node residual resources of the current cluster resource cannot meet the resource requirement of the Pod to be scheduled, the Pod enters a Pending state (waiting state), the K8S scheduling regularly detects the Pending Pod and all the current clusters meeting the Pod scheduling in the subsequent process, if the Pending Pod meets the scheduling requirement, the scheduling is carried out, and if the Pending cluster does not meet the scheduling requirement, the Pending cluster continues the scheduling; online applications, i.e., online tasks, which directly provide online services to users, need to run for a long time, such as online games, media websites, and the like; the off-line task is generally not directly providing service for the user, and is finished after running for a period of time, such as a timing task, an AI training task and a big data off-line computing task.
For the convenience of understanding of the present application, the technical solutions provided in the present application will be described in detail below with reference to specific embodiments.
Fig. 1 is a flowchart of a scheduling method according to an embodiment of the present application. As shown in fig. 1, a scheduling method provided in the embodiment of the present application includes the following steps:
s101: and acquiring a newly added task to be scheduled in the cluster.
In specific implementation, when a new task to be scheduled is detected to be added in a cluster, the new task to be scheduled is obtained, and the resource amount required by the task to be scheduled is determined.
The cluster comprises a control node and a plurality of computing nodes, wherein the control node is responsible for distributing the scheduling tasks to the computing nodes, and the computing nodes are used for processing the scheduling tasks and providing computing resources for the scheduling tasks.
It should be noted that the request amount for the online application (online task) is generally not constant, such as the request amount is higher in the daytime and lower in the evening, or the request amount is suddenly increased when an activity day is encountered, such as twenty-one. When the amount of requests for online applications is suddenly increased, HPA techniques may be employed to automatically increase the scheduling task (online task) to process the request according to the amount of requests.
Preferably, a main body executing the scheduling method provided by the present application may be a control node in a cluster.
S102: if it is determined that at least one candidate scheduling node with the residual resource amount meeting the resource amount required by the task to be scheduled exists in the computing nodes of the cluster, and the task to be scheduled is an online task, whether a node belonging to an online task processing area exists in the at least one candidate scheduling node is judged.
In the specific implementation, for any newly-added task to be scheduled, the resource amount required by the task to be scheduled is determined, and then whether at least one candidate scheduling node with the residual resource amount meeting the resource amount required by the task to be scheduled exists in the computing nodes of the cluster is judged, that is, the computing nodes in the cluster are pre-selected, the candidate scheduling node meeting the resource required by the task to be scheduled is obtained, if the candidate scheduling node exists, the task type of the task to be scheduled is judged, and if the task to be scheduled is an online task, whether a node belonging to an online task processing area exists in the candidate scheduling nodes meeting the resource required by the task to be scheduled is judged.
It should be noted that, in the present application, the computing nodes in the cluster are divided into different task processing areas in advance, where there are two task processing areas, that is, an online task processing area and a mixed task processing area, where the computing nodes in the online task processing area are only responsible for processing online tasks, and the computing nodes in the mixed task processing area may be responsible for processing online tasks and offline tasks, and considering that online tasks need to be processed in time and offline tasks may be processed later, for this reason, the priority of online tasks is set higher than the priority of offline tasks. In general, the amount of resources of the online task processing area is greater than the amount of resources of the mixed task processing area, so that the online task can be sufficiently processed, and preferably, the resource occupation ratio of the online task processing area can be set to be higher than 80%.
Here, one compute node can process multiple scheduled tasks at the same time, and the scheduled tasks can be distributed to only one compute node for processing.
In an example, a cluster has three computing nodes, which are respectively a computing node a, a computing node B, and a computing node C, the total CPU amount of the computing node a is 20 cores, the current CPU remaining amount is 10 cores, the total CPU amount of the computing node B is 20 cores, the current CPU remaining amount is 5 cores, the total CPU amount of the computing node C is 20 cores, and the current CPU remaining amount is 3 cores. The resource amount required by the task to be scheduled is not the CPU6 core, the residual CPU 5 core of the computing node B and the residual CPU3 core of the computing node C meet the resource amount CPU6 core required by the task to be scheduled, and the computing node A is selected from the computing node A, the computing node B and the computing node C as a candidate scheduling node only if the residual CPU10 core of the computing node A meets the resource amount CPU6 core required by the task to be scheduled.
Here, after determining whether there is a node belonging to the online task processing area in the at least one candidate scheduling node in step S102, it is determined that there is no node belonging to the online task processing area in the at least one candidate scheduling node, which indicates that the online task processing area cannot process the task to be scheduled as the online task, and resources of the online task processing area are insufficient, at this time, it is necessary to remind the operation and maintenance staff to add a physical computer, that is, to prompt that the number of nodes in the online task processing area is insufficient, so that the operation and maintenance staff increase the computing nodes in the online task processing area, and avoid a situation that the request cannot be processed in time due to insufficient resources of the online task processing area.
Further, the task to be scheduled may be an online task or an offline task, and if it is determined that at least one candidate scheduling node exists in the computing nodes of the cluster, where the remaining resource amount of the candidate scheduling node satisfies the resource amount required by the task to be scheduled, and the task to be scheduled is an offline task, the method further includes the following steps:
calculating a second resource occupancy rate of an offline task containing the task to be scheduled in the mixed task processing area; judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion or not; if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node; if not, refusing to process the task to be scheduled.
In a specific implementation, if it is determined that the task to be scheduled is an offline task, since the online task processing area can only process the online task, the mixed task processing area can process the online task, and can also process the offline task, the task to be scheduled can only be scheduled to the mixed task processing area, but it is now required to determine whether a node belonging to the mixed task processing area exists in the candidate scheduling nodes satisfying the amount of resources required by the task to be scheduled, and it is also required to determine whether the amount of resources required by the offline task in the mixed task processing area has reached a threshold, and if the threshold is reached, since the priority of the online task is higher than that of the offline task, the task to be scheduled cannot be processed regardless of whether there is a computing node satisfying the requirements in the mixed task processing area, so that the mixed task processing area has room to process the online task, specifically, the second resource occupancy rate of the offline task to be scheduled in the mixed task processing area is calculated, and if the second resource occupancy rate is smaller than or equal to the second preset proportion, selecting a target scheduling node from the nodes belonging to the mixed task processing area in the candidate scheduling nodes, if the second resource occupancy rate is larger than the second preset proportion, refusing to process the task to be scheduled, waiting for other offline tasks in the mixed task processing area to be processed, and processing the task to be scheduled when the second resource occupancy rate is smaller than the second preset proportion.
Here, the second preset ratio may be set according to the actual resource amount of the mixed task processing area and the resource amount required by the offline task, and is preferably set to be greater than 50%, where the sum of the first preset ratio and the second preset ratio is 1.
Further, if the second resource occupancy is less than or equal to a second preset proportion and there is no node belonging to the mixed task processing area in the at least one candidate scheduling node, the method further includes the following steps: and refusing to process the task to be scheduled.
In a specific implementation, because the online task processing area can only process online tasks, the mixed task processing area can process online tasks and can also process offline tasks, if the second resource occupancy rate is less than or equal to a second preset proportion, but nodes which belong to the mixed task processing area and meet resources required by the tasks to be scheduled do not exist in the candidate scheduling nodes, at this time, the tasks to be scheduled can be enabled to be in a waiting state, the tasks to be scheduled are currently refused to be processed, and when nodes which belong to the mixed task processing area and meet the resources required by the tasks to be scheduled exist, the tasks to be scheduled are processed.
Here, there is another case that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, which indicates that the resource of the cluster is insufficient, and if the task to be scheduled is an online task, since the priority of the online task is higher than that of the offline task, at this time, preemptive scheduling is started, the scheduling task with the lowest priority in the mixed task processing area, which is the offline task, may be suspended or deleted, and the candidate scheduling node is screened for the task to be scheduled again, so that the task to be scheduled may be processed, specifically, the offline task processed by the nodes in the mixed task processing area may be deleted, so that the nodes in the mixed task processing area process the task to be scheduled. In addition, in the computing nodes of the cluster, there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled, which indicates that the resource of the cluster is insufficient.
Here, after a task to be scheduled, which is an online task, is scheduled to a mixed task processing area, there may be a case that the task to be scheduled is not processed in time, and during this time, there may be a node satisfying the processing of the task to be scheduled in the online task processing area, and the task to be scheduled may be rescheduled to a target node satisfying a resource requirement in the online task processing area, that is, if the time of the task to be scheduled in the mixed task processing area is greater than or equal to a preset time length and there is a target node satisfying the amount of resources required by the task to be scheduled in the online task processing area, the task to be scheduled is scheduled to the target node in the online task processing area.
It should be noted that, the capacity expansion sequence of Pod copies (scheduling tasks) in the K8s cluster is strongly related to the capacity reduction sequence, for example, if there is one original copy P1, there are P1 and P2 after capacity expansion to two copies, and then capacity reduction to one copy is performed, then the remaining P1 expands the capacity according to Pod in the K8s cluster, and the reverse reason of capacity reduction is that after a series of capacity expansion/capacity reduction scheduling, there may be a case that resources in an online task processing area are sufficient, but scheduling tasks of the online tasks still remain in the mixed task processing area for a long time, and if this occurs in a large number, the mixed task processing area will accumulate a large number of Pod of online tasks, thereby causing the mixed task processing area to lose its guarantee effect. Therefore, the rescheduling function is designed, the condition of the online tasks in the online task processing area can be tracked at regular time, whether the current online task processing area has enough resources or not is judged, if the online task processing area has enough resources, rescheduling is carried out, and the online tasks in the mixed task processing area meeting the conditions are rescheduled to the online task processing area.
S103: if yes, selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node.
In the specific implementation, in the computing nodes of the cluster, candidate scheduling nodes meeting the resource quantity required by the task to be scheduled are determined, and the task to be scheduled is determined to be an online task, further, whether nodes belonging to an online task processing area exist in the candidate scheduling nodes meeting the resource quantity required by the task to be scheduled is judged, if yes, the task to be scheduled is preferably scheduled to the nodes in the online task processing area, and if not, the nodes meeting the resource quantity required by the task to be scheduled do not exist in the nodes in the online task processing area, and at this time, a target scheduling node can be selected from the nodes belonging to a mixed task processing area in the candidate scheduling nodes meeting the resource quantity required by the task to be scheduled to process the task to be scheduled.
Here, the selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node in step S102 includes:
counting the number of nodes belonging to the online task processing area in the at least one candidate scheduling node; if at least two nodes exist, selecting the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm; and if one node exists, taking the node as the target scheduling node.
In this step, after determining that the task to be scheduled is an online task, preferably scheduling the task to be scheduled to a node in an online task processing area, where it is first determined whether a node belonging to the online task processing area exists in the at least one candidate scheduling node satisfying resources required by the task to be scheduled, and if so, selecting a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node, where the number of nodes belonging to the online task processing area in the at least one candidate scheduling node may be counted first, and if there is only one node satisfying the above requirement, the node is directly determined as the target scheduling node; if more than two nodes meet the requirements, the target scheduling node can be screened out from at least two nodes meeting the requirements according to a preset scheduling node screening algorithm.
It should be noted that the preset scheduling node screening algorithm is a conventional scheduling node screening method, such as a load balancing algorithm (balanced _ resource _ allocation), a least consumed node algorithm (least _ requested), a node preference _ available _ points algorithm (node _ preference _ points), a selector _ spreading algorithm (selector _ spreading), and the like, where the load balancing algorithm is to select a node with the most balanced resource usage rate from a node list, and the least consumed node algorithm is to select a node with the least consumed (CPU (total capacity-sum (used): 10/total capacity)) according to an idle ratio).
It should be noted that, when the request amount for online application is suddenly increased, the HPA technique may be used to automatically increase the scheduling task for processing the request according to the amount of the request amount, so as to avoid the situation that the newly increased scheduling task cannot be processed in time due to the increase of the scheduling task, a reasonable scheduling scheme needs to be provided, and the newly increased scheduling task is processed without changing the number of the computing nodes in the cluster as much as possible. Therefore, the cluster nodes are divided into two task processing areas, one is an online task processing area, the other is a mixed task processing area, the online task processing area is only responsible for processing online tasks, the mixed task processing area only processes offline tasks under the condition that resources of the online task processing area are sufficient, and the online tasks can be processed except the offline tasks under the condition that the resources of the online task processing area are insufficient.
Further, after the target scheduling node is selected from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node in step S103, the method further includes the following steps: calculating a first resource occupancy rate of an online task in the mixed task processing area; judging whether the first resource occupancy rate is greater than or equal to a first preset proportion or not; and if so, prompting that the number of the nodes in the mixed task processing area is insufficient.
In the specific implementation, if the task to be scheduled is an online task and there is no node belonging to the online task processing area in the candidate scheduling nodes, it is indicated that there is no node satisfying the resource amount required by the task to be scheduled in the nodes of the online task processing area, and at this time, a target scheduling node may be selected from the nodes belonging to the mixed task processing area in the candidate scheduling nodes satisfying the resource amount required by the task to be scheduled to process the task to be scheduled. After a task to be scheduled, which is an online task, is scheduled to a target scheduling node in a mixed task processing area, calculating the proportion of resources corresponding to the online task in the mixed task processing area to total resources in the mixed task processing area, namely, a first resource occupancy rate, further judging whether the first resource occupancy rate is greater than or equal to a first preset proportion, if so, indicating that more online tasks are scheduled to the mixed task processing area, and indicating that the resources in the mixed task processing area are insufficient because the mixed task processing area is mainly used for processing offline tasks, at this time, indicating that the resources in the mixed task processing area are insufficient, prompting an operation and maintenance worker that the number of nodes in the mixed task processing area is insufficient is needed, so that the operation and maintenance worker can add an entity computer in time to increase the number of the computing nodes in the mixed task processing area. According to the method, the operation and maintenance personnel are prompted that the number of the nodes in the mixed task processing area is insufficient when the first resource occupancy rate of the online task in the mixed task processing area is larger than or equal to the first preset proportion, at the moment, the mixed task processing area can process the scheduling task through resources, therefore, the time for increasing the computer equipment is reserved for the operation and maintenance personnel, the computing nodes are additionally arranged before the resources in the mixed task processing area are completely used, and the situation that the request cannot be processed in time due to the fact that the resources in the offline task processing area are insufficient is avoided.
Here, the first preset ratio may be set according to an actual amount of resources of the mixed task processing area and an amount of resources required for the offline task, and the first preset ratio is preferably set to be less than 50%.
It should be noted that, when the resources of the online task processing area are insufficient or the resources of the mixed task processing area reach a certain online state, the method and the system can trigger an alarm to remind cluster operation and maintenance personnel to increase server nodes in advance, so that the cluster can process the request from the user in time.
Further, in step S103, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node, includes the following steps:
calculating a resource surplus rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node; and selecting the node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In the specific implementation, for a task to be scheduled which is an offline task, a node which satisfies the task to be scheduled and has a small amount of residual resources of resources required by the task to be scheduled is selected as much as possible and is used as a target scheduling node to process the task to be scheduled. In this way, the mixed task processing area can be made available to compute nodes to process online tasks.
Here, if a node only includes cpu and memory resources, the calculation formula of the resource remaining rate of each node is as follows:
Score＝(UsedCpu/cpuNodeCap acity+UsedMemory/MemoryNode Capacity)/2
wherein, the cpnodecacity is the total amount of the cpu of the node, and the usedCpu is the applied cpu amount; memory NodeCapacity is the total amount of the node memory, and used memory is the amount of the memory which has been applied.
S104: and scheduling the task to be scheduled to the target scheduling node.
In specific implementation, after a target scheduling node for processing a task to be scheduled is determined, the task to be scheduled is scheduled to the target scheduling node, that is, the task to be scheduled is allocated to the target scheduling node.
It should be noted that, in the present application, by partitioning a cluster in advance (an online task processing area and a mixed task processing area), defining a scheduling task priority (an online task priority is high, an offline task priority is low), and matching with a customized target scheduling node screening method, the present application can effectively avoid the problem of HPA capacity expansion failure (scheduling task increase), and simultaneously, by the mixed portion of the online task processing area and the mixed task processing area, the utilization of resources in the mixed task processing area is improved, and cluster nodes can be supplemented in time through an alarm mechanism.
In the embodiment of the application, a newly-added task to be scheduled in a cluster is obtained, if it is determined that candidate scheduling nodes meeting the resource quantity required by the task to be scheduled exist in computing nodes of the cluster and the task to be scheduled is an online task, whether nodes belonging to an online task processing area exist in the candidate scheduling nodes is judged, and if the candidate scheduling nodes exist, a target scheduling node is selected from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. And for the on-line task to be preferentially scheduled to the on-line task processing area, when the resources of the on-line task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the condition that the on-line task fails to be scheduled due to the increase of the scheduling task can be avoided, the on-line task can be ensured to be processed in time, and further, the service quality of the on-line task is ensured.
Based on the same application concept, a scheduling apparatus corresponding to the scheduling method provided in the foregoing embodiment is also provided in this embodiment of the present application, and since the principle of the apparatus in this embodiment to solve the problem is similar to the scheduling method in the foregoing embodiment of the present application, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not described again.
Fig. 2 is a functional block diagram of a scheduling apparatus 200 according to an embodiment of the present application; FIG. 3 is a functional block diagram of the selection block 230 of FIG. 2; fig. 4 shows a second functional block diagram of a scheduling apparatus 200 according to an embodiment of the present application.
As shown in fig. 2, a scheduling apparatus 200, the scheduling apparatus 200 includes:
an obtaining module 210, configured to obtain a task to be scheduled newly added in a cluster;
a determining module 220, configured to determine whether a node belonging to an online task processing area exists in at least one candidate scheduling node if it is determined that at least one candidate scheduling node exists in the computing nodes of the cluster, where a remaining resource amount of the candidate scheduling node satisfies a resource amount required by the task to be scheduled, and the task to be scheduled is an online task;
a selecting module 230, configured to select a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node, if the target scheduling node exists; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node;
and the scheduling module 240 is configured to schedule the task to be scheduled to the target scheduling node.
In one possible implementation, as shown in fig. 3, the selecting module 230 includes:
a counting unit 231, configured to count the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
a selecting unit 232, configured to select the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm if there are at least two nodes;
a determining unit 233, configured to, if there is a node, use the node as the target scheduling node.
In a possible implementation, as shown in fig. 4, the scheduling apparatus 200 further includes:
and a first prompting module 250, configured to prompt that the number of nodes in the online task processing area is insufficient.
In a possible embodiment, as shown in fig. 4, the scheduling apparatus 200 further includes a second prompting module 260; the second prompting module 260 is configured to:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is greater than or equal to a first preset proportion or not;
and if so, prompting that the number of the nodes in the mixed task processing area is insufficient.
In a possible implementation manner, as shown in fig. 4, if it is determined that there is at least one candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an offline task, the scheduling apparatus 200 further includes a determining module 270; the determining module 270 is configured to:
calculating a second resource occupancy rate of an offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion or not;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
In a possible implementation manner, as shown in fig. 4, if the second resource occupancy is less than or equal to a second preset ratio, and there is no node belonging to the mixed task processing area in the at least one candidate scheduling node, the determining module 270 is further configured to:
and refusing to process the task to be scheduled.
In one possible implementation, as shown in fig. 4, the determining module 270 is configured to determine the target scheduling node according to the following steps:
calculating a resource surplus rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting the node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
In a possible implementation manner, as shown in fig. 4, if it is determined that there is no candidate scheduling node whose remaining resource amount satisfies the resource amount required by the task to be scheduled in the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling apparatus 200 further includes a deleting module 280; the deletion module 280 is configured to:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
In one possible implementation, as shown in fig. 4, the scheduling module 240 is further configured to:
and if the time of the task to be scheduled in the mixed task processing area is longer than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, scheduling the task to be scheduled to the target node in the online task processing area.
Based on the same application concept, referring to fig. 5, a schematic structural diagram of an electronic device 500 provided in the embodiment of the present application includes: a processor 510, a memory 520 and a bus 530, the memory 520 storing machine-readable instructions executable by the processor 510, the processor 510 and the memory 520 communicating via the bus 530 when the electronic device 500 is operating, the machine-readable instructions when executed by the processor 510 performing the steps of the scheduling method according to any of the embodiments described above.
In particular, the machine readable instructions, when executed by the processor 510, may perform the following:
acquiring a newly added task to be scheduled in a cluster;
if it is determined that at least one candidate scheduling node with the residual resource amount meeting the resource amount required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node;
if yes, selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node;
and scheduling the task to be scheduled to the target scheduling node.
In the embodiment of the application, a newly-added task to be scheduled in a cluster is obtained, if it is determined that candidate scheduling nodes meeting the resource quantity required by the task to be scheduled exist in computing nodes of the cluster and the task to be scheduled is an online task, whether nodes belonging to an online task processing area exist in the candidate scheduling nodes is judged, and if the candidate scheduling nodes exist, a target scheduling node is selected from the nodes in the online task processing area; otherwise, selecting a target scheduling node from the nodes in the mixed task processing area. And for the on-line task to be preferentially scheduled to the on-line task processing area, when the resources of the on-line task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that the condition that the on-line task fails to be scheduled due to the increase of the scheduling task can be avoided, the on-line task can be ensured to be processed in time, and further, the service quality of the on-line task is ensured.
Based on the same application concept, embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the scheduling method provided in the foregoing embodiments are executed.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, when a computer program on the storage medium is run, the scheduling method can be executed, the online task is preferentially scheduled to the online task processing area, and when resources of the online task processing area are insufficient, the task to be scheduled can be scheduled to the mixed task processing area, so that a situation that the online task fails to be scheduled due to the increase of scheduling tasks can be avoided, the online task can be ensured to be processed in time, and further, the service quality of the online task is ensured.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. A scheduling method, characterized in that the scheduling method comprises:
acquiring a newly added task to be scheduled in a cluster;
if it is determined that at least one candidate scheduling node with the residual resource amount meeting the resource amount required by the task to be scheduled exists in the computing nodes of the cluster and the task to be scheduled is an online task, judging whether a node belonging to an online task processing area exists in the at least one candidate scheduling node;
if yes, selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node;
and scheduling the task to be scheduled to the target scheduling node.
2. The scheduling method according to claim 1, wherein the selecting a target scheduling node from the nodes belonging to the online task processing area in the at least one candidate scheduling node comprises:
counting the number of nodes belonging to the online task processing area in the at least one candidate scheduling node;
if at least two nodes exist, selecting the target scheduling node from the at least two nodes according to a preset scheduling node screening algorithm;
and if one node exists, taking the node as the target scheduling node.
3. The method according to claim 1, wherein if there is no node belonging to the online task processing area in the at least one candidate scheduling node, the method further comprises:
and prompting that the number of the nodes in the online task processing area is insufficient.
4. The scheduling method according to claim 1, wherein after the selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, the scheduling method further comprises:
calculating a first resource occupancy rate of an online task in the mixed task processing area;
judging whether the first resource occupancy rate is greater than or equal to a first preset proportion or not;
and if so, prompting that the number of the nodes in the mixed task processing area is insufficient.
5. The scheduling method according to claim 1, wherein if it is determined that there is at least one candidate scheduling node having a remaining resource amount that satisfies a resource amount required by the task to be scheduled among the computing nodes of the cluster, and the task to be scheduled is an offline task, the scheduling method further comprises:
calculating a second resource occupancy rate of an offline task containing the task to be scheduled in the mixed task processing area;
judging whether the second resource occupancy rate is smaller than or equal to a second preset proportion or not;
if yes, determining the target scheduling node based on a node belonging to the mixed task processing area in the at least one candidate scheduling node;
if not, refusing to process the task to be scheduled.
6. The scheduling method of claim 5, wherein if the second resource occupancy is less than or equal to a second predetermined ratio and there is no node belonging to the hybrid task processing area in the at least one candidate scheduling node, the scheduling method further comprises:
and refusing to process the task to be scheduled.
7. The scheduling method according to claim 5, wherein the determining the target scheduling node based on the node belonging to the mixed task processing area in the at least one candidate scheduling node comprises:
calculating a resource surplus rate corresponding to each node belonging to the mixed task processing area in the at least one candidate scheduling node;
and selecting the node with the minimum resource residual rate from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node, and determining the node as the target scheduling node.
8. The scheduling method according to claim 1, wherein if it is determined that there is no candidate scheduling node having a remaining resource amount that satisfies a resource amount required by the task to be scheduled among the computing nodes of the cluster, and the task to be scheduled is an online task, the scheduling method further comprises:
and deleting the offline tasks processed by the nodes in the mixed task processing area so that the nodes in the mixed task processing area process the tasks to be scheduled.
9. The scheduling method of claim 8, further comprising:
and if the time of the task to be scheduled in the mixed task processing area is longer than or equal to the preset time length and a target node meeting the resource quantity required by the task to be scheduled exists in the online task processing area, scheduling the task to be scheduled to the target node in the online task processing area.
10. A scheduling apparatus, comprising:
the acquisition module is used for acquiring a newly added task to be scheduled in the cluster;
the judging module is used for judging whether a node belonging to an online task processing area exists in at least one candidate scheduling node if the computing nodes of the cluster are determined that the residual resource amount meets the resource amount required by the task to be scheduled and the task to be scheduled is an online task;
a selecting module, configured to select a target scheduling node from nodes belonging to the online task processing area in the at least one candidate scheduling node, if the target scheduling node exists; otherwise, selecting the target scheduling node from the nodes belonging to the mixed task processing area in the at least one candidate scheduling node;
and the scheduling module is used for scheduling the task to be scheduled to the target scheduling node.
11. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the scheduling method of any of claims 1 to 9.
12. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the scheduling method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299381.XA CN112269641B (en) | 2020-11-18 | 2020-11-18 | Scheduling method, scheduling device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299381.XA CN112269641B (en) | 2020-11-18 | 2020-11-18 | Scheduling method, scheduling device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112269641A true CN112269641A (en) | 2021-01-26 |
| CN112269641B CN112269641B (en) | 2023-09-15 |
Family
ID=74340705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011299381.XA Active CN112269641B (en) | 2020-11-18 | 2020-11-18 | Scheduling method, scheduling device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269641B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113382077A (en) * | 2021-06-18 | 2021-09-10 | 广西电网有限责任公司 | Micro-service scheduling method and device, computer equipment and storage medium |
| CN113590317A (en) * | 2021-07-27 | 2021-11-02 | 杭州朗和科技有限公司 | Scheduling method, device, medium and computing equipment of offline service |
| CN113760549A (en) * | 2021-08-30 | 2021-12-07 | 聚好看科技股份有限公司 | Pod deployment method and device |
| CN114401263A (en) * | 2022-01-21 | 2022-04-26 | 中兴飞流信息科技有限公司 | Image ETL access elastic processing analysis method and system based on microservice implementation |
| CN114629960A (en) * | 2022-03-14 | 2022-06-14 | 北京字节跳动网络技术有限公司 | Resource scheduling method, device, system, device, medium, and program product |
| CN114979282A (en) * | 2022-07-28 | 2022-08-30 | 北京金山云网络技术有限公司 | Task scheduling method and device, storage medium and electronic equipment |
| CN115454680A (en) * | 2022-10-12 | 2022-12-09 | 中航信移动科技有限公司 | Application control system |
| WO2023207623A1 (en) * | 2022-04-24 | 2023-11-02 | 阿里巴巴（中国）有限公司 | Task processing method |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105843670A (en) * | 2016-03-22 | 2016-08-10 | 浙江大学 | Cloud platform based virtual cluster deployment and integration method |
| US20170176959A1 (en) * | 2015-12-22 | 2017-06-22 | Invensys Systems, Inc. | Process optimization by grouping mixed integer nonlinear programming constraints |
| CN107621973A (en) * | 2016-07-13 | 2018-01-23 | 阿里巴巴集团控股有限公司 | A kind of method for scheduling task and device across cluster |
| CN107968810A (en) * | 2016-10-20 | 2018-04-27 | 阿里巴巴集团控股有限公司 | A kind of resource regulating method of server cluster, device and system |
| CN109597688A (en) * | 2018-11-23 | 2019-04-09 | 金色熊猫有限公司 | Online resource management method, device, storage medium and electronic equipment |
| CN110647394A (en) * | 2018-06-27 | 2020-01-03 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and equipment |
| CN110830391A (en) * | 2018-08-10 | 2020-02-21 | 阿里巴巴集团控股有限公司 | Resource allocation method and device and cluster system |
| CN111026553A (en) * | 2019-12-13 | 2020-04-17 | 深圳先进技术研究院 | Resource scheduling method for offline mixed part operation and server system |
| CN111399989A (en) * | 2020-04-10 | 2020-07-10 | 中国人民解放军国防科技大学 | Task preemption scheduling method and system for container cloud |
| CN111459678A (en) * | 2020-04-02 | 2020-07-28 | 上海极链网络科技有限公司 | Resource scheduling method and device, storage medium and electronic equipment |
| CN111506423A (en) * | 2020-04-09 | 2020-08-07 | 网易（杭州）网络有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111722928A (en) * | 2020-06-12 | 2020-09-29 | 北京字节跳动网络技术有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111736957A (en) * | 2020-06-29 | 2020-10-02 | 平安普惠企业管理有限公司 | Multi-type service mixed deployment method, device, equipment and storage medium |
-
2020
- 2020-11-18 CN CN202011299381.XA patent/CN112269641B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170176959A1 (en) * | 2015-12-22 | 2017-06-22 | Invensys Systems, Inc. | Process optimization by grouping mixed integer nonlinear programming constraints |
| CN105843670A (en) * | 2016-03-22 | 2016-08-10 | 浙江大学 | Cloud platform based virtual cluster deployment and integration method |
| CN107621973A (en) * | 2016-07-13 | 2018-01-23 | 阿里巴巴集团控股有限公司 | A kind of method for scheduling task and device across cluster |
| CN107968810A (en) * | 2016-10-20 | 2018-04-27 | 阿里巴巴集团控股有限公司 | A kind of resource regulating method of server cluster, device and system |
| CN110647394A (en) * | 2018-06-27 | 2020-01-03 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and equipment |
| CN110830391A (en) * | 2018-08-10 | 2020-02-21 | 阿里巴巴集团控股有限公司 | Resource allocation method and device and cluster system |
| CN109597688A (en) * | 2018-11-23 | 2019-04-09 | 金色熊猫有限公司 | Online resource management method, device, storage medium and electronic equipment |
| CN111026553A (en) * | 2019-12-13 | 2020-04-17 | 深圳先进技术研究院 | Resource scheduling method for offline mixed part operation and server system |
| CN111459678A (en) * | 2020-04-02 | 2020-07-28 | 上海极链网络科技有限公司 | Resource scheduling method and device, storage medium and electronic equipment |
| CN111506423A (en) * | 2020-04-09 | 2020-08-07 | 网易（杭州）网络有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111399989A (en) * | 2020-04-10 | 2020-07-10 | 中国人民解放军国防科技大学 | Task preemption scheduling method and system for container cloud |
| CN111722928A (en) * | 2020-06-12 | 2020-09-29 | 北京字节跳动网络技术有限公司 | Resource scheduling method and device, electronic equipment and storage medium |
| CN111736957A (en) * | 2020-06-29 | 2020-10-02 | 平安普惠企业管理有限公司 | Multi-type service mixed deployment method, device, equipment and storage medium |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113382077A (en) * | 2021-06-18 | 2021-09-10 | 广西电网有限责任公司 | Micro-service scheduling method and device, computer equipment and storage medium |
| CN113590317A (en) * | 2021-07-27 | 2021-11-02 | 杭州朗和科技有限公司 | Scheduling method, device, medium and computing equipment of offline service |
| CN113760549A (en) * | 2021-08-30 | 2021-12-07 | 聚好看科技股份有限公司 | Pod deployment method and device |
| CN113760549B (en) * | 2021-08-30 | 2024-03-15 | 聚好看科技股份有限公司 | Pod deployment method and device |
| CN114401263A (en) * | 2022-01-21 | 2022-04-26 | 中兴飞流信息科技有限公司 | Image ETL access elastic processing analysis method and system based on microservice implementation |
| CN114629960A (en) * | 2022-03-14 | 2022-06-14 | 北京字节跳动网络技术有限公司 | Resource scheduling method, device, system, device, medium, and program product |
| CN114629960B (en) * | 2022-03-14 | 2023-09-19 | 抖音视界有限公司 | Resource scheduling method, device, system, equipment, medium and program product |
| WO2023207623A1 (en) * | 2022-04-24 | 2023-11-02 | 阿里巴巴（中国）有限公司 | Task processing method |
| CN114979282A (en) * | 2022-07-28 | 2022-08-30 | 北京金山云网络技术有限公司 | Task scheduling method and device, storage medium and electronic equipment |
| CN115454680A (en) * | 2022-10-12 | 2022-12-09 | 中航信移动科技有限公司 | Application control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112269641B (en) | 2023-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112269641A (en) | Scheduling method, scheduling device, electronic equipment and storage medium | |
| US20200287961A1 (en) | Balancing resources in distributed computing environments | |
| CN110858161B (en) | Resource allocation method, device, system, equipment and medium | |
| CN109936604B (en) | Resource scheduling method, device and system | |
| CN111966500A (en) | Resource scheduling method and device, electronic equipment and storage medium | |
| EP2725862A1 (en) | Resource allocation method and resource management platform | |
| US11496413B2 (en) | Allocating cloud computing resources in a cloud computing environment based on user predictability | |
| CN114356543A (en) | Kubernetes-based multi-tenant machine learning task resource scheduling method | |
| CN112988390A (en) | Calculation power resource allocation method and device | |
| CN113867959A (en) | Training task resource scheduling method, device, equipment and medium | |
| CN111798113A (en) | Resource allocation method, device, storage medium and electronic equipment | |
| US20210357269A1 (en) | Quality of service scheduling with workload profiles | |
| CN112860383A (en) | Cluster resource scheduling method, device, equipment and storage medium | |
| CN114968601B (en) | Scheduling method and scheduling system for AI training jobs with resources reserved in proportion | |
| CN113032102A (en) | Resource rescheduling method, device, equipment and medium | |
| WO2014136302A1 (en) | Task management device and task management method | |
| CN115658311A (en) | Resource scheduling method, device, equipment and medium | |
| CN107203256B (en) | Energy-saving distribution method and device under network function virtualization scene | |
| CN112214288B (en) | Pod scheduling method, device, equipment and medium based on Kubernetes cluster | |
| CN111796933A (en) | Resource scheduling method, device, storage medium and electronic equipment | |
| CN108694083B (en) | Data processing method and device for server | |
| CN113010309B (en) | Cluster resource scheduling method, device, storage medium, equipment and program product | |
| CN112073532B (en) | Resource allocation method and device | |
| CN114116173A (en) | Method, device and system for dynamically adjusting task allocation | |
| CN106126319B (en) | Central processing unit resource allocation methods and calculate node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |