CN114253688A - Method and application for rescheduling application load in cloud environment - Google Patents

Method and application for rescheduling application load in cloud environment Download PDF

Info

Publication number
CN114253688A
CN114253688A CN202111553945.2A CN202111553945A CN114253688A CN 114253688 A CN114253688 A CN 114253688A CN 202111553945 A CN202111553945 A CN 202111553945A CN 114253688 A CN114253688 A CN 114253688A
Authority
CN
China
Prior art keywords
cluster
application
rescheduling
resource
scheduling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111553945.2A
Other languages
Chinese (zh)
Inventor
胡正川
黄茂峰
仇大玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Anchaoyun Software Co ltd
Original Assignee
Shanghai Anchaoyun Software Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Anchaoyun Software Co ltd filed Critical Shanghai Anchaoyun Software Co ltd
Priority to CN202111553945.2A priority Critical patent/CN114253688A/en
Publication of CN114253688A publication Critical patent/CN114253688A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a method for rescheduling application load in a cloud environment and application thereof, wherein the method comprises the following steps: acquiring historical operating data of an application load on a cluster node to generate an application portrait; converting the application representation into a scheduling rule, wherein the scheduling rule comprises a resource utilization rate and an application type; calling the application type in the scheduling rule to acquire the resource distribution condition of the cloud platform; calculating the imbalance of cluster resources according to the resource distribution condition of the cloud platform; and rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource utilization rate in the scheduling rule and the first scheduling constraint. The method can improve the utilization rate of the cloud platform resources, and effectively solves the problem that the resources cannot be fully used due to resource fragmentation caused by scheduling in the cloud environment.

Description

Method and application for rescheduling application load in cloud environment
Technical Field
The invention relates to the field of cloud computing, in particular to a method for rescheduling application load in a cloud environment and application thereof.
Background
More than all, the resource scheduling system of the cloud platform is primarily scheduled, that is, a certain scheduling strategy is adopted to allocate a request for resources to a certain node in the cloud platform according to the distribution condition of current resources, the primary scheduling is based on the distribution condition of the current resources, the condition of future resource load is not considered, and the problem of serious fragmentation of the resources exists after the platform operates for a period of time, so that the resources cannot be fully utilized.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
Disclosure of Invention
The invention aims to provide a method for rescheduling application load in a cloud environment and application thereof, and solves the problem that resources cannot be fully used due to fragmentation of the resources caused by scheduling in the cloud environment.
To achieve the above object, an embodiment of the present invention provides a method for rescheduling an application load in a cloud environment.
In one or more embodiments of the invention, the method comprises: acquiring historical operating data of an application load on a cluster node to generate an application portrait; converting the application representation into a scheduling rule, wherein the scheduling rule comprises a resource utilization rate and an application type; calling the application type in the scheduling rule to acquire the resource distribution condition of the cloud platform; calculating the imbalance of cluster resources according to the resource distribution condition of the cloud platform; and rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource utilization rate in the scheduling rule and the first scheduling constraint.
In one or more embodiments of the present invention, acquiring historical operating data of an application load on a cluster node to generate an application representation includes: preprocessing the historical operating data and storing the preprocessed historical operating data in a time sequence database; and performing histogram calculation on corresponding data of the running load at each moment in the time sequence database to obtain the resource usage of the application load.
In one or more embodiments of the present invention, the cloud platform resource n includes at least one of a CPU, a memory, a disk, and a network; calculating a cluster according to the resource distribution condition of the cloud platformThe resource imbalance specifically includes: determining the resource allocation rate eve of each node in the cluster according to the resource distribution condition of the cloud platform(n,k)And the average resource allocation rate avg of the clustern(ii) a And calculating the imbalance degree of the cluster resources according to the deviation of the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster.
In one or more embodiments of the invention, according to the resource distribution situation of the cloud platform, the resource allocation rate eve of each node in the cluster is determined(n,k)And the average resource allocation rate avg of the clusternThe method comprises the following steps:
Figure BDA0003417965830000021
and k is the number of the nodes in the cluster, app is the actual usage amount of the same cloud platform resource of the nodes in the cluster, and node is the actual capacity of the same cloud platform resource of the nodes in the cluster.
In one or more embodiments of the present invention, calculating an imbalance degree of cluster resources according to a deviation between a resource allocation rate of each node in the cluster and an average resource allocation rate of the cluster includes: calculating the deviation between the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster:
Figure BDA0003417965830000022
calculating the imbalance degree C of the cluster resources:
C＝∑wn。
in one or more embodiments of the present invention, rescheduling an application load on a node in a cluster according to an imbalance of the cluster resource, a resource usage rate in a scheduling rule, and a first scheduling constraint specifically includes: judging whether the residual capacity of the cluster resources is larger than a preset capacity value or not; if so, selecting the node with the deviation exceeding a preset deviation value, and calculating the application load distribution condition after rescheduling; if not, calculating the application load distribution condition of the nodes in the cluster after global rescheduling.
In one or more embodiments of the present invention, rescheduling an application load on a node in a cluster according to an imbalance of the cluster resource, a resource usage rate in a scheduling rule, and a first scheduling constraint further includes: arranging the sequence of application load scheduling according to the application load scheduling condition, and generating a node scheduling rule of the designated node; and injecting the scheduling rule into a primary scheduler and rescheduling the application load by combining the node scheduling rule.
In another aspect of the present invention, an apparatus for rescheduling an application load in a cloud environment is provided, which includes an obtaining module, a transforming module, a calling module, a calculating module, and a rescheduling module.
The acquisition module is used for acquiring historical operating data of an application load on the cluster node to generate an application portrait;
the conversion module is used for converting the application portrait into a scheduling rule, and the scheduling rule comprises resource utilization rate and application type;
the calling module is used for calling the application type in the scheduling rule to acquire the resource distribution condition of the cloud platform;
the computing module is used for computing the imbalance degree of the cluster resources according to the resource distribution condition of the cloud platform; and
and the rescheduling module is used for rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource utilization rate in the scheduling rule and the first scheduling constraint.
In one or more embodiments of the present invention, the obtaining module is further configured to: preprocessing the historical operating data and storing the preprocessed historical operating data in a time sequence database; and performing histogram calculation on corresponding data of the running load at each moment in the time sequence database to obtain the resource usage of the application load.
In one or more embodiments of the invention, the computing module is further configured to: according to the cloud and the tableDetermining resource distribution condition of station, determining resource allocation rate eve of each node in cluster(n,k)And the average resource allocation rate avg of the clustern(ii) a And calculating the imbalance degree of the cluster resources according to the deviation of the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster.
In one or more embodiments of the invention, the computing module is further configured to:
Figure BDA0003417965830000041
and k is the number of the nodes in the cluster, app is the actual usage amount of the same cloud platform resource of the nodes in the cluster, and node is the actual capacity of the same cloud platform resource of the nodes in the cluster.
In one or more embodiments of the invention, the computing module is further configured to: calculating the deviation between the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster:
Figure BDA0003417965830000042
calculating the imbalance degree C of the cluster resources:
C＝∑wn。
in one or more embodiments of the present invention, the rescheduling module is further configured to: judging whether the residual capacity of the cluster resources is larger than a preset capacity value or not; if so, selecting the node with the deviation exceeding a preset deviation value, and calculating the application load distribution condition after rescheduling; if not, calculating the application load distribution condition of the nodes in the cluster after global rescheduling.
In one or more embodiments of the present invention, the rescheduling module is further configured to: arranging the sequence of application load scheduling according to the application load scheduling condition, and generating a node scheduling rule of the designated node; and injecting the scheduling rule into a primary scheduler and rescheduling the application load by combining the node scheduling rule.
In another aspect of the present invention, there is provided an electronic device including: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a method of applying load rescheduling in a cloud environment as described above.
In another aspect of the invention, a computer readable storage medium is provided, having stored thereon a computer program, which when executed by a processor, carries out the steps of the method of applying load rescheduling in a cloud environment as described.
Compared with the prior art, according to the method and the application for rescheduling the application load in the cloud environment, the running data of the application load can be monitored in multiple dimensions, the corresponding application portrait and the corresponding scheduling rule can be generated, the imbalance degree of cluster resources can be obtained according to the distribution condition of the running data of all the application loads in the cluster, rescheduling is performed on the basis of first scheduling according to the imbalance degree, the utilization rate of cloud platform resources is improved, and the problems that the resources cannot be fully used due to fragmentation caused by scheduling in the cloud environment are effectively solved.
Drawings
FIG. 1 is a flow diagram of a method of applying load rescheduling in a cloud environment, according to an embodiment of the present invention;
FIG. 2 is a general block diagram of a method for applying load rescheduling in a cloud environment according to an embodiment of the present invention;
FIG. 3 is a detailed flowchart of a method for application load rescheduling in a cloud environment according to an embodiment of the present invention;
FIG. 4 is a block diagram of a generation module of a method for applying load rescheduling in a cloud environment according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an example of CPU allocation rates for a method of applying load rescheduling in a cloud environment according to an embodiment of the present invention;
fig. 6 is a structural diagram of an apparatus for applying load rescheduling in a cloud environment according to an embodiment of the present invention;
fig. 7 is a hardware configuration diagram of a computing device applying load rescheduling in a cloud environment according to an embodiment of the present invention.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
The technical solutions provided by the embodiments of the present invention are described in detail below with reference to the accompanying drawings.
Example 1
As shown in fig. 1 to 2, a method for applying load rescheduling in a cloud environment according to an embodiment of the present invention is described, which includes the following steps.
In step S101, historical operating data of application loads on cluster nodes is obtained to generate an application representation.
The method comprises the steps of collecting operation data of a physical machine and an application load on a node, reporting the operation data to a monitoring analysis component for unified storage and analysis, analyzing the operation data collected in a cloud platform by the monitoring analysis component to obtain the resource use condition of the application load at a certain moment, and generating an application portrait according to the resource use condition of the application load.
In step S102, the application representation is converted into a scheduling rule.
And converting the application image into a scheduling rule which can be sensed by a rescheduling system, wherein the scheduling rule comprises the resource utilization rate and the application type. In this embodiment, the CPU utilization, the memory utilization, the disk utilization, and the network bandwidth utilization are used for subsequent scheduling of application loads, and the application types are used by the rescheduler, so that a batch of applications with too much load demand on the same class are prevented from being scheduled too much on one physical machine, and resource contention under the over-selling condition is further prevented.
In step S103, the application type in the scheduling rule is called to obtain the resource distribution of the cloud platform.
And the rescheduling system calls the monitoring analysis component to provide an interface according to a preset period, acquires monitoring data of a CPU, a memory, a disk, a network and the like of the physical machine in the cluster, and allocates all application loads on the physical machine.
In step S104, the imbalance of the cluster resources is calculated according to the resource distribution of the cloud platform.
Calculating the imbalance degree of cluster resources based on the distribution conditions of the CPUs, the memories, the disks and the networks of all application loads in the whole cluster, which is specifically as follows:
calculating the average resource allocation rate avg of all node CPUs in the clustercpuAnd resource allocation rate of each node CPU:
Figure BDA0003417965830000071
and k is the number of nodes in the cluster, app is the actual usage amount of the node CPU in the cluster, and node is the actual capacity of the node CPU in the cluster.
avgcpuRepresents the proportion of the CPU consumption of all nodes in the cluster and the sum of the actual CPU capacity of all nodes, namely the average CPU resource allocation rate, eve, of the whole cluster(cpu,k)The ratio of the consumption of the CPU of the application on each node in the cluster to the actual capacity of the CPU of each node, namely the resource allocation rate of the CPU of each node, is shown.
Calculating the standard deviation between the resource allocation rate of the CPU nodes of all the nodes in the cluster and the average allocation rate of the CPU resources:
Figure BDA0003417965830000072
the method aims to ensure that the resource utilization rate deviation of each node in the whole cluster is not large, and the effect of resource balance of all nodes is achieved.
By analogy, the average resource allocation rate of the memory, the disk and the network, the resource allocation rate of the node and the corresponding standard deviation can be obtained, which is specifically as follows:
Figure BDA0003417965830000081
Figure BDA0003417965830000082
Figure BDA0003417965830000083
Figure BDA0003417965830000084
Figure BDA0003417965830000085
Figure BDA0003417965830000086
calculating the sum of standard deviations of a CPU, a memory, a disk and a network of the application load, and representing the unbalance degree C of the cluster resources:
C＝wcpu+wmemory device+wMagnetic disk+wNetwork
And if the imbalance degree of the cluster resources exceeds a preset threshold value, triggering the rescheduling of the cluster.
In step S105, the application load on the nodes in the cluster is rescheduled according to the imbalance of the cluster resources, the resource usage rate in the scheduling rule, and the first scheduling constraint.
And when the imbalance degree of the whole cluster is not serious and the residual capacity of the resources is large, selecting a node with large deviation ideal value of the balance degree and adjusting the application load on the node. If the cluster resource residual capacity is tense and the fragmentation is serious, the global rescheduling is adopted, and the distribution of all application loads is obtained by optimized solution, so that the imbalance degree of the whole cluster is minimum.
In addition, the constraint of the first scheduling and the scheduling rule generated after the portrait is applied need to be taken into consideration during the rescheduling calculation, so that the resource distribution after rescheduling can meet the first scheduling rule, and the distribution of the clusters can be more balanced.
Specifically, based on the scheduling conditions of the application loads, the scheduling conditions of the application loads are sequenced to generate a node scheduling rule of the designated node, the scheduling rule generated according to the application portrait is injected into a primary scheduler to reschedule the application loads, and the node scheduling rule of the designated node is combined in the scheduling process. And deleting the rescheduling rule after the whole rescheduling is finished.
Through the operation data of the multi-dimensional monitoring application loads, the corresponding application portrait and the corresponding scheduling rule are generated, the imbalance degree of cluster resources can be further obtained according to the distribution condition of the operation data of all the application loads in the cluster, rescheduling is performed on the basis of primary scheduling according to the imbalance degree, the utilization rate of cloud platform resources is further improved, and the problems that resources are fragmented and cannot be fully used due to scheduling in a cloud environment are effectively solved.
Example 2
As shown in fig. 3 to 5, a method for applying load rescheduling in a cloud environment according to an embodiment of the present invention is described, which includes the following steps.
In step S201, the historical operating data is preprocessed and stored in the time-series database.
As shown in fig. 4, the monitoring acquisition end operates on the physical machine as a monitoring log collector, collects the operating conditions of the application load operating on the physical machine, acquires the operating data including the CPU utilization rate, the memory utilization rate, the disk utilization rate and the network bandwidth utilization rate, and reports the monitored operating data to the monitoring analysis component for uniform storage and analysis in an http manner according to a preset acquisition period.
The monitoring analysis component preprocesses the reported data, adopts (CPU _ cost, memory _ cost, disk _ cost, network _ cost) quadruple to describe the resource use condition of the application load at a certain moment, and stores the data into a time sequence database, thereby facilitating the subsequent analysis processing. The API interface provided by the monitoring analysis component is used by other platforms, and the analysis result is the basis of the rescheduling decision.
In step S202, histogram calculation is performed on the corresponding data of the operation load at each time in the time-series database to obtain the resource usage amount of the application load.
The histogram calculation is performed on the time series data indicating the operation condition of the application load at each time, and in the present embodiment, 80% of the data interval is used to indicate the resource usage amount of the application load. If the consumption of the application load in the CPU exceeds a preset threshold, the application is defined as calculating the consumption, and is similarly applicable to memory, disk I/O and network I/O, which is not described herein any more, and finally the image of the application is obtained.
The following procedure is taken as an example:
Figure BDA0003417965830000101
the application name is test, the CPU usage of the application is 200, the memory usage is 512Mi, the disk usage is 100, and the network usage is 128, aiming at the test application, only the CPU exceeds a preset threshold value, the application is defined as calculating consumable property, the usage of other operation data does not exceed the threshold value, and the image of the test application can be obtained according to the data.
In step S203, the application representation is converted into a scheduling rule.
In the embodiment, the CPU utilization rate, the memory utilization rate, the disk utilization rate and the network bandwidth utilization rate are used for subsequent application load scheduling, the application types are used by a rescheduler, a batch of applications with too much load demands of the same type are prevented from being too much scheduled on one physical machine, and resource contention under the condition of over-selling is further avoided.
In step S204, the imbalance of the cluster resources is calculated according to the resource distribution of the cloud platform.
And the rescheduling system calls the monitoring analysis component to provide an interface according to a preset period, acquires monitoring data of a CPU, a memory, a disk, a network and the like of the physical machine in the cluster, and allocates all application loads on the physical machine.
Calculating the average resource allocation rate of the whole cluster based on the allocation conditions of the CPUs, the memories, the disks and the networks of all application loads in the whole cluster, wherein the average resource allocation rate of the whole cluster is as follows:
calculating the average resource allocation rate avg of all node CPUs in the clustercpuAnd resource allocation rate of each node CPU:
Figure BDA0003417965830000111
and k is the number of nodes in the cluster, app is the actual usage amount of the node CPU in the cluster, and node is the actual capacity of the node CPU in the cluster.
avgcpuRepresents the proportion of the CPU consumption of all nodes in the cluster and the sum of the actual CPU capacity of all nodes, namely the average CPU resource allocation rate, eve, of the whole cluster(cpu,k)The ratio of the consumption of the CPU of the application on each node in the cluster to the actual capacity of the CPU of each node, namely the resource allocation rate of the CPU of each node, is shown.
Calculating the standard deviation between the resource allocation rate of the CPU nodes of all the nodes in the cluster and the average allocation rate of the CPU resources:
Figure BDA0003417965830000121
the method aims to ensure that the resource utilization rate deviation of each node in the whole cluster is not large, and the effect of resource balance of all nodes is achieved.
By analogy, the average resource allocation rate of the memory, the disk and the network, the resource allocation rate of the node and the corresponding standard deviation can be obtained, which is specifically as follows:
Figure BDA0003417965830000122
Figure BDA0003417965830000123
Figure BDA0003417965830000124
Figure BDA0003417965830000125
Figure BDA0003417965830000126
Figure BDA0003417965830000131
calculating the sum of standard deviations of a CPU, a memory, a disk and a network of the application load, and representing the unbalance degree C of the cluster resources:
C＝wcpu+wmemory device+wMagnetic disk+wNetwork
And if the imbalance degree of the cluster resources exceeds a preset threshold value, triggering the rescheduling of the cluster.
In step S205, the application load on the nodes in the cluster is rescheduled according to the imbalance of the cluster resources, the resource usage rate in the scheduling rule, and the first scheduling constraint.
And when the imbalance degree of the whole cluster is not serious and the residual capacity of the resources is large, selecting a node with large deviation ideal value of the balance degree and adjusting the application load on the node. If the cluster resource residual capacity is tense and the fragmentation is serious, the global rescheduling is adopted, and the distribution of all application loads is obtained by optimized solution, so that the imbalance degree of the whole cluster is minimum.
The rescheduling algorithm is described by taking adjustment based on a CPU as an example, and the rescheduling algorithm aims at redistributing the application load on the nodes through the algorithm so that the imbalance degree of the whole cluster is minimum. Specifically, the CPU allocation rates of the nodes in the cluster are sorted, as shown in fig. 5, each point represents a certain node CPU allocation rate, the CPU allocation rates of most of the nodes are near the expected water level of the CPU of the whole cluster, the allocation rates of some nodes are lower, the allocation rates of some nodes are higher, and the application load on the higher node is selected to be adjusted to the node with the lower allocation rate. When selecting the application load to be adjusted, the constraint of the first scheduling and the scheduling rule of the application sketch generation need to be considered, and whether the application load is allowed to be adjusted is judged. The imbalance of the whole cluster is recalculated after the adjustment. An application load distribution is finally obtained through repeated adjustment and calculation, and the imbalance can be minimized.
In addition, the constraint of the first scheduling and the scheduling rule generated after the portrait is applied need to be taken into consideration during the rescheduling calculation, so that the resource distribution after rescheduling can meet the first scheduling rule, and the distribution of the clusters can be more balanced.
Specifically, based on the scheduling conditions of the application loads, the scheduling conditions of the application loads are sequenced to generate a node scheduling rule of the designated node, the scheduling rule generated according to the application portrait is injected into a primary scheduler to reschedule the application loads, and the node scheduling rule of the designated node is combined in the scheduling process. And deleting the rescheduling rule after the whole rescheduling is finished.
As shown in fig. 6, an apparatus for applying load rescheduling in a cloud environment according to an embodiment of the present invention is introduced.
In the embodiment of the present invention, the apparatus for rescheduling an application load in a cloud environment includes an obtaining module 601, a converting module 602, a calling module 603, a calculating module 604, and a rescheduling module 605.
An obtaining module 601, configured to obtain historical operating data of an application load on a cluster node to generate an application representation;
a conversion module 602, configured to convert the application image into a scheduling rule, where the scheduling rule includes a resource usage rate and an application type;
the calling module 603 is configured to call an application type in the scheduling rule to obtain a resource distribution condition of the cloud platform;
the calculating module 604 is configured to calculate an imbalance of the cluster resources according to a resource distribution condition of the cloud platform; and
and a rescheduling module 605, configured to reschedule the application load on the node in the cluster according to the imbalance of the cluster resources, the resource usage rate in the scheduling rule, and the first scheduling constraint.
The obtaining module 601 is further configured to: preprocessing historical operating data and storing the preprocessed historical operating data in a time sequence database; and performing histogram calculation on corresponding data of the running load at each moment in the time sequence database to obtain the resource usage of the application load.
The calculation module 604 is further configured to: determining the resource allocation rate eve of each node in the cluster according to the resource distribution condition of the cloud platform(n,k)And the average resource allocation rate avg of the clustern(ii) a And calculating the imbalance degree of the cluster resources according to the deviation of the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster.
The calculation module 604 is further configured to:
Figure BDA0003417965830000151
and k is the number of nodes in the cluster, app is the actual usage amount of the same cloud platform resource of the nodes in the cluster, and node is the actual capacity of the same cloud platform resource of the nodes in the cluster.
The calculation module 604 is further configured to: calculating the deviation between the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster:
Figure BDA0003417965830000152
calculating the imbalance degree C of the cluster resources:
C＝∑wn。
the rescheduling module 605 is further configured to: judging whether the residual capacity of the cluster resources is larger than a preset capacity value or not; if so, selecting the nodes with the deviation exceeding the preset deviation value, and calculating the application load distribution condition after rescheduling; and if not, calculating the application load distribution condition of the nodes in the cluster after global rescheduling.
The rescheduling module 605 is further configured to: arranging the sequence of application load scheduling according to the application load scheduling condition, and generating a node scheduling rule of the designated node; and injecting the scheduling rules into the primary scheduler and rescheduling the application load by combining the node scheduling rules.
Fig. 7 illustrates a hardware block diagram of a computing device 70 for applying load rescheduling in a cloud environment, according to an embodiment of the present description. As shown in fig. 7, computing device 70 may include at least one processor 701, storage 702 (e.g., non-volatile storage), memory 703, and communication interface 704, and the at least one processor 701, storage 702, memory 703, and communication interface 704 are connected together via a bus 705. The at least one processor 701 executes at least one computer readable instruction stored or encoded in the memory 702.
It should be appreciated that the computer-executable instructions stored in the memory 702, when executed, cause the at least one processor 701 to perform the various operations and functions described above in connection with fig. 1-7 in the various embodiments of the present description.
In embodiments of the present description, computing device 70 may include, but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, Personal Digital Assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronics, and so forth.
According to one embodiment, a program product, such as a machine-readable medium, is provided. A machine-readable medium may have instructions (i.e., elements described above as being implemented in software) that, when executed by a machine, cause the machine to perform various operations and functions described above in connection with fig. 1-7 in the various embodiments of the present specification. Specifically, a system or apparatus may be provided which is provided with a readable storage medium on which software program code implementing the functions of any of the above embodiments is stored, and causes a computer or processor of the system or apparatus to read out and execute instructions stored in the readable storage medium.
According to the method and the application for rescheduling the application load in the cloud environment, the running data of the application load can be monitored in a multi-dimensional mode, the corresponding application portrait and the corresponding scheduling rule can be generated, the imbalance degree of cluster resources can be obtained according to the distribution condition of the running data of all the application loads in the cluster, rescheduling is performed on the basis of first scheduling according to the imbalance degree, the utilization rate of cloud platform resources is further improved, and the problem that the resources cannot be fully used due to fragmentation caused by scheduling in the cloud environment is effectively solved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims (10)

1. A method for application load rescheduling in a cloud environment, the method comprising:
acquiring historical operating data of an application load on a cluster node to generate an application portrait;
converting the application representation into a scheduling rule, wherein the scheduling rule comprises a resource utilization rate and an application type;
calling the application type in the scheduling rule to acquire the resource distribution condition of the cloud platform;
calculating the imbalance of cluster resources according to the resource distribution condition of the cloud platform; and
and rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource utilization rate in the scheduling rule and the first scheduling constraint.
2. The method for rescheduling application load in a cloud environment according to claim 1, wherein obtaining historical operating data of application load on cluster nodes to generate an application representation specifically comprises:
preprocessing the historical operating data and storing the preprocessed historical operating data in a time sequence database;
and performing histogram calculation on corresponding data of the running load at each moment in the time sequence database to obtain the resource usage of the application load.
3. The method of application load rescheduling in a cloud environment of claim 1, wherein said cloud platform resources n comprise at least one of a CPU, memory, disk, and network;
calculating the imbalance of the cluster resources according to the resource distribution condition of the cloud platform, specifically comprising:
determining the resource allocation rate eve of each node in the cluster according to the resource distribution condition of the cloud platform(n,k)And the average resource allocation rate avg of the clustern；
And calculating the imbalance degree of the cluster resources according to the deviation of the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster.
4. The method for application load rescheduling in a cloud environment according to claim 3, wherein the resource allocation rate eve of each node in the cluster is determined according to the resource distribution of the cloud platform(n,k)And the average resource allocation rate avg of the clusternThe method comprises the following steps:
Figure FDA0003417965820000021
and k is the number of the nodes in the cluster, app is the actual usage amount of the same cloud platform resource of the nodes in the cluster, and node is the actual capacity of the same cloud platform resource of the nodes in the cluster.
5. The method of claim 4, wherein calculating the imbalance of cluster resources according to the deviation between the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster comprises:
calculating the deviation between the resource allocation rate of each node in the cluster and the average resource allocation rate of the cluster:
Figure FDA0003417965820000022
calculating the imbalance degree C of the cluster resources:
C＝∑wn。
6. the method of claim 5, wherein the rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource usage rate in the scheduling rules, and the first scheduling constraint specifically comprises:
judging whether the residual capacity of the cluster resources is larger than a preset capacity value or not; if so,
selecting the nodes with the deviation exceeding a preset deviation value, and calculating the application load distribution condition after rescheduling; if not, the user can not select the specific application,
and calculating the application load distribution condition of the nodes in the cluster after global rescheduling.
7. The method of claim 6, wherein the rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource usage in the scheduling rules, and the first scheduling constraint further comprises:
arranging the sequence of application load scheduling according to the application load scheduling condition, and generating a node scheduling rule of the designated node;
and injecting the scheduling rule into a primary scheduler and rescheduling the application load by combining the node scheduling rule.
8. An apparatus for applying load rescheduling in a cloud environment, the apparatus comprising:
the acquisition module is used for acquiring historical operating data of an application load on the cluster node to generate an application portrait;
the conversion module is used for converting the application portrait into a scheduling rule, and the scheduling rule comprises resource utilization rate and application type;
the calling module is used for calling the application type in the scheduling rule to acquire the resource distribution condition of the cloud platform;
the computing module is used for computing the imbalance degree of the cluster resources according to the resource distribution condition of the cloud platform; and
and the rescheduling module is used for rescheduling the application load on the nodes in the cluster according to the imbalance of the cluster resources, the resource utilization rate in the scheduling rule and the first scheduling constraint.
9. An electronic device, comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of applying load rescheduling in a cloud environment of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of applying load rescheduling in a cloud environment according to any one of claims 1 to 7.
CN202111553945.2A 2021-12-17 2021-12-17 Method and application for rescheduling application load in cloud environment Pending CN114253688A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111553945.2A CN114253688A (en) 2021-12-17 2021-12-17 Method and application for rescheduling application load in cloud environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111553945.2A CN114253688A (en) 2021-12-17 2021-12-17 Method and application for rescheduling application load in cloud environment

Publications (1)

Publication Number Publication Date
CN114253688A true CN114253688A (en) 2022-03-29

Family

ID=80792846

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111553945.2A Pending CN114253688A (en) 2021-12-17 2021-12-17 Method and application for rescheduling application load in cloud environment

Country Status (1)

Country Link
CN (1) CN114253688A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115665157A (en) * 2022-11-14 2023-01-31 杭州谐云科技有限公司 Balanced scheduling method and system based on application resource types

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115665157A (en) * 2022-11-14 2023-01-31 杭州谐云科技有限公司 Balanced scheduling method and system based on application resource types

Similar Documents

Publication Publication Date Title
WO2021159638A1 (en) Method, apparatus and device for scheduling cluster queue resources, and storage medium
CN109739627B (en) Task scheduling method, electronic device and medium
CN115543624A (en) Heterogeneous computing power arrangement scheduling method, system, equipment and storage medium
CN114500339B (en) Node bandwidth monitoring method and device, electronic equipment and storage medium
CN114253688A (en) Method and application for rescheduling application load in cloud environment
CN115981843A (en) Task scheduling method and device in cloud-edge cooperative power system and computer equipment
CN110096339B (en) System load-based capacity expansion and contraction configuration recommendation system and method
CN112948109B (en) Quota flexible scheduling method, device and medium for AI computing cluster
CN106933673B (en) Method and device for adjusting number of logical threads of component
CN117271100B (en) Algorithm chip cluster scheduling method, device, computer equipment and storage medium
CN116302580B (en) Method and device for scheduling calculation force resources of nano relay
CN112463361A (en) Method and equipment for distributing elastic resources of distributed computation
CN112148483A (en) Container migration method and related device
CN111124439A (en) Intelligent dynamic unloading algorithm with cloud edge cooperation
CN111813546B (en) Resource allocation method, system and related device for multi-network connection application
CN115994029A (en) Container resource scheduling method and device
CN112130974B (en) Cloud computing resource configuration method and device, electronic equipment and storage medium
CN114860449A (en) Data processing method, device, equipment and storage medium
CN113419863A (en) Data distribution processing method and device based on node capability
CN109901931B (en) Reduction function quantity determination method, device and system
CN109190789B (en) Medium-and-long-term wind power prediction method and device, computer equipment and storage medium
CN111598390A (en) Server high availability evaluation method, device, equipment and readable storage medium
CN111951123A (en) Method and device for controlling electrical load, computer equipment and storage medium
CN113535378A (en) Resource allocation method, storage medium and terminal equipment
CN113760484A (en) Data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination