CN115981865A - Service resource scheduling method and device, electronic equipment and storage medium - Google Patents
Service resource scheduling method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115981865A CN115981865A CN202211743243.5A CN202211743243A CN115981865A CN 115981865 A CN115981865 A CN 115981865A CN 202211743243 A CN202211743243 A CN 202211743243A CN 115981865 A CN115981865 A CN 115981865A
- Authority
- CN
- China
- Prior art keywords
- task
- scheduling
- node
- execution
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 230000002776 aggregation Effects 0.000 claims abstract description 124
- 238000004220 aggregation Methods 0.000 claims abstract description 124
- 238000010586 diagram Methods 0.000 claims abstract description 62
- 238000009826 distribution Methods 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000035484 reaction time Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008602 contraction Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005293 physical law Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a service resource scheduling method, a service resource scheduling device, electronic equipment and a storage medium, which are used for solving the technical problems of low resource utilization rate and low service scheduling efficiency of the conventional service resource scheduling method. The invention comprises the following steps: acquiring scheduling arrangement information of each task; generating a service sequence chart by adopting the scheduling arrangement information; performing affinity aggregation on each service based on the service sequence diagram to obtain an aggregation grouping; acquiring the minimum resource demand and the execution start-stop time of the aggregation grouping; determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution starting and stopping time and a preset node resource upper limit; and acquiring machine nodes based on the node leasing requirements, and distributing each task to the machine nodes for resource scheduling according to the node task distribution information.
Description
Technical Field
The present invention relates to the field of resource scheduling technologies, and in particular, to a method and an apparatus for scheduling service resources, an electronic device, and a storage medium.
Background
With the rise of cloud computing, resource on-demand dynamic scaling has become a normal state. How to more efficiently realize automatic scaling service so as to avoid excessive waste of resources becomes a research focus. At present, the open source community provides three solutions for automatic expansion and contraction based on a Kubernetes scheme: pod Horizontal auto-scaling (HPA), pod Vertical auto-scaling (VPA) and Cluster auto-scaling (CA). On the basis of the open source community scheme, a task queue mechanism is introduced to improve the scheme, and based on the task queue, the clusters can be logically grouped, so that project management of users is facilitated. On one hand, different quota computing resources can be configured for different projects, and on the other hand, different types of services can be divided into different task queues for management. And more refined management is carried out on business and computing resources.
However, even though cloud computing has good elasticity, the elasticity is still restricted by real physical laws, and meanwhile, elastic expansion and contraction need to depend on data collection and analysis, and by default, an open source community can only passively capture the CPU and memory usage of Pod once every 10 seconds. And the summary analysis is performed once every minute (the frequency of grabbing and summarizing analysis is too high, which causes resource waste). If the new nodes are created too much, resources are wasted. While the time required to create a new Pod on a new node is determined by four main factors: HPA reaction time, CA reaction time, node reaction time, and Pod creation time. Creating a new node in the process of executing a service may cause a time delay and affect the service efficiency.
Disclosure of Invention
The invention provides a service resource scheduling method, a service resource scheduling device, electronic equipment and a storage medium, which are used for solving the technical problems of low resource utilization rate and low service scheduling efficiency of the conventional service resource scheduling method.
The invention provides a service resource scheduling method, wherein the service comprises a plurality of tasks; the method comprises the following steps:
acquiring scheduling arrangement information of each task;
generating a service sequence chart by adopting the scheduling arrangement information;
performing affinity aggregation on each service based on the service timing diagram to obtain aggregation groups;
acquiring the minimum resource demand and the execution start-stop time of the aggregation grouping;
determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution starting and stopping time and a preset node resource upper limit;
and acquiring machine nodes based on the node leasing requirements, and distributing each task to the machine nodes for resource scheduling according to the node task distribution information.
Optionally, the step of generating a service timing diagram by using the scheduling information includes:
generating an execution resource demand graph by adopting the scheduling and arranging information;
acquiring task starting time of each execution resource demand graph;
and generating a business sequence diagram according to the task starting time and each execution resource demand graph.
Optionally, the step of generating an execution resource demand graph by using the scheduling information includes:
acquiring resource demand information of each task from the scheduling and arranging information;
acquiring a dependency relationship among the tasks;
dividing all the tasks into a plurality of task groups according to the dependency relationship;
acquiring the scheduling time of each task in the task group;
and generating an execution resource demand graph by adopting the resource demand information and the scheduling duration of each task in the task group.
Optionally, the step of generating a service timing diagram according to the task start time and each execution resource demand graph includes:
calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling time of each task and the dependency relationship;
and generating a service timing chart of the service according to the scheduling time of all tasks of the service.
Optionally, the execution start-stop time includes an execution start time and an execution end time; the aggregation grouping comprises a plurality of aggregation tasks; the step of obtaining the minimum resource demand amount and the execution start-stop time of the aggregation grouping comprises the following steps:
calculating the execution ending time of each aggregation task according to the scheduling time length and the scheduling time of each aggregation task;
taking the scheduling time with the most time in the aggregation task as the execution starting time of the aggregation grouping;
taking the last execution ending time in the aggregation task as the execution ending time of the aggregation grouping;
acquiring a resource demand lower limit value of each aggregation task;
and calculating the sum of the lower limit values of the resource demands of all the aggregation tasks of the aggregation grouping to obtain the lowest resource demand of the aggregation grouping.
Optionally, the machine node leasing requirements include the number of machine nodes and the leasing duration of the machine nodes; the step of determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution starting and stopping time and a preset node resource upper limit comprises the following steps:
determining the number of the participating nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and a preset node resource upper limit;
splicing the participating nodes of each aggregation grouping according to the number of the participating nodes of each aggregation grouping and the execution starting and ending time to generate a plurality of participating node combinations, and determining the duration of each participating node combination;
determining the number of the participating node combinations as the number of machine nodes, and determining the duration of the participating node combinations as the lease duration of the machine nodes;
and determining node task allocation information according to the tasks in each participating node combination.
The invention also provides a service resource scheduling device, wherein the service comprises a plurality of tasks; the device comprises:
the scheduling and arranging information acquisition module is used for acquiring scheduling and arranging information of each task;
a service sequence chart generating module, configured to generate a service sequence chart by using the scheduling and scheduling information;
the aggregation module is used for carrying out affinity aggregation on each service based on the service time sequence chart to obtain an aggregation grouping;
a minimum resource demand and execution start-stop time obtaining module, configured to obtain a minimum resource demand and an execution start-stop time of the aggregation group;
a machine node leasing requirement and node task allocation information determining module, configured to determine a machine node leasing requirement and node task allocation information by using the minimum resource demand amount, the execution start-stop time, and a preset node resource upper limit;
and the scheduling module is used for acquiring the machine nodes based on the node leasing requirements and allocating each task to the machine nodes according to the node task allocation information to perform resource scheduling.
Optionally, the service timing diagram generating module includes:
the execution resource demand graph generation submodule is used for generating an execution resource demand graph by adopting the scheduling and arranging information;
a task starting time obtaining submodule, configured to obtain a task starting time of each execution resource demand graph;
and the business sequence diagram generating submodule is used for generating a business sequence diagram according to the task starting time and each execution resource demand diagram.
The invention also provides an electronic device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the traffic resource scheduling method according to instructions in the program code.
The present invention also provides a computer-readable storage medium for storing a program code for executing the traffic resource scheduling method as described in any one of the above.
According to the technical scheme, the invention has the following advantages: the invention provides a service resource scheduling method, which comprises the following steps: acquiring scheduling and arranging information of each task; generating a service sequence chart by adopting scheduling arrangement information; performing affinity aggregation on each service based on the service timing diagram to obtain aggregation groups; acquiring the minimum resource demand and the execution start-stop time of the aggregation grouping; determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, execution start-stop time and a preset node resource upper limit; and acquiring the machine nodes based on the node leasing requirements, and distributing each task to the machine nodes for resource scheduling according to the node task distribution information. The invention determines the relation between the execution time of each task by generating the service sequence diagram of the tasks in the service, thereby carrying out affinity aggregation on each task according to the relation, dividing the tasks in the same time interval together from the angle of the execution time sequence to carry out the allocation of machine nodes, and further improving the resource utilization rate of the machine nodes as much as possible; meanwhile, the machine nodes can be created in advance by pre-calculating the requirements for the machine nodes, so that the machine nodes can be directly called when resource scheduling is carried out, waiting time is reduced, and the efficiency of service scheduling is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a flowchart illustrating steps of a method for scheduling service resources according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating steps of a method for scheduling service resources according to another embodiment of the present invention;
FIG. 3 is a diagram illustrating an initial execution resource requirement graph according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an execution resource requirement graph according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a service timing diagram according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of machine node allocation based on an execution resource requirement graph;
fig. 7 is a schematic diagram of an aggregation grouping division according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a participating node allocation according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a join node splicing according to an embodiment of the present invention;
fig. 10 is a block diagram of a service resource scheduling apparatus according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a service resource scheduling method, a service resource scheduling device, electronic equipment and a storage medium, which are used for solving the technical problems of low resource utilization rate and low service scheduling efficiency of the conventional service resource scheduling method.
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for scheduling service resources according to an embodiment of the present invention.
The invention provides a service resource scheduling method, wherein a service comprises a plurality of tasks, and the method specifically comprises the following steps:
in the embodiment of the present invention, the scheduling and scheduling information may include the name and type of the task, the scheduling time of the task, and the computing resources and memory resources required by the task.
and the service timing diagram is a timing diagram for executing tasks in the service, and the execution sequence of each task can be known through the service timing diagram.
After the scheduling orchestration information of the tasks is obtained, a business timing graph may be generated based on the scheduling orchestration information of all tasks of the business.
103, performing affinity aggregation on each service based on the service timing diagram to obtain aggregation groups;
in the embodiment of the present invention, the affinity refers to a time overlapping degree of execution times of the tasks, and the higher the time overlapping degree is, the stronger the affinity is.
And aggregating the services according to the time overlapping degree of the services in the service timing diagram to obtain different aggregation groups. The execution time of tasks in the same aggregation group is similar.
104, acquiring the minimum resource demand and the execution start-stop time of the aggregation grouping;
the minimum resource requirement of the aggregate cluster is the sum of the minimum required computational and memory resources for each task within the aggregate cluster.
The execution start-stop time is a start time of a task that starts to be executed first and an end time of a task that completes last in the aggregated cluster.
According to the minimum resource demand of the aggregation clustering, the upper limit of the node resource required by the aggregation clustering can be known; according to the execution starting and ending time of the aggregation clustering, the service time of the nodes required by the aggregation clustering can be known.
the node resource upper limit refers to a resource upper limit that a node can provide in a unit time, and the resource includes a computing resource and a memory resource.
The machine node leasing requirement refers to the leasing number and the leasing duration of the machine nodes needing to be leased and with known resource upper limits.
In the cloud computing, the machine nodes are called for paying, so that the number of leases of the machine nodes and the lease duration are reduced, and the lease cost can be effectively saved. Meanwhile, the number and the duration of leases of the machine nodes are controlled, and the utilization rate of resources can be improved under the condition that the task execution requirement is met.
The node task allocation information is information recording tasks allocated to machine nodes to run, and different machine nodes can allocate different tasks.
In the embodiment of the invention, after the lowest resource demand, the execution starting and ending time and the preset node resource upper limit are obtained, the machine node leasing demand and the node task allocation information can be determined according to the execution starting and ending time and the lowest resource demand.
And step 106, acquiring the machine nodes based on the node leasing requirements, and distributing each task to the machine nodes for resource scheduling according to the node task distribution information.
In the embodiment of the invention, the machine nodes meeting the lease requirement can be obtained, and each task is distributed to the corresponding machine node for resource scheduling according to the node task distribution information, so that the normal operation of the task is realized.
The invention determines the relation between the execution time of each task by generating the service sequence diagram of the tasks in the service, thereby carrying out affinity aggregation on each task according to the relation, dividing the tasks in the same time interval together from the angle of the execution time sequence to carry out the allocation of machine nodes, and further improving the resource utilization rate of the machine nodes as much as possible; meanwhile, the machine node can be created in advance by pre-calculating the requirement on the machine node so as to be directly called when resource scheduling is carried out, waiting time is reduced, and the efficiency of service scheduling is improved.
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for scheduling service resources according to another embodiment of the present invention. The method specifically comprises the following steps:
in the embodiment of the present invention, the scheduling and scheduling information may include the name and type of the task, the scheduling time of the task, and the computing resources and memory resources required by the task.
Example codes of scheduling orchestration information are as follows:
and the resource demand graph is used for representing resources required by the execution of a series of tasks with the dependency relationship. And recording the dependency relationship among the tasks in the resource demand graph, and the name, type, scheduling time and required resources of each task.
In the embodiment of the invention, the execution resource demand graph for executing the series tasks can be generated according to the scheduling and arranging information of each task.
In one example, the step of generating an execution resource requirement map using scheduling orchestration information may comprise the sub-steps of:
s21, acquiring resource demand information of each task from the scheduling and arranging information;
s22, acquiring the dependency relationship among tasks;
s23, dividing all tasks into a plurality of task groups according to the dependency relationship;
s24, acquiring the scheduling duration of each task in the task group;
and S25, generating an execution resource demand graph by adopting the resource demand information and the scheduling duration of each task in the task group.
In a specific implementation, a service may include several task groups having dependency relationships, and each task group may generate an execution resource requirement graph. Dependencies refer to the need for completion of one task based on the results of execution of another task.
Therefore, by obtaining the dependency relationship among the tasks, a plurality of task groups can be obtained, and each task group is arranged according to the dependency relationship, so that an initial execution resource demand graph as shown in fig. 3 can be obtained. In FIG. 3, A, B, C, D is four tasks, where task B, C is based on the results of task A and task D is based on the results of task B and task C. Each task is recorded with a corresponding Type (e.g., apple: 3.7), resource requirement information (CPU: 100m, memory.
Considering that the scheduling time of each task is different, the execution time of the task caused based on the dependency relationship may be affected by the execution time of the previous task and the size of the scheduling time, and for accurate scheduling of each task, the scheduling time of each task may be added in the initial execution resource demand graph, so as to calculate the execution time of each task. Accordingly, the generated execution resource requirement map carrying the scheduling Duration (Duration) is shown in fig. 4.
after the execution resource demand graphs of the task groups are obtained, the task groups can be arranged according to time according to the task starting time of the execution resource demand graphs, and a service timing chart of the service is obtained.
In one example, the step of generating the business sequence diagram according to the task starting time and each execution resource requirement diagram may include the following sub-steps:
s41, calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling duration and the dependency relationship of each task;
and S42, generating a service sequence diagram of the service according to the scheduling time of all tasks of the service.
The task start time for executing the resource requirement diagram refers to the execution time for executing the task which is first started to be executed in the resource requirement diagram.
The scheduling time is an execution time at which each task starts execution.
In specific implementation, the scheduling time of each task can be calculated according to the task start time of the execution resource demand graph and the scheduling duration and the dependency relationship of each task.
Taking fig. 4 as an example, task a is the first task to start execution in the execution resource demand graph, and if the scheduling time is 0 minutes 0 seconds and the scheduling time is 32 seconds, the scheduling time of task B and task C depending on the execution result is 0 minutes 32 seconds; the scheduling time of the task B is 35 seconds, and the scheduling time of the task C is 55 seconds; since task D needs to be started depending on the common result of tasks B and C, the scheduling time of task D is based on the end time of task C that ends latest, and thus the scheduling time of task D is 1 minute 27 seconds.
And similarly, the scheduling time of each task in each execution resource requirement graph can be calculated.
As shown in fig. 5, fig. 5 is a schematic structural diagram of a service timing diagram according to an embodiment of the present invention. Assuming that the business comprises 26 tasks of A-Z, A-D, E-G, H-I, J-O, P-V, W-Z6 task groups can be formed according to the dependency relationship. Wherein, in the process of executing the service, the task group H-I needs to be executed twice. So that 7 execution resource requirement maps can be generated. Based on the scheduling time of the tasks in each execution resource demand graph, a corresponding business timing graph may be generated.
affinity is a time overlap of execution times of tasks, and the higher the time overlap, the stronger the affinity.
In an actual scenario, if machine nodes are allocated according to the execution resource demand graph, a machine node allocation result as shown in fig. 6 is formed.
In cloud computing, the invocation of the machine node needs to pay. According to the machine node allocation manner of fig. 6, the task is divided into three machine nodes (machine nodes 1, 2, and 3), and the utilization rate of the machine nodes is low. In order to reduce the payment cost, the number of machine nodes and the rental time need to be reduced.
In the embodiment of the invention, the tasks can be aggregated through affinity, so that the tasks in the same time interval can be processed in a centralized manner, and the number of machine nodes participating in the process can be reduced by processing the tasks which are originally processed by different machine nodes in a scattered manner in a centralized manner.
In a specific implementation, the services may be aggregated according to the time overlapping degree between each service in the service timing diagram, so as to obtain different aggregation groups. The execution time of the tasks in the same aggregation group is similar.
It should be noted that, the embodiment of the present invention does not constrain the aggregation algorithm, and those skilled in the art may adopt any feasible aggregation algorithm to perform aggregation by using affinity as an aggregation condition.
In one example, based on the traffic timing diagram of FIG. 5, the aggregated clusters shown in FIG. 7 may be aggregated, including aggregated clusters 1-4.
after the aggregation grouping is obtained, the minimum resource demand and the execution start-stop time of the aggregation grouping can be calculated, so that the resource scheduling is performed for the aggregation grouping arrangement node.
In one example, the execution start-stop time includes an execution start time and an execution end time; the aggregation grouping comprises a plurality of aggregation tasks; the step of obtaining the minimum resource demand of the aggregated cluster and the execution start-stop time may include the following sub-steps:
s61, calculating the execution ending time of each aggregation task according to the scheduling duration and the scheduling time of each aggregation task;
s62, taking the scheduling time with the most front time in the aggregation task as the execution starting time of the aggregation grouping;
s63, taking the last execution ending time in the aggregation task as the execution ending time of the aggregation grouping;
s64, acquiring a resource demand lower limit value of each aggregation task;
and S65, calculating the sum of the lower limit values of the resource demands of all the aggregation tasks of the aggregation grouping to obtain the lowest resource demand of the aggregation grouping.
In a practical scenario, the loading capacity of a single machine node is limited, the demand computing power of each task is a range, the demand memory of the task is a range, and the task is also limited by the scheduling time. Therefore, according to the resource requirement of each task, the scheduling time and the loading capacity of the machine node, the number of tasks which can be accommodated by each machine node can be calculated. Similarly, according to the resource demand and the scheduling time of the aggregation tasks in the aggregation cluster, the number of the participating nodes (pre-distributed machine nodes) and the lease duration required by the aggregation cluster can be calculated.
In a specific implementation, each aggregation group may include a plurality of aggregation tasks, the execution end time of each aggregation task may be calculated according to the scheduling time and the scheduling time of each aggregation task, and the lease time of a required participating node may be calculated according to the execution end time and the scheduling time of each task. And calculating the total minimum resource demand of the aggregation grouping according to the lower limit value of the resource demand of each aggregation task.
in the embodiment of the invention, the machine node leasing requirement and the node task allocation information can be determined by adopting the minimum resource demand, the execution starting and ending time and the preset node resource upper limit.
In one example, the machine node lease requirements include the number of machine nodes and lease duration of the machine nodes; the method comprises the following steps of determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, execution start-stop time and preset node resource upper limit, wherein the steps comprise:
s71, determining the number of the participating nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and a preset node resource upper limit;
s72, splicing the participating nodes of each aggregation grouping according to the number of the participating nodes of each aggregation grouping and the execution starting and ending time to generate a plurality of participating node combinations, and determining the duration of each participating node combination;
s73, determining the number of the participating node combinations as the number of the machine nodes, and determining the duration of the participating node combinations as the lease duration of the machine nodes;
and S74, determining node task allocation information according to the tasks in each participating node combination.
The preset node resource upper limit refers to the resource upper limit of a participating node (pre-distributed machine node), and simultaneously comprises a computing resource upper limit and a memory resource upper limit.
In a specific implementation, the number of participating nodes required by each aggregation cluster can be calculated according to the minimum resource demand of each aggregation cluster and a preset node resource upper limit. As shown in fig. 8, a, b, c, d, e are five participating nodes.
And splicing all the participating nodes according to the execution time of the participating nodes to obtain a plurality of participating node combinations, wherein the number of the participating node combinations can be the number of the machine nodes, the total duration of the participating node combinations can be the lease duration of the machine nodes, and the task in each participating node combination is the node task (recorded in the node task allocation information) allocated to the corresponding machine node. The splicing results are shown in fig. 9.
And 208, acquiring the machine nodes based on the node leasing requirements, and distributing each task to the machine nodes according to the node task distribution information to perform resource scheduling.
In the embodiment of the invention, the machine nodes meeting the lease requirement can be obtained, and each task is distributed to the corresponding machine node for resource scheduling according to the node task distribution information, so that the normal operation of the task is realized.
The invention determines the relation between the execution time of each task by generating the service sequence diagram of the tasks in the service, thereby carrying out affinity aggregation on each task according to the relation, dividing the tasks in the same time interval together from the angle of the execution time sequence to carry out the allocation of machine nodes, and further improving the resource utilization rate of the machine nodes as much as possible; meanwhile, the machine nodes can be created in advance by pre-calculating the requirements for the machine nodes, so that the machine nodes can be directly called when resource scheduling is carried out, waiting time is reduced, and the efficiency of service scheduling is improved.
Referring to fig. 10, fig. 10 is a block diagram illustrating a service resource scheduling apparatus according to an embodiment of the present invention.
The embodiment of the invention provides a service resource scheduling device, wherein a service comprises a plurality of tasks; the device comprises:
a scheduling information acquiring module 1001 configured to acquire scheduling information of each task;
a service sequence chart generating module 1002, configured to generate a service sequence chart by using the scheduling information;
the aggregation module 1003 is configured to perform affinity aggregation on each service based on the service timing diagram to obtain an aggregation group;
a minimum resource demand and execution start-stop time obtaining module 1004, configured to obtain a minimum resource demand and execution start-stop time of the aggregation grouping;
a machine node lease demand and node task allocation information determining module 1005, configured to determine machine node lease demand and node task allocation information by using a minimum resource demand amount, execution start-stop time, and a preset node resource upper limit;
and the scheduling module 1006 is configured to acquire the machine nodes based on the node lease requirements, and allocate each task to the machine node according to the node task allocation information to perform resource scheduling.
In this embodiment of the present invention, the service timing diagram generating module 1002 includes:
the execution resource demand graph generation submodule is used for generating an execution resource demand graph by adopting the scheduling and arranging information;
the task starting time obtaining submodule is used for obtaining the task starting time of each execution resource demand graph;
and the business sequence diagram generating submodule is used for generating a business sequence diagram according to the task starting time and each execution resource demand diagram.
In the embodiment of the present invention, the executing resource requirement diagram generating sub-module includes:
the resource demand information acquisition unit is used for acquiring the resource demand information of each task from the scheduling and arranging information;
the dependency relationship acquisition unit is used for acquiring the dependency relationship among the tasks;
the task group dividing unit is used for dividing all tasks into a plurality of task groups according to the dependency relationship;
the scheduling time acquiring unit is used for acquiring the scheduling time of each task in the task group;
and the execution resource demand graph generating unit is used for generating the execution resource demand graph by adopting the resource demand information and the scheduling duration of each task in the task group.
In the embodiment of the present invention, the service timing diagram generation submodule includes:
the scheduling time calculation unit is used for calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling duration and the dependency relationship of each task;
and the service sequence chart generating unit is used for generating the service sequence chart of the service according to the scheduling time of all the tasks of the service.
In the embodiment of the invention, the execution start-stop time comprises an execution start time and an execution end time; the aggregation grouping comprises a plurality of aggregation tasks; the minimum resource requirement and execution start-stop time obtaining module 1004 includes:
the execution end time calculation sub-module is used for calculating the execution end time of each aggregation task according to the scheduling time and the scheduling time of each aggregation task;
the execution starting time determining submodule is used for taking the scheduling time with the most front time in the aggregation task as the execution starting time of the aggregation grouping;
the execution termination time determining submodule is used for taking the last execution termination time in the aggregation task as the execution termination time of the aggregation grouping;
the resource demand lower limit value acquisition submodule is used for acquiring the resource demand lower limit value of each aggregation task;
and the minimum resource demand determining submodule is used for calculating the sum of the lower limit values of the resource demands of all the aggregation tasks of the aggregation grouping to obtain the minimum resource demand of the aggregation grouping.
In the embodiment of the invention, the machine node leasing requirements comprise the number of machine nodes and the leasing duration of the machine nodes; a module 1005 for determining lease requirement and node task allocation information of machine nodes includes:
the number determining submodule is used for determining the number of the participating nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and a preset node resource upper limit;
the time length determining submodule is used for splicing the participating nodes of each aggregation grouping according to the number and the execution starting and ending time of the participating nodes of each aggregation grouping to generate a plurality of participating node combinations and determine the time length of each participating node combination;
the machine node number and machine node lease duration determining submodule is used for determining the number of the participating node combinations as the number of the machine nodes and determining the duration of the participating node combinations as the lease duration of the machine nodes;
and the node task allocation information determining submodule is used for determining node task allocation information according to the tasks in each participating node combination.
An embodiment of the present invention further provides an electronic device, where the device includes a processor and a memory:
the memory is used for storing the program codes and transmitting the program codes to the processor;
the processor is used for executing the service resource scheduling method according to the instruction in the program code.
The embodiment of the invention also provides a computer-readable storage medium, which is used for storing the program code, and the program code is used for executing the service resource scheduling method of the embodiment of the invention.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The embodiments in the present specification are all described in a progressive manner, and each embodiment focuses on differences from other embodiments, and portions that are the same and similar between the embodiments may be referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or terminal apparatus that comprises the element.
The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for scheduling service resources is characterized in that the service comprises a plurality of tasks; the method comprises the following steps:
acquiring scheduling arrangement information of each task;
generating a service sequence chart by adopting the scheduling arrangement information;
performing affinity aggregation on each service based on the service timing diagram to obtain aggregation groups;
acquiring the minimum resource demand and the execution start-stop time of the aggregation grouping;
determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution starting and stopping time and a preset node resource upper limit;
and acquiring machine nodes based on the node leasing requirements, and distributing each task to the machine nodes for resource scheduling according to the node task distribution information.
2. The method of claim 1, wherein the step of generating the service timing diagram using the scheduling information comprises:
generating an execution resource demand graph by adopting the scheduling and arranging information;
acquiring task starting time of each execution resource demand graph;
and generating a business sequence diagram according to the task starting time and each execution resource demand graph.
3. The method of claim 1, wherein the step of generating an execution resource requirement map using the scheduling orchestration information comprises:
acquiring resource demand information of each task from the scheduling and arranging information;
acquiring a dependency relationship among the tasks;
dividing all the tasks into a plurality of task groups according to the dependency relationship;
acquiring the scheduling time of each task in the task group;
and generating an execution resource demand graph by adopting the resource demand information and the scheduling duration of each task in the task group.
4. The method according to claim 3, wherein the step of generating a business sequence diagram according to the task start time and each of the execution resource requirement maps comprises:
calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling duration of each task and the dependency relationship;
and generating a service timing chart of the service according to the scheduling time of all tasks of the service.
5. The method of claim 4, wherein the execution start-stop time comprises an execution start time and an execution end time; the aggregation grouping comprises a plurality of aggregation tasks; the step of obtaining the minimum resource demand and the execution start-stop time of the aggregation grouping comprises:
calculating the execution ending time of each aggregated task according to the scheduling time length and the scheduling time of each aggregated task;
taking the scheduling time with the most time in the aggregation task as the execution starting time of the aggregation grouping;
taking the execution ending time of the last time in the aggregation task as the execution ending time of the aggregation grouping;
acquiring a resource demand lower limit value of each aggregation task;
and calculating the sum of the lower limit values of the resource demands of all the aggregation tasks of the aggregation grouping to obtain the lowest resource demand of the aggregation grouping.
6. The method of claim 1, wherein the machine node lease requirements include a number of machine nodes and a machine node lease duration; the step of determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution starting and stopping time and a preset node resource upper limit comprises the following steps:
determining the number of the participating nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and a preset node resource upper limit;
splicing the participating nodes of each aggregation grouping according to the number of the participating nodes of each aggregation grouping and the execution starting and ending time to generate a plurality of participating node combinations, and determining the duration of each participating node combination;
determining the number of the participating node combinations as the number of machine nodes, and determining the duration of the participating node combinations as the lease duration of the machine nodes;
and determining node task allocation information according to the tasks in each participating node combination.
7. A service resource scheduling device is characterized in that the service comprises a plurality of tasks; the device comprises:
the scheduling and arranging information acquisition module is used for acquiring scheduling and arranging information of each task;
a service sequence chart generating module, configured to generate a service sequence chart by using the scheduling and scheduling information;
the aggregation module is used for carrying out affinity aggregation on each service based on the service timing diagram to obtain aggregation groups;
a minimum resource demand and execution start-stop time obtaining module, configured to obtain a minimum resource demand and an execution start-stop time of the aggregation group;
the machine node leasing demand and node task allocation information determining module is used for determining machine node leasing demand and node task allocation information by adopting the minimum resource demand, the execution starting and ending time and a preset node resource upper limit;
and the scheduling module is used for acquiring the machine nodes based on the node leasing requirements and allocating each task to the machine nodes for resource scheduling according to the node task allocation information.
8. The apparatus of claim 7, wherein the traffic sequence chart generating module comprises:
the execution resource demand graph generation submodule is used for generating an execution resource demand graph by adopting the scheduling and arranging information;
a task starting time obtaining submodule, configured to obtain a task starting time of each execution resource demand graph;
and the business sequence diagram generating submodule is used for generating a business sequence diagram according to the task starting time and each execution resource demand diagram.
9. An electronic device, wherein the device comprises a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the traffic resource scheduling method according to any one of claims 1 to 6 according to instructions in the program code.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium is configured to store program code for executing the traffic resource scheduling method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211743243.5A CN115981865B (en) | 2022-12-29 | 2022-12-29 | Service resource scheduling method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211743243.5A CN115981865B (en) | 2022-12-29 | 2022-12-29 | Service resource scheduling method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115981865A true CN115981865A (en) | 2023-04-18 |
| CN115981865B CN115981865B (en) | 2024-05-07 |
Family
ID=85966459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211743243.5A Active CN115981865B (en) | 2022-12-29 | 2022-12-29 | Service resource scheduling method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115981865B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109725993A (en) * | 2018-06-01 | 2019-05-07 | 中国平安人寿保险股份有限公司 | Task executing method, device, equipment and computer readable storage medium |
| US20210103482A1 (en) * | 2019-10-03 | 2021-04-08 | Microsoft Technology Licensing, Llc | Multi-phase distributed task coordination |
| CN112884319A (en) * | 2021-02-10 | 2021-06-01 | 腾讯大地通途（北京）科技有限公司 | Task allocation method and device, computer equipment and storage medium |
| CN114896067A (en) * | 2022-05-26 | 2022-08-12 | 中国平安人寿保险股份有限公司 | Automatic generation method and device of task request information, computer equipment and medium |
| CN114896049A (en) * | 2022-06-16 | 2022-08-12 | 中国电力科学研究院有限公司 | Method, system, equipment and medium for scheduling operation tasks of electric power artificial intelligence platform |
| CN115098240A (en) * | 2022-07-25 | 2022-09-23 | 中诚华隆计算机技术有限公司 | Multiprocessor application scheduling method and system and storage medium |
-
2022
- 2022-12-29 CN CN202211743243.5A patent/CN115981865B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109725993A (en) * | 2018-06-01 | 2019-05-07 | 中国平安人寿保险股份有限公司 | Task executing method, device, equipment and computer readable storage medium |
| US20210103482A1 (en) * | 2019-10-03 | 2021-04-08 | Microsoft Technology Licensing, Llc | Multi-phase distributed task coordination |
| CN112884319A (en) * | 2021-02-10 | 2021-06-01 | 腾讯大地通途（北京）科技有限公司 | Task allocation method and device, computer equipment and storage medium |
| CN114896067A (en) * | 2022-05-26 | 2022-08-12 | 中国平安人寿保险股份有限公司 | Automatic generation method and device of task request information, computer equipment and medium |
| CN114896049A (en) * | 2022-06-16 | 2022-08-12 | 中国电力科学研究院有限公司 | Method, system, equipment and medium for scheduling operation tasks of electric power artificial intelligence platform |
| CN115098240A (en) * | 2022-07-25 | 2022-09-23 | 中诚华隆计算机技术有限公司 | Multiprocessor application scheduling method and system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115981865B (en) | 2024-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108572873B (en) | Load balancing method and device for solving Spark data inclination problem | |
| Shen et al. | Scheduling jobs in the cloud using on-demand and reserved instances | |
| CN105912399B (en) | Task processing method, device and system | |
| CN111459641B (en) | Method and device for task scheduling and task processing across machine room | |
| US20110202657A1 (en) | Method for scheduling cloud-computing resource and system applying the same | |
| JP2004021982A (en) | Method and system for dynamically allocating computer system resource | |
| CN108279974B (en) | Cloud resource allocation method and device | |
| CN112685153A (en) | Micro-service scheduling method and device and electronic equipment | |
| CN112416585A (en) | GPU resource management and intelligent scheduling method for deep learning | |
| CN109189572B (en) | Resource estimation method and system, electronic equipment and storage medium | |
| CN111309440B (en) | Method and equipment for managing and scheduling multiple types of GPUs | |
| CN112130966A (en) | Task scheduling method and system | |
| US8028291B2 (en) | Method and computer program product for job selection and resource allocation of a massively parallel processor | |
| CN115543624A (en) | Heterogeneous computing power arrangement scheduling method, system, equipment and storage medium | |
| CN111124644B (en) | Method, device and system for determining task scheduling resources | |
| CN112286623B (en) | Information processing method and device and storage medium | |
| CN115981865B (en) | Service resource scheduling method and device, electronic equipment and storage medium | |
| CN108664322A (en) | Data processing method and system | |
| CN111143063A (en) | Task resource reservation method and device | |
| CN113419863B (en) | Data distribution processing method and device based on node capacity | |
| CN112988363B (en) | Resource scheduling method, device, server and storage medium | |
| Wang et al. | On optimal budget-driven scheduling algorithms for MapReduce jobs in the hetereogeneous cloud | |
| CN111343152B (en) | Data processing method and device, electronic equipment and storage medium | |
| Wang et al. | On scheduling algorithms for mapreduce jobs in heterogeneous clouds with budget constraints | |
| CN113342526A (en) | Dynamic management and control method, system, terminal and medium for cloud computing mobile network resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |