CN107070965A - A kind of Multi-workflow resource provision method virtualized under container resource - Google Patents
A kind of Multi-workflow resource provision method virtualized under container resource Download PDFInfo
- Publication number
- CN107070965A CN107070965A CN201611199049.XA CN201611199049A CN107070965A CN 107070965 A CN107070965 A CN 107070965A CN 201611199049 A CN201611199049 A CN 201611199049A CN 107070965 A CN107070965 A CN 107070965A
- Authority
- CN
- China
- Prior art keywords
- resource
- workflow
- container
- task
- virtualization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1023—Server selection for load balancing based on a hash applied to IP addresses or costs
Abstract
The present invention discloses a kind of Multi-workflow resource provision method virtualized under container resource, workflow is scheduled using intensified learning, provisioning resources are carried out, define resource utility index U, the supply-demand relationship between the task of operation and virtualization container resource in each scheduling of resource moment container cluster is established, the design of reward functions meets the requirement of the Multi-workflow resource generation using container cluster as granularity：It should ensure that the number amount and type of container cluster inner pressurd vessel unit meet the operational process of cloud workflow, avoid the workflow of different QoS requirements to violate service level agreements again, improve whole container cluster resource utilization.The status information of task in each container cluster can be obtained in real time, by workflow task distribution and the mutual collaboration of virtual resources supply.
Description
Technical field
The present invention relates to field of cloud calculation, and in particular to the Multi-workflow resource provision side under a kind of virtualization container resource
Method.
Background technology
Stream task and the cooperative self-adapted scheduling right and wrong of virtual resources are operated under instantaneous ten thousand cloud computing environments become
Often difficult.Such as Amazon, IBM, Microsoft, Yahoo data center possess hundreds of thousands platform server, what Google possessed
Number of servers has been even more than 1,000,000, and number is huger after various physical resource virtualizations, physical node and virtualization
Unit delay machine, be dynamically added and cancel etc. and happen occasionally, management is got up, and technical difficulty is big, complexity is high.And for example, with multiple bayes method
It is often unpredictable due to loading changing rule caused by accident exemplified by Service Workflow.
For task optimization distribution angle, scheduling of various types of cloud workflow tasks on multiple processing units is
It is proved to be the complete problems of NP.For resource optimization supply angle, on the one hand dummy unit is placed need to consider energy resource consumption, i.e.,
Reduce activation physical machine and using the network equipment quantity, now virtualization unit place can it is abstract be bin packing, this is one
The individual complete problems of NP；On the other hand transmission of the data between dummy unit need to be considered, that is, reduces the use to the network bandwidth, this
When dummy unit place can it is abstract be quadratic assignment problem, this is equally a complete problem of NP.Existing cloud workflow schedule
Or the workflow task distribution laid particular emphasis under fixed virtual resources, or the flexible resource confession laid particular emphasis under the change of work current load
Give, or how lay particular emphasis on by among existing Workflow Management System involvement cloud platform, it is impossible to by workflow task distribution and void
Planization resource provision is mutually cooperateed with.
The content of the invention
Workflow task can be distributed and virtualized there is provided one kind present invention aim to address the defect of prior art
The workflow resource supply method that resource provision is mutually cooperateed with, the technical scheme of use is as follows：
A kind of Multi-workflow resource provision method virtualized under container resource, using the container resource based on intensified learning
Generation strategy, comprises the following steps：
Definition status space：State space is represented with five-tuple S=(WR, RA, AW, IM, PJ), wherein WR is work to be dispatched
Make the workload of stream task, RA is resource pot life, and AW is the amount of work of workflow task in waiting list, and IM is the free time
Container number of resources, PJ is the ratio of each user's submission workflow task in queue；
Define motion space：Motion space includes two actions of number of resources of workflow task to be allocated and request；
Set reward functions Re=λeW+(1-λe) U, wherein λe∈ [0,1] is control coefrficient；W is task responsiveness：Execution time are that workflow task performs time, waitng
Time is the workflow task stand-by period, and U is resource utility index：[Tk,...,Tk+1] represent
Resource provision decision-making moment, PkRepresent [Tk,...,Tk+1] used vessel resource, f in moment container clusternRepresent TNMoment workflow
Task execution time summation；
Set reward functions higher limit Ru, lower limit Rl, hold in range Rm~Rn；
Pending workflow task is selected from motion space, the task of selection is performed, detection obtains reward functions Rε；
If reward functions RεMore than Ru, then in the follow-up implementation procedure of the task, the virtualization container in increase cloud platform
Resource, if reward functions RεLess than Ru, then in the follow-up implementation procedure of the task, the virtualization container resource of cloud platform is reduced,
If reward functions RεIn Rm~RnIn the range of, then the virtualization container resource in cloud platform is kept constant.
The present invention is scheduled to workflow using intensified learning, carries out provisioning resources, is defined resource utility index U, is built
The supply-demand relationship between the task of operation and virtualization container resource, reward functions in each scheduling of resource moment container cluster are found
Design meet using container cluster as granularity Multi-workflow resource generation requirement：The number of container cluster inner pressurd vessel unit should be ensured
Amount and type meet the operational process of cloud workflow, avoid the workflow of different QoS requirements to violate service level agreements again, carry
High whole container cluster resource utilization.The status information of task in each container cluster can be obtained in real time, by workflow task distribution and
The mutual collaboration of virtual resources supply.
Preferably, present invention additionally comprises being disposed to virtualization container resource, specifically including：
Container hierarchical clustering is virtualized in cluster based on minimal cut；
Optimize network traffics using local search algorithm；
Placed using best match algorithm optimization virtualization container：When placing the virtualization container newly created, from having made
First physical machine starts to search for successively, finds and is placed with the physical machine that the virtualization container is most matched, only when
All physical machines used just enable a new physical machine when can not all accommodate the virtualization container.
It is full that most matching of the present invention refers to the current remaining available resource (including CPU, internal memory, bandwidth) for having enabled physical machine
The new requirement for creating virtualization container to resource of foot, while surplus resources most probable meets the new virtualization container that creates next time to money
The requirement in source, or surplus resources are minimum.
The utilization local search algorithm optimizes network traffics：With maximum link utilization or hot-spot link number
For object function, selection produces congestion link on the basis of minimal cut hierarchical clustering result and the virtualization of maximum flow is held
Device, is swapped, then calculating target function with the container under the neighbor switch of left and right at random：If target function value reduces,
Then receive this time to exchange；Do not reduce such as, then refusal is exchanged, and is repeated in, until being recycled to the iterations of setting.
Compared with prior art, beneficial effects of the present invention：
The present invention is scheduled to workflow using intensified learning, carries out provisioning resources, is defined resource utility index U, is built
The supply-demand relationship between the task of operation and virtualization container resource, reward functions in each scheduling of resource moment container cluster are found
Design meet using container cluster as granularity Multi-workflow resource generation requirement：The number of container cluster inner pressurd vessel unit should be ensured
Amount and type meet the operational process of cloud workflow, avoid the workflow of different QoS requirements to violate service level agreements again, carry
High whole container cluster resource utilization, the status information of task in each container cluster can be obtained in real time, by workflow task distribution and
The mutual collaboration of virtual resources supply.
Brief description of the drawings
Fig. 1 is the system model schematic diagram of the present invention；
Fig. 2 is the cluster inner pressurd vessel hierarchical clustering schematic diagram based on minimal cut of the present invention.
Embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.
Embodiment：
The system model of Multi-workflow resource provision under virtualization container resource is as shown in figure 1, the supply of resource aspect
It is to submit the demand of workflow task to correspond to specific resource user so that the best performance of application, while the utilization of resource
Rate is also improved as far as possible.Workflow under cloud environment is made up of a series of subtasks, the now supply of resource aspect,
How to choose suitable available resources to complete the establishment of dummy unit first, that is, dummy unit generation problem；Its
Secondary is the occupancy for the physical machine number and network bandwidth for how reducing activation, that is, virtual unit Placement Problems.
The present embodiment reaches that resource provision mutually cooperates with the effect of matching with workflow task using following technical scheme：
A kind of Multi-workflow resource provision method virtualized under container resource, using the container resource based on intensified learning
Generation strategy, comprises the following steps：
Definition status space：State space is represented with five-tuple S=(WR, RA, AW, IM, PJ), wherein WR is work to be dispatched
Make the amount of stream task, RA is resource pot life, and AW is the total amount of workflow task in waiting list, and IM is free container resource
Number, PJ is the ratio of each user's submission workflow task in queue；
Define motion space：Motion space includes two actions of number of resources of workflow task to be allocated and request；
Set reward functions Re=λeW+(1-λe) U, wherein λe∈ [0,1] is control coefrficient；W is task responsiveness：Execution time are that workflow task performs time, waitng
Time is the workflow task stand-by period, and U is resource utility index：[Tk,...,Tk+1] represent
Resource provision decision-making moment, PkRepresent [Tk,...,Tk+1] used vessel resource, f in moment container clusternRepresent TNMoment workflow
Task execution time summation；
Set reward functions higher limit Ru, lower limit Rl, hold in range Rm~Rn；
Pending workflow task is selected from motion space, the task of selection is performed, detection obtains reward functions Rε；
If reward functions RεMore than Ru, then in the follow-up implementation procedure of the task, the virtualization container in increase cloud platform
Resource, if reward functions RεLess than Ru, then in the follow-up implementation procedure of the task, the virtualization container resource of cloud platform is reduced,
If reward functions RεIn Rm~RnIn the range of, then the virtualization container resource in cloud platform is kept constant.
The present embodiment also includes disposing virtualization container resource, specifically includes：
Container hierarchical clustering is virtualized in cluster based on minimal cut；
Optimize network traffics using local search algorithm；
Placed using best match algorithm optimization virtualization container：When placing the virtualization container newly created, from having made
First physical machine starts to search for successively, finds and is placed with the physical machine that the virtualization container is most matched, only when
All physical machines used just enable a new physical machine when can not all accommodate the virtualization container.
Cloud workflow DAG figures are represented with G=(V, E), wherein V represents container cluster, and E represents the flow between cluster inner pressurd vessel,
Node set is expressed asThe set expression on side is δ (Q).Then figure G in, a summit on side in set Q, another
Summit belong to V Q, whenOr during Q ≠ V, the side in δ (Q) just constitutes a cut set, it is expressed as (Q, V Q).For each
Side (i, j) ∈ E, there is the capacity C of a non-negativei,j.And the capacity of a cut set can be defined as each edge capacity in cut set
Summation, is represented by：C (Q, V Q)=∑i,j∈δ(Q)C(i,j)。
Hierarchical clustering based on minimal cut is exactly that the minimum cut set of a capacity is looked in figure G.Illustrate such as by taking Fig. 2 as an example
Under, figure G minimal cut hierarchical clusterings result can be represented with binary tree T (V), and left subtree TL is the node in Q, and weight is boundary values in Q
And W (TL)=∑i,j∈δ(Q)C(i,j)；Right subtree TR be V Q node, weight be V in Q boundary values summation W (TR)=
∑i,j∈δ(Q)C (i, j), if W (TL)<W (TR), then exchange left and right subtree, to ensure that communication flows is big always in left subtree TL
In right subtree.
The utilization local search algorithm optimizes network traffics：With maximum link utilization or hot-spot link number
For object function, selection produces congestion link on the basis of minimal cut hierarchical clustering result and the virtualization of maximum flow is held
Device, is swapped, then calculating target function with the container under the neighbor switch of left and right at random：If target function value reduces,
Then receive this time to exchange；Do not reduce such as, then refusal is exchanged, and is repeated in, until being recycled to the iterations of setting.
Claims (3)
1. a kind of Multi-workflow resource provision method virtualized under container resource, it is characterised in that using based on intensified learning
Container resource generation strategy, comprise the following steps：
Definition status space：State space is represented with five-tuple S=(WR, RA, AW, IM, PJ), wherein WR is to treat traffic control stream
The workload of task, RA is resource pot life, and AW is the amount of work of workflow task in waiting list, and IM is free container
Number of resources, PJ is the ratio of each user's submission workflow task in queue；
Define motion space：Motion space includes two actions of number of resources of workflow task to be allocated and request；
Set reward functions Re=λeW+(1-λe) U, wherein λe∈ [0,1] is control coefrficient, and W is task responsiveness：Execution time are that workflow task performs time, waitng
Time is the workflow task stand-by period, and U is resource utility index：[Tk,...,Tk+1] represent
Resource provision decision-making moment, PkRepresent [Tk,...,Tk+1] used vessel resource, f in moment container clusternRepresent TNMoment workflow
Task execution time summation；
Set reward functions higher limit Ru, lower limit Rl, hold in range Rm~Rn；
Pending workflow task is selected from motion space, the task of selection is performed, detection obtains reward functions Rε；
If reward functions RεMore than Ru, then in the follow-up implementation procedure of the task, the virtualization container money in increase cloud platform
Source, if reward functions RεLess than Ru, then in the follow-up implementation procedure of the task, the virtualization container resource of cloud platform is reduced, if
Reward functions RεIn Rm~RnIn the range of, then the virtualization container resource in cloud platform is kept constant.
2. a kind of Multi-workflow resource provision method virtualized under container resource according to claim 1, its feature exists
In, in addition to virtualization container resource is disposed, specifically include：
Container hierarchical clustering is virtualized in cluster based on minimal cut；
Optimize network traffics using local search algorithm；
Placed using best match algorithm optimization virtualization container：When placing the virtualization container newly created, from what is used
First physical machine starts to search for successively, finds and is placed with the physical machine that the virtualization container is most matched, only when all
The physical machine used just enables a new physical machine when can not all accommodate the virtualization container.
3. a kind of Multi-workflow resource provision method virtualized under container resource according to claim 2, its feature exists
In the utilization local search algorithm optimization network traffics are specially：Using maximum link utilization or hot-spot link number as mesh
Scalar functions, selection produces the virtualization container of congestion link and maximum flow on the basis of minimal cut hierarchical clustering result, with
Machine is swapped with the container under the neighbor switch of left and right, then calculating target function：If target function value reduces, receive
This time exchange；Do not reduce such as, then refusal is exchanged, and is repeated in, until being recycled to the iterations of setting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199049.XA CN107070965B (en) | 2016-12-22 | 2016-12-22 | Multi-workflow resource supply method under virtualized container resource |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199049.XA CN107070965B (en) | 2016-12-22 | 2016-12-22 | Multi-workflow resource supply method under virtualized container resource |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107070965A true CN107070965A (en) | 2017-08-18 |
| CN107070965B CN107070965B (en) | 2021-02-19 |
Family
ID=59618941
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611199049.XA Active CN107070965B (en) | 2016-12-22 | 2016-12-22 | Multi-workflow resource supply method under virtualized container resource |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107070965B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107908457A (en) * | 2017-11-08 | 2018-04-13 | 河海大学 | A kind of containerization cloud resource distribution method based on stable matching |
| CN108829494A (en) * | 2018-06-25 | 2018-11-16 | 杭州谐云科技有限公司 | Container cloud platform intelligence method for optimizing resources based on load estimation |
| CN109639791A (en) * | 2018-12-06 | 2019-04-16 | 广东石油化工学院 | Cloud workflow schedule method and system under a kind of container environment |
| CN115022188A (en) * | 2022-05-27 | 2022-09-06 | 国网经济技术研究院有限公司 | Container placement method and system in power edge cloud computing network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007080047A1 (en) * | 2006-01-10 | 2007-07-19 | International Business Machines Corporation | Method and system for sending and processing electronic messages |
| CN103699446A (en) * | 2013-12-31 | 2014-04-02 | 南京信息工程大学 | Quantum-behaved particle swarm optimization (QPSO) algorithm based multi-objective dynamic workflow scheduling method |
| CN105260230A (en) * | 2015-10-30 | 2016-01-20 | 广东石油化工学院 | Resource scheduling method for data center virtual machine based on segmented service level agreement |
| CN105897864A (en) * | 2016-03-28 | 2016-08-24 | 东南大学 | Scheduling method for cloud workflow |
-
2016
- 2016-12-22 CN CN201611199049.XA patent/CN107070965B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007080047A1 (en) * | 2006-01-10 | 2007-07-19 | International Business Machines Corporation | Method and system for sending and processing electronic messages |
| CN103699446A (en) * | 2013-12-31 | 2014-04-02 | 南京信息工程大学 | Quantum-behaved particle swarm optimization (QPSO) algorithm based multi-objective dynamic workflow scheduling method |
| CN105260230A (en) * | 2015-10-30 | 2016-01-20 | 广东石油化工学院 | Resource scheduling method for data center virtual machine based on segmented service level agreement |
| CN105897864A (en) * | 2016-03-28 | 2016-08-24 | 东南大学 | Scheduling method for cloud workflow |
Non-Patent Citations (1)
| Title |
|---|
| 董健康: "IaaS环境下改进能源效率和网络性能的虚拟机放置方法", 《通信学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107908457A (en) * | 2017-11-08 | 2018-04-13 | 河海大学 | A kind of containerization cloud resource distribution method based on stable matching |
| CN107908457B (en) * | 2017-11-08 | 2020-03-17 | 河海大学 | Containerized cloud resource allocation method based on stable matching |
| CN108829494A (en) * | 2018-06-25 | 2018-11-16 | 杭州谐云科技有限公司 | Container cloud platform intelligence method for optimizing resources based on load estimation |
| CN108829494B (en) * | 2018-06-25 | 2020-09-29 | 杭州谐云科技有限公司 | Container cloud platform intelligent resource optimization method based on load prediction |
| CN109639791A (en) * | 2018-12-06 | 2019-04-16 | 广东石油化工学院 | Cloud workflow schedule method and system under a kind of container environment |
| CN115022188A (en) * | 2022-05-27 | 2022-09-06 | 国网经济技术研究院有限公司 | Container placement method and system in power edge cloud computing network |
| CN115022188B (en) * | 2022-05-27 | 2024-01-09 | 国网经济技术研究院有限公司 | Container placement method and system in electric power edge cloud computing network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107070965B (en) | 2021-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Abdulhamid et al. | Fault tolerance aware scheduling technique for cloud computing environment using dynamic clustering algorithm | |
| Liu et al. | Solving the multi-objective problem of IoT service placement in fog computing using cuckoo search algorithm | |
| Wadhwa et al. | TRAM: Technique for resource allocation and management in fog computing environment | |
| Mansouri et al. | Cost-based job scheduling strategy in cloud computing environments | |
| CN114138486A (en) | Containerized micro-service arranging method, system and medium for cloud edge heterogeneous environment | |
| CN104601664B (en) | A kind of control system of cloud computing platform resource management and scheduling virtual machine | |
| CN107070965A (en) | A kind of Multi-workflow resource provision method virtualized under container resource | |
| Kimpan et al. | Heuristic task scheduling with artificial bee colony algorithm for virtual machines | |
| Li et al. | An effective scheduling strategy based on hypergraph partition in geographically distributed datacenters | |
| CN106201701A (en) | A kind of workflow schedule algorithm of band task duplication | |
| Kansal et al. | Classification of resource management approaches in fog/edge paradigm and future research prospects: a systematic review | |
| Mahato et al. | Balanced task allocation in the on‐demand computing‐based transaction processing system using social spider optimization | |
| Hu et al. | Ecsched: Efficient container scheduling on heterogeneous clusters | |
| Delavar et al. | A synthetic heuristic algorithm for independent task scheduling in cloud systems | |
| Selvi et al. | Resource allocation issues and challenges in cloud computing | |
| Rodrigues et al. | Network-aware container scheduling in multi-tenant data center | |
| de Souza Toniolli et al. | Resource allocation for multiple workflows in cloud-fog computing systems | |
| Seth et al. | Dynamic threshold-based dynamic resource allocation using multiple VM migration for cloud computing systems | |
| Singh et al. | Artificial Intelligence Based Virtual Machine Allocation and Migration Policy using Improved MBFD | |
| Chalack et al. | Resource allocation in cloud environment using approaches based particle swarm optimization | |
| Singh et al. | Efficient resource management technique for performance improvement in cloud computing | |
| Imdoukh et al. | Optimizing scheduling decisions of container management tool using many‐objective genetic algorithm | |
| Hamzeh et al. | A new approach to calculate resource limits with fairness in kubernetes | |
| Malathy et al. | Performance improvement in cloud computing using resource clustering | |
| Oh et al. | Job placement using reinforcement learning in GPU virtualization environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |