CN106095579B - Container resource allocation methods and device - Google Patents

Container resource allocation methods and device Download PDF

Info

Publication number
CN106095579B
CN106095579B CN201610416933.8A CN201610416933A CN106095579B CN 106095579 B CN106095579 B CN 106095579B CN 201610416933 A CN201610416933 A CN 201610416933A CN 106095579 B CN106095579 B CN 106095579B
Authority
CN
China
Prior art keywords
container
node
bandwidth
resource allocation
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610416933.8A
Other languages
Chinese (zh)
Other versions
CN106095579A (en
Inventor
雷磊
王志军
房秉毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China United Network Communications Group Co Ltd
Original Assignee
China United Network Communications Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China United Network Communications Group Co Ltd filed Critical China United Network Communications Group Co Ltd
Priority to CN201610416933.8A priority Critical patent/CN106095579B/en
Publication of CN106095579A publication Critical patent/CN106095579A/en
Application granted granted Critical
Publication of CN106095579B publication Critical patent/CN106095579B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • G06F9/5088Techniques for rebalancing the load in a distributed system involving task migration

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The present invention provides a kind of container resource allocation methods and device, belongs to field of communication technology.Container resource allocation methods of the invention, comprising: S1, according to the available bandwidth between the container and container of container N number of in network, using the available bandwidth between two containers as side right value, constructs undirected complete graph of having the right using container as node;S2, according to the complete graph, calculate in each cluster node, the bandwidth between any two node is one the smallest, bandwidth the smallest one between two node any in all clusters is compared, the maximum cluster node of bandwidth is calculated, it wherein, include K node, K≤N in each cluster node;S3, big data is affixed one's name on the cluster node top being calculated, is communicated.

Description

Container resource allocation methods and device
Technical field
The invention belongs to fields of communication technology, and in particular to a kind of container resource allocation methods and device.
Background technique
With universal, the carrying environment that container is replacing virtual machine to run as application of lightweight linux container.Together When, since the network connection situation of carrying physical host is different, the network connection situation gap between each container is larger, together When, due to being influenced by the variation of bottom physical machine network condition utilization rate, the network connection situation between each container is also Dynamic change, but this variation is fluctuated less within the scope of certain time.However, MPI/ is with lightweight linux container It is universal, container is replacing virtual machine to become the carrying environment of application operation.Simultaneously as the network of carrying physical host connects Situation difference is connect, the network connection situation gap between each container is larger, simultaneously as by bottom physical machine network condition The influence of utilization rate variation, the network connection situation between each container is also dynamic change, but this is changed certain Fluctuation is little in time range.However, the big data application of MPI/Hadoop type, needs to have preferable net between each node Therefore how network connectivity identifies the preferable one group of container of network connection, becomes the important class for ensureing big data application performance Topic.
And big data currently is disposed in container in application, relying primarily on the network connectivty choosing between static physical resource Destination node deployment container is taken, and then in the method for upper vessel portion administration big data system and application, the optimization ginseng that this method is chosen Number is static, the i.e. bandwidth of physical node.
However, on the one hand, the network occupancy situation of bottom physical resource be it is dynamic, i.e., static 10Mb/s bandwidth is not The available bandwidth for representing the current physical resource is 10Mb/s, meanwhile, static bandwidth is bigger, available bandwidth also bigger hypothesis from It is so also invalid.On the other hand, the performance of the physical resource of the bottom of this method concern, and the performance of bottom physical resource is not It can be simply mapped to the performance of the container resource on upper layer, if bottom physical resource available bandwidth is larger, but due to carrying too The available bandwidth of more containers, each container may be smaller.
In conclusion being applicable in current method is difficult to select the preferable one group of container of network connection in a period of time, One group of very poor container of network connectivty is possibly even selected, is answered in the upper vessel portion administration big data that current method selects With it is low to may cause big data effectiveness, waste of resource, extends the execution time of big data operation.
Summary of the invention
The technical problems to be solved by the invention include asking for above-mentioned existing for existing container resource allocation methods Topic provides the container resource allocation methods and device of a kind of performance that the big data that optimization is run on container is applied.
Solving technical solution used by present invention problem is a kind of container resource allocation methods, comprising:
S1, according to the available bandwidth between the container and container of container N number of in network, using container as node, with two hold Available bandwidth between device is side right value, constructs undirected complete graph of having the right;
S2, according to the complete graph, calculate in each cluster node, the bandwidth between any two node is one the smallest, Bandwidth the smallest one between two node any in all clusters is compared, the maximum cluster node of bandwidth is calculated, In, it include K node, K≤N in each cluster node;
S3, big data is affixed one's name on the cluster node top being calculated, is communicated.
Preferably, after step s 3 further include:
The available bandwidth between any two container is measured at regular intervals, repeats step S1 and S2.
It may further be preferable that described be at regular intervals measured the available bandwidth between any two container, It specifically includes:
By IPerf bandwidth measuring unit, the available bandwidth between any two container is surveyed at regular intervals It is fixed.
Preferably, band is calculated in described be compared bandwidth the smallest one between two node any in all clusters A wide maximum cluster node, specifically includes:
Using heuristic greedy principle, the smallest middle bandwidth of bandwidth between any two node is calculated in all clusters A maximum cluster node.
It may further be preferable that the inspiration condition of the heuristic greedy principle is the bottleneck figure based on the complete graph Data structure.
Solving technical solution used by present invention problem is a kind of container resource allocation device, comprising:
Complete graph constructing module, for the available bandwidth between the container and container according to container N number of in network, with container Undirected complete graph of having the right is constructed using the available bandwidth between two containers as side right value for node;
Optimal cluster node obtains module, for calculating in each cluster node according to the complete graph, any two node it Between bandwidth it is one the smallest, bandwidth the smallest one between two node any in all clusters is compared, bandwidth is calculated A maximum cluster node, wherein include K node, K≤N in each cluster node;
Communication module is communicated for affixing one's name to big data on the cluster node top being calculated.
Preferably, the container resource allocation device further include: bandwidth measuring unit, it is right at regular intervals to be used for Available bandwidth between any two container is measured.
It may further be preferable that the bandwidth measuring unit includes IPerf bandwidth measuring unit.
Preferably, the optimal cluster node obtains module and is specifically used for, and using heuristic greedy principle, institute is calculated There is in cluster the bandwidth the smallest one middle maximum cluster node of bandwidth between any two node.
It may further be preferable that the inspiration condition of the heuristic greedy principle is the bottleneck figure based on the complete graph Data structure.
The invention has the following beneficial effects:
Container resource allocation device provided by the present invention can identify the preferable cluster of connectivity in all containers Container node, with heuristic feasible operator principle ensure the aggregate bandwidth between the cluster inner pressurd vessel maximum, and the container across cluster it Between bandwidth it is minimum, so that the container preferentially chosen with group disposes big data application, so optimize run on it is big on container The performance of data application.
Detailed description of the invention
Fig. 1 is the flow chart of 1 container resource allocation methods of the embodiment of the present invention;
Fig. 2 is by the step S1 of 1 container resource allocation methods of the embodiment of the present invention undirected complete graph of having the right constructed Schematic diagram;
Fig. 3 is the schematic diagram of 1 container resource allocation device of the embodiment of the present invention.
Specific embodiment
Technical solution in order to enable those skilled in the art to better understand the present invention, with reference to the accompanying drawing and specific embodiment party Present invention is further described in detail for formula.
Embodiment 1:
As shown in Figure 1, including the following steps: the present embodiment provides a kind of container resource allocation methods
S1, according to the available bandwidth between the container and container of container N number of in network, using container as node, with two hold Available bandwidth between device is side right value, constructs undirected complete graph of having the right.
As shown in Fig. 2, each container can regard a node in figure as.Due to container connection between any two, and two Communication between container is not necessarily to forward via other containers, then the network topology of whole containers can regard as one it is undirected completely Figure, meanwhile, the bandwidth between any two container node can be by IPerf equiband measuring unit results of regular determination, and then gives Statistics available bandwidth in certain time interval out.The weight setting on side in Fig. 1 can be counted available bandwidth thus, such as connection section The weight on the side (u, v) of point u and node v is the statistics available bandwidth between container node u and v, is indicated with b ω (u, v), then The problem of finding connectivity preferable one group of container can be converted into the Graph partition problem of complete graph.
S2, according to the complete graph, calculate in each cluster node, the bandwidth between any two node is one the smallest, Bandwidth the smallest one between two node any in all clusters is compared, the maximum cluster node of bandwidth is calculated, In, it include K node, K≤N in each cluster node.
The step, cluster container node refer to the combination of multiple containers node;By taking the container node of N=10 i.e. 10 as an example, If necessary to the node cluster interior joint number K=3 of selection, then the node cluster to be chosen of the application is that 3 are chosen from 10 nodes One of all possibilities of node, it is known that it may select to be 10x9x8=720 kind, that is, what is chosen is possible in 720 1 kind in scheme.Specifically, in step sl according to the constructing network topology of container one undirected complete graph of having the right, therefore, Container node optimization sub-clustering problem can be converted to the weight size that one connects side according to node and carry out the excellent of complete graph division Change problem, optimization aim is that the bandwidth between the container node ensured in same node cluster is as big as possible, if being at most divided into k A network cluster, node cluster can use Ci, i=1...k expression, then the target equation of optimization problem may be defined as formula (1).
max(bω(u,v))u,v∈Ci, i=1...k (1)
Simultaneously it is desirable that the bandwidth resources between each container node cluster are evenly distributed, i.e., to all container node clusters and Speech maximizes the minimum value of bandwidth between any two container node in cluster, then target equation is shown in formula (2).
Wherein, container node optimization sub-clustering problem (Optimized Virt.Nodes k-Clustering problem) Optimize sub-clustering problem as container node using the container node sub-clustering problem that formula (2) is optimization aim equation.
For convenience of explanation, the present invention introduces a classical problem in complete graph Research on partition field in graph theory, and minimum is most The definition of big k cluster partition problem.
Minimax k cluster partition problem (min-max k-Clustering problem) given one have the right it is undirected completely Figure, is shown in formula (3),
Minimax k cluster partition problem, which is defined as dividing G=(V, G), to be at most k cluster and makes appointing in any cluster The maximum value for the weight on side between two nodes of anticipating is minimum, i.e. hypothesis C1....CkIt is ready-portioned cluster, then optimization aim equation is shown in formula (4)。
Wherein, container node optimization sub-clustering problem and minimax k cluster partition problem are equivalence problems.
It proves: enabling
Then formula (4) is converted into formula (6):
It enables
F (b ω (u, v))=b ω (u, v), b ω (u, v) > 0, u, v ∈ Ci, i=1...k (7)
According to the mathematical optimization problem definition of standard, formula (2) can be write
According to the mathematical optimization problem definition of standard, formula be can be written as
Formula (11) can be obtained by formula (7) and formula (8) and know that f (b ω (u, v)) and g (b ω (u, v)) change with b ω (u, v) Trend is on the contrary, so formula
Max (f (b ω (u, v)))=min (g (b ω (u, v))), b ω (u, v) > 0 (11)
So formula (9) and (10) are of equal value, i.e., formula (2) and formula (6) are of equal value.
However, minimax k cluster partition problem is NP-Hard problem, only when the weight on side between node meets triangle When inequality, i.e., formula (12) meets
Just there is the approximate algorithm for determining the approximation coefficient upper bound in minimax k cluster partition problem, i.e. the problem is that APX is asked Topic.
Bandwidth the smallest one between two node any in all clusters is compared and bandwidth is calculated most by the present embodiment A big cluster node is calculated in all clusters that bandwidth is the smallest between any two node specifically using heuristic greedy principle One middle maximum cluster node of bandwidth.Namely a kind of heuritic approach solving virtual resource node is devised in the present embodiment Optimize sub-clustering problem, inspiration condition is that the bottleneck figure based on complete graph finds maximum independent set, is then with these independent sets Central configuration node cluster.
Wherein, bottleneck figure (bottleneck graph) gives specific weight values ω, the bottleneck of an authorized graph G=(V, G) Figure is defined as formula (13).
WhereE (ω)={ e ∈ E: ω (e)≤ω } (13) Gb (ω)=G (V, (ω))
The bottleneck figure Gb (ω) that side of all weights greater than ω is formed original image G is deleted from original image, for side In the case that weight is positive value, Gb (0) is exactly the vertex set of original image G, and side collection is sky;Gb(ωmax) it is exactly original image G itself.
S3, big data is affixed one's name on the cluster node top being calculated, is communicated.
It after step s 3, can also include being surveyed at regular intervals to the available bandwidth between any two container Fixed step continues to repeat step S1 and S2 later;It wherein, specifically can be by IPerf bandwidth measuring unit, every certain Time is measured the available bandwidth between any two container.Wherein, time interval please can specifically be limited according to specifically, And why to be measured at regular intervals to the available bandwidth between any two container is because the occupancy of bandwidth is more It is few different time according to user number and it is different, therefore this kind of mode may insure that container resource allocation is optimal.
In conclusion container resource allocation methods provided by the present embodiment can identify connectivity in all containers Preferable cluster container node ensures that the aggregate bandwidth between the cluster inner pressurd vessel is maximum with heuristic feasible operator principle, and across Bandwidth between the container of cluster is minimum, to preferentially choose the container deployment big data application with group, and then optimizes and runs on appearance The performance of big data application on device.
Embodiment 2:
As shown in figure 3, the container money in embodiment 1 can be used the present embodiment provides a kind of container resource allocation device Source distribution method.Container resource allocation device in the present embodiment includes: complete graph constructing module, optimal cluster node acquisition mould Block, communication module;Wherein, complete graph constructing module, for the available band between the container and container according to container N number of in network Width, using the available bandwidth between two containers as side right value, constructs undirected complete graph of having the right using container as node;Optimal cluster knot Point obtains module, for calculating in each cluster node, the bandwidth between any two node is the smallest by one according to the complete graph Bandwidth the smallest one between two node any in all clusters is compared and the maximum cluster node of bandwidth is calculated by item, It wherein, include K node, K≤N in each cluster node;Communication module, in the big number of cluster node top administration being calculated According to being communicated.
Wherein, the optimal cluster node obtains module and is specifically used for, and using heuristic greedy principle, all clusters are calculated In the bandwidth the smallest one middle maximum cluster node of bandwidth between any two node.The inspiration item of the heuristic greedy principle Part is the data structure of the bottleneck figure based on the complete graph.
Certainly, the device resource allocation device in the present embodiment can also include: bandwidth measuring unit, be used for every certain Time is measured the available bandwidth between any two container.Wherein, bandwidth measuring unit includes IPerf bandwidth measurement list Member.
In conclusion container resource allocation device provided by the present embodiment can identify connectivity in all containers Preferable cluster container node ensures that the aggregate bandwidth between the cluster inner pressurd vessel is maximum with heuristic feasible operator principle, and across Bandwidth between the container of cluster is minimum, to preferentially choose the container deployment big data application with group, and then optimizes and runs on appearance The performance of big data application on device.
It is understood that the principle that embodiment of above is intended to be merely illustrative of the present and the exemplary implementation that uses Mode, however the present invention is not limited thereto.For those skilled in the art, essence of the invention is not being departed from In the case where mind and essence, various changes and modifications can be made therein, these variations and modifications are also considered as protection scope of the present invention.

Claims (10)

1. a kind of container resource allocation methods characterized by comprising
S1, according to the available bandwidth between the container and container of container N number of in network, using container as node, with two containers it Between available bandwidth be side right value, construct undirected complete graph of having the right;
S2, according to the complete graph, calculate in each cluster node, the bandwidth between any two node is one the smallest, by institute There is in cluster between any two node bandwidth the smallest one to be compared calculating, obtain cluster where maximum one, wherein It include K node, K≤N in each cluster node;
S3, big data is affixed one's name on the cluster node top being calculated, is communicated.
2. container resource allocation methods according to claim 1, which is characterized in that after step s 3 further include:
The available bandwidth between any two container is measured at regular intervals, repeats step S1 and S2.
3. container resource allocation methods according to claim 2, which is characterized in that described at regular intervals to any two Available bandwidth between a container is measured, and is specifically included:
By IPerf bandwidth measuring unit, the available bandwidth between any two container is measured at regular intervals.
4. container resource allocation methods according to claim 1, which is characterized in that described by two node any in all clusters Between bandwidth the smallest one be compared the maximum cluster node of bandwidth be calculated, specifically include:
Using heuristic greedy principle, it is maximum that bandwidth the smallest one middle bandwidth between any two node is calculated in all clusters A cluster node.
5. container resource allocation methods according to claim 4, which is characterized in that the inspiration of the heuristic greedy principle Condition is the data structure of the bottleneck figure based on the complete graph.
6. a kind of container resource allocation device characterized by comprising
Complete graph constructing module is section with container for the available bandwidth between the container and container according to container N number of in network Point constructs undirected complete graph of having the right using the available bandwidth between two containers as side right value;
Optimal cluster node obtains module, for calculating in each cluster node, between any two node according to the complete graph Bandwidth is one the smallest, is compared calculating for bandwidth the smallest one between two node any in all clusters, obtains wherein most Cluster where one of big, wherein include K node, K≤N in each cluster node;
Communication module is communicated for affixing one's name to big data on the cluster node top being calculated.
7. container resource allocation device according to claim 6, which is characterized in that the container resource allocation device also wraps Include: bandwidth measuring unit is used at regular intervals be measured the available bandwidth between any two container.
8. container resource allocation device according to claim 7, which is characterized in that the bandwidth measuring unit includes IPerf bandwidth measuring unit.
9. container resource allocation device according to claim 6, which is characterized in that the optimal cluster node obtains module tool Body is used for, and using heuristic greedy principle, the smallest middle bandwidth of bandwidth between any two node is calculated in all clusters A maximum cluster node.
10. container resource allocation device according to claim 9, which is characterized in that the heuristic greedy principle opens Clockwork spring part is the data structure of the bottleneck figure based on the complete graph.
CN201610416933.8A 2016-06-14 2016-06-14 Container resource allocation methods and device Active CN106095579B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610416933.8A CN106095579B (en) 2016-06-14 2016-06-14 Container resource allocation methods and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610416933.8A CN106095579B (en) 2016-06-14 2016-06-14 Container resource allocation methods and device

Publications (2)

Publication Number Publication Date
CN106095579A CN106095579A (en) 2016-11-09
CN106095579B true CN106095579B (en) 2019-06-07

Family

ID=57846759

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610416933.8A Active CN106095579B (en) 2016-06-14 2016-06-14 Container resource allocation methods and device

Country Status (1)

Country Link
CN (1) CN106095579B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106911777A (en) * 2017-02-24 2017-06-30 郑州云海信息技术有限公司 A kind of data processing method and server
CN109525407B (en) * 2017-09-18 2020-05-26 中国科学院声学研究所 Method for generating same-layer intersection-free full-coverage nested container and readable storage medium
CN109491790B (en) * 2018-11-02 2021-08-27 中山大学 Container-based industrial Internet of things edge computing resource allocation method and system
CN111273871B (en) * 2020-01-19 2021-05-04 星辰天合(北京)数据科技有限公司 Method and device for dynamically allocating storage resources on container platform
CN112104725B (en) * 2020-09-09 2022-05-27 中国联合网络通信集团有限公司 Container mirror image duplicate removal method, system, computer equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104507111A (en) * 2014-12-26 2015-04-08 清华大学 Cluster-based cooperative communication method and device in cellular network
CN104636204A (en) * 2014-12-04 2015-05-20 中国联合网络通信集团有限公司 Task scheduling method and device

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8909602B2 (en) * 2011-02-03 2014-12-09 Vmware, Inc. Programmatic snapshot and revert of virtualized data center inventory
US8813088B2 (en) * 2011-05-26 2014-08-19 International Business Machines Corporation Scheduling flows in a multi-platform cluster environment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104636204A (en) * 2014-12-04 2015-05-20 中国联合网络通信集团有限公司 Task scheduling method and device
CN104507111A (en) * 2014-12-26 2015-04-08 清华大学 Cluster-based cooperative communication method and device in cellular network

Also Published As

Publication number Publication date
CN106095579A (en) 2016-11-09

Similar Documents

Publication Publication Date Title
CN106095579B (en) Container resource allocation methods and device
US20220209971A1 (en) Methods and apparatus to aggregate telemetry data in an edge environment
US11704144B2 (en) Creating virtual machine groups based on request
US11018979B2 (en) System and method for network slicing for service-oriented networks
US10200261B2 (en) Multiple-computing-node system job node selection
CN107196869B (en) The adaptive load balancing method, apparatus and system of Intrusion Detection based on host actual loading
CN109032801B (en) Request scheduling method, system, electronic equipment and storage medium
Chen et al. Energy-efficient fault-tolerant data storage and processing in mobile cloud
Mechalikh et al. PureEdgeSim: A simulation framework for performance evaluation of cloud, edge and mist computing environments
Zhu et al. BLOT: Bandit learning-based offloading of tasks in fog-enabled networks
US20130263151A1 (en) Consistent Hashing Table for Workload Distribution
US10621002B2 (en) Iterative task centric resource scheduling for a user program between different computing frameworks
US10027596B1 (en) Hierarchical mapping of applications, services and resources for enhanced orchestration in converged infrastructure
CN108475207A (en) The joint auto zoom of cloud application
CN109947551A (en) A kind of more round method for allocating tasks, edge calculations system and its storage medium
CN106790332B (en) Resource scheduling method, system and main node
US20230119552A1 (en) Resource management mechanisms for stateful serverless clusters in edge computing
EP4300310A1 (en) Opportunistic placement of compute in an edge network
Li et al. Resource scheduling based on improved spectral clustering algorithm in edge computing
Ferrer et al. Ad-hoc edge cloud: A framework for dynamic creation of edge computing infrastructures
Dinh et al. An energy efficient integration model for sensor cloud systems
Yin et al. Joint scheduling of data and computation in geo-distributed cloud systems
CN110532060A (en) A kind of hybrid network environmental data collecting method and system
US20210319281A1 (en) Subtask Assignment for an Artificial Intelligence Task
Fukunaga et al. Virtual machine placement for minimizing connection cost in data center networks

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant