CN117435306A - Cluster container expansion and contraction method, device, equipment and storage medium - Google Patents
Cluster container expansion and contraction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117435306A CN117435306A CN202311448954.4A CN202311448954A CN117435306A CN 117435306 A CN117435306 A CN 117435306A CN 202311448954 A CN202311448954 A CN 202311448954A CN 117435306 A CN117435306 A CN 117435306A
- Authority
- CN
- China
- Prior art keywords
- container
- resource
- cluster
- expansion
- occupation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000008602 contraction Effects 0.000 title claims description 82
- 238000012544 monitoring process Methods 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 37
- 230000015654 memory Effects 0.000 claims description 20
- 238000013507 mapping Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000013468 resource allocation Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for expanding and shrinking a cluster container, which relate to the field of computers and comprise the following steps: monitoring current performance parameters of all containers in a target cluster, inputting the current performance parameters into a pre-constructed resource prediction model to perform resource occupation prediction, obtaining occupation prediction information of all containers, and expanding and contracting the capacity of all the containers based on the occupation prediction information; according to the invention, the current performance parameters of each container in the monitored target cluster are input into the pre-constructed resource prediction model to predict the resource occupation, so that the containers are expanded and contracted based on the predicted occupation prediction information of each container, the real-time performance of load balancing is ensured, the real-time allocation of resources in various business scenes is effectively adapted, and the resource requirements of each container are met.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for expanding and shrinking a cluster container.
Background
Kubernetes (K8 s) can be used for automated deployment, expansion and contraction of subsystem containers in large-scale systems. In order to improve the utilization rate of equipment resources, K8s can be integrated and deployed in the same server or a plurality of server clusters, for each container, a technician is required to accurately define the requirements of services on material resources such as CPU, memory and the like, so that each service operates normally, otherwise, resource waste or system performance reduction can be caused, and when serious, the container service is started due to insufficient resources, so that the system is crashed. The current container expansion and contraction mode generally requires technicians to manually formulate strategies, has low resource allocation efficiency, cannot ensure the real-time performance of load balancing, and cannot effectively adapt to the real-time resource allocation requirements of real service scenes.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a method, a device, equipment and a storage medium for expanding and shrinking a cluster container, and aims to solve the technical problems that in the prior art, the resource allocation efficiency is low, the real-time performance of load balancing cannot be ensured, and the real-time allocation requirement of resources of a real service scene cannot be effectively met.
In order to achieve the above object, the present invention provides a method for expanding and shrinking a cluster container, the method comprising the following steps:
monitoring current performance parameters of all containers in the target cluster;
inputting the current performance parameters into a pre-constructed resource prediction model to perform resource occupation prediction, and obtaining occupation prediction information of each container;
and expanding and contracting the capacity of each container based on the occupancy prediction information.
Optionally, before the current performance parameter is input to a pre-constructed resource prediction model to perform resource occupation prediction, obtaining occupation prediction information of each container includes:
acquiring a resource occupation log of each container in a target cluster;
acquiring historical operation information of each container based on the resource occupation log;
modeling a training set according to the historical operation information;
and constructing a resource prediction model based on the model training set.
Optionally, the modeling training set according to the historical running information includes:
acquiring historical resource occupation data and historical time data of each container in a historical operation process based on the historical operation information;
associating the historical resource occupation data with the historical time data, and determining a mapping relation between the historical resource occupation data and the historical time data;
and constructing a model training set according to the mapping relation.
Optionally, the expanding and shrinking the containers based on the occupancy prediction information includes:
determining resource demand indexes of all containers and resource demand time corresponding to the resource demand indexes according to the occupation prediction information;
and expanding and contracting the capacity of each container based on the resource demand index and the resource demand time.
Optionally, the expanding and contracting each container based on the resource requirement index and the resource requirement time includes:
sequencing the resource demands of all the containers based on the resource demand time to obtain the resource demand sequence of all the containers;
constructing a capacity expansion sequence according to the resource demand sequence and the resource demand index;
and expanding and contracting the volume of each container based on the expansion and contraction sequence.
Optionally, the constructing a capacity expansion sequence according to the resource demand sequence and the resource demand index includes:
acquiring service types and container performance parameters of each container;
performing weight distribution on each container based on the resource demand index, the service type and the container performance parameter to obtain the expansion and contraction capacity weight corresponding to each container;
and constructing a capacity expansion sequence according to the capacity expansion weight and the resource demand sequence.
Optionally, after expanding and shrinking each container based on the occupancy prediction information, the method further includes:
monitoring the expansion and contraction result of the target cluster;
updating the model training set based on the expansion and contraction result;
and iterating the resource prediction model based on the updated model training set.
In addition, in order to achieve the above object, the present invention further provides a cluster container expansion and contraction device, where the cluster container expansion and contraction device includes:
the container monitoring module is used for monitoring the current performance parameters of each container in the target cluster;
the resource prediction module is used for inputting the current performance parameters into a pre-constructed resource prediction model to perform resource occupation prediction, so as to obtain occupation prediction information of each container;
and the expansion and contraction module is used for expanding and contracting the volume of each container based on the occupation prediction information.
In addition, in order to achieve the above object, the present invention further provides a cluster container expansion and contraction device, where the cluster container expansion and contraction device includes: a memory, a processor, and a cluster container scaling program stored on the memory and executable on the processor, the cluster container scaling program configured to implement the steps of the cluster container scaling method as described above.
In addition, to achieve the above object, the present invention further proposes a storage medium having stored thereon a cluster container scaling program which, when executed by a processor, implements the steps of the cluster container scaling method as described above.
The method comprises the steps of inputting current performance parameters of containers in a target cluster to a pre-constructed resource prediction model to perform resource occupation prediction to obtain occupation prediction information of the containers, and expanding and shrinking the containers based on the occupation prediction information; according to the invention, the current performance parameters of each container in the monitored target cluster are input into the pre-constructed resource prediction model to predict the resource occupation, so that the containers are expanded and contracted based on the predicted occupation prediction information of each container, the real-time performance of load balancing is ensured, the real-time allocation of resources in various business scenes is effectively adapted, and the resource requirements of each container are met.
Drawings
FIG. 1 is a schematic structural diagram of a cluster container expansion and contraction device of a hardware running environment according to an embodiment of the present invention;
FIG. 2 is a flow chart of a first embodiment of a method for expanding and shrinking a cluster container according to the present invention;
FIG. 3 is a flow chart of a second embodiment of the method for expanding and shrinking a cluster container according to the present invention;
FIG. 4 is a flow chart of a third embodiment of a method for expanding and shrinking a cluster container according to the present invention;
fig. 5 is a block diagram of a first embodiment of a cluster container expansion and contraction device according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a cluster container expansion and contraction device of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the cluster container expansion device may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in fig. 1 does not constitute a limitation of the cluster vessel expansion and contraction device, and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a cluster container expansion and contraction program may be included in the memory 1005 as one type of storage medium.
In the cluster container expansion and contraction device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the cluster container expansion and contraction device of the present invention may be disposed in the cluster container expansion and contraction device, where the cluster container expansion and contraction device invokes a cluster container expansion and contraction program stored in the memory 1005 through the processor 1001, and executes the cluster container expansion and contraction method provided by the embodiment of the present invention.
The embodiment of the invention provides a method for expanding and shrinking a cluster container, and referring to fig. 2, fig. 2 is a flow chart of a first embodiment of the method for expanding and shrinking the cluster container.
In this embodiment, the method for expanding and shrinking the cluster container includes the following steps:
step S10: the current performance parameters of each container in the target cluster are monitored.
It should be understood that the execution body of the method of this embodiment may be a cluster container expansion and contraction device with functions of data processing, network communication and program running, such as a computer, or other apparatuses or devices capable of implementing the same or similar functions, which is described herein by taking the above cluster container expansion and contraction device (hereinafter referred to as an expansion and contraction device) as an example.
It should be noted that the target cluster may be a Kubernetes cluster that needs to perform resource configuration and/or capacity expansion management. The current performance parameters may be performance occupation data, load data, and the like of each container in the target cluster.
It can be appreciated that the capacity expansion device of the embodiment responds to load changes more quickly by monitoring the performance index of the container in real time.
In a specific implementation, the capacity expansion device collects service type, name, performance and load data of the container. The method comprises subsystem description, CPU utilization, memory usage, network traffic, and mapping relationship data of each index and time.
Step S20: and inputting the current performance parameters into a pre-constructed resource prediction model to perform resource occupation prediction, and obtaining occupation prediction information of each container.
It should be noted that the resource prediction model may be a neural network model previously constructed based on training data, for example, the resource prediction model may be constructed based on a neural network such as a cyclic neural network (Recurrent Neural Network, RNN) and a Long Short-Term Memory (LSTM).
It can be understood that the capacity expansion and contraction device of this embodiment selects a machine learning algorithm, such as RNN, LSTM, and the like, and performs time series prediction according to the index change history data. The training data should include the container name, container resource, current time and other performance index data in the K8s cluster, so that the model learns the time sequence relation of container resource allocation, and the future resource demand is predicted according to the performance information of the current container and the container service type.
It should be understood that, in this embodiment, the performance requirement of a future period is predicted by inputting the performance data of each current container into a pre-constructed resource prediction model at regular time, so as to obtain the occupancy prediction information of each container, and the containers are configured with resources based on the occupancy prediction information of each container, so as to implement automatic expansion and contraction.
Step S30: and expanding and contracting the capacity of each container based on the occupancy prediction information.
It should be noted that, in the capacity expansion and contraction device of this embodiment, the resource occupation requirement parameters of each container in the target cluster are determined based on the occupation prediction information, a corresponding configuration file is generated based on the resource occupation requirement parameters, and the capacity expansion and contraction is performed on each container based on the configuration file.
It can be appreciated that the capacity expansion device predicts the resource occupation requirement of the container in real time by using a pre-constructed resource prediction model, monitors the resource occupation of each container in real time by using a control such as Prometheus in some embodiments, records time information, and predicts the resource requirement of the container in real time by using the resource prediction model.
In some embodiments, the capacity expansion and contraction device may perform automatic capacity expansion and contraction through a pre-edited automation script, generate a configuration file according to the resource occupation requirement information in the prediction result of the resource prediction model, and periodically run the configuration file to implement automatic capacity expansion and contraction of the container of the target cluster.
Further, in order to improve the performance of the model, after the step S30, the method further includes:
monitoring the expansion and contraction result of the target cluster;
updating the model training set based on the expansion and contraction result;
and iterating the resource prediction model based on the updated model training set.
It can be understood that the capacity expansion and contraction device continuously collects new performance data according to the running condition after the system is deployed on line by monitoring the result of the actual capacity expansion and contraction, and uses the data for iteration and optimization of the model to continuously optimize the model prediction capability.
According to the embodiment, the current performance parameters of all containers in a target cluster are monitored, the current performance parameters are input into a pre-constructed resource prediction model to perform resource occupation prediction, occupation prediction information of all the containers is obtained, and expansion and contraction of all the containers are performed based on the occupation prediction information; according to the method, the device and the system, the current performance parameters of all containers in the monitored target cluster are input into the pre-constructed resource prediction model to conduct resource occupation prediction, so that expansion and contraction of all the containers are conducted on the basis of predicted occupation prediction information of all the containers, instantaneity of load balancing is guaranteed, real-time resource allocation is effectively conducted in various business scenes, and resource requirements of all the containers are met.
Referring to fig. 3, fig. 3 is a flow chart of a second embodiment of a cluster container expansion and contraction method according to the present invention.
Based on the first embodiment, in this embodiment, before step S20, the method includes:
step S21: and obtaining the resource occupation log of each container in the target cluster.
It should be noted that the resource occupation log may be a resource occupation log file recorded with each container in the target cluster in a historical period of time. The resource occupation log may include historical resource usage of each container, high frequency time and low frequency time of historical resource usage, and the like.
Step S22: and acquiring historical operation information of each container based on the resource occupation log.
It should be noted that the historical operation information may be information related to the operation condition of each container in the historical period, for example, the historical operation information may include the resource occupation condition of each container in the historical period, and a time node, an occupation duration, and the like corresponding to the resource occupation condition
It can be understood that, in this embodiment, the operation condition of each container in the historical time period is extracted by analyzing the resource occupation log, for example, the historical time period may be 1 month, 3 months, one year, etc.
It should be understood that the capacity expansion and contraction device obtains the resource occupation data of each container in the historical time period and the historical time data corresponding to the resource occupation data based on the resource occupation log, and analyzes the historical operation condition of each container based on the resource occupation data and the historical time data.
In some embodiments, the capacity expansion device determines a resource occupancy ratio parameter of the container in a historical time period by analyzing the resource occupancy log, determines a high-frequency occupancy ratio time period and a low-frequency occupancy ratio time period of the container based on the resource occupancy ratio parameter, and analyzes the historical operation condition of the container based on the high-frequency occupancy ratio time period and the low-frequency occupancy ratio time period, wherein the high-frequency occupancy ratio time period can be a time period when the container occupies more resources, and the low-frequency occupancy ratio time period can be a time period when the container occupies less resources.
Step S23: and modeling a training set according to the historical operation information.
It can be understood that the capacity expansion and contraction device of this embodiment selects a machine learning algorithm, such as RNN, LSTM, and the like, and performs time series prediction according to the index change history data. The training data should include the container name, container resource, current time and other performance index data in the K8s cluster, so that the model learns the time sequence relation of container resource allocation, and the future resource demand is predicted according to the performance information of the current container and the container service type.
Further, in order to improve the model training efficiency and improve the model performance, the step S23 may include:
step S231: acquiring historical resource occupation data and historical time data of each container in a historical operation process based on the historical operation information;
step S232: associating the historical resource occupation data with the historical time data, and determining a mapping relation between the historical resource occupation data and the historical time data;
step S233: and constructing a model training set according to the mapping relation.
It should be noted that, the capacity expansion and contraction device discovers the mode, rule and trend of the container operation from a large amount of data through model prediction, and then applies the knowledge to new data to perform tasks such as prediction, classification optimization and the like.
As a data support for a resource prediction model, a large amount of historical performance data of a corresponding container is collected first to predict the change rule of the container performance and time in a day. The performance data generally includes CPU utilization, CPU Load, memory usage, cache/buffer usage, total memory usage, and disk and network read/write conditions, such as container disk space usage, disk reading, writing speed, network downstream and upstream traffic, and the like, and corresponding moments. Data attributes may be added according to service class or device type to increase prediction rationality.
It can be understood that the resource prediction model has a certain performance parameter prediction capability after learning a large amount of data, and can predict the change condition of a certain attribute (such as CPU and memory) according to the current performance and time data.
Step S24: and constructing a resource prediction model based on the model training set.
It should be understood that, in this embodiment, the performance requirement of a future period is predicted by inputting the performance data of each current container into a pre-constructed resource prediction model at regular time, so as to obtain the occupancy prediction information of each container, and the containers are configured with resources based on the occupancy prediction information of each container, so as to implement automatic expansion and contraction.
It will be appreciated that the present embodiment may be implemented by collecting service type, name, performance and load data for the container. The method comprises subsystem description, CPU utilization rate, memory usage amount and network flow, mapping relation data of each index and time, arranging the data into a format which can be used for machine learning model training, constructing a model training set based on the arranged data, carrying out model training based on the model training set, and constructing a resource prediction model.
According to the embodiment, the resource occupation log of each container in the target cluster is obtained, the historical operation information of each container is obtained based on the resource occupation log, a model training set is modeled according to the historical operation information, and a resource prediction model is built based on the model training set; according to the method, the device and the system, the historical operation information of each container is extracted by acquiring the resource occupation log of each container in the target cluster, the training set is constructed based on the historical operation information, model training is carried out based on the training set, and the resource prediction model is constructed, so that the resource demand prediction efficiency is improved, and the resource occupation demand of each container in a future time period is accurately predicted.
Referring to fig. 4, fig. 4 is a flow chart of a third embodiment of a cluster container expansion and contraction method according to the present invention.
Based on the first embodiment, in this embodiment, the step S40 includes:
step S41: determining resource demand indexes of all containers and resource demand time corresponding to the resource demand indexes according to the occupation prediction information;
step S42: and expanding and contracting the capacity of each container based on the resource demand index and the resource demand time.
It should be noted that the resource requirement index may be a parameter index of a resource requirement of a container in a future time period, for example, the resource requirement index may be a CPU resource that a certain container needs to occupy 80% of the resources at 18 hours and 20 minutes, and the resource requirement time may be a time point and a duration that the container occupies resources.
It can be understood that the expansion and contraction device predicts the resource demand index of each container at the future time through the resource prediction model, and predicts the demand time length and the demand time point corresponding to the resource demand index, so that the resource allocation is performed at the demand time point of each container based on the resource demand index and the demand time length, the resource demand of each container is ensured to be satisfied, and the load change of each container is effectively responded.
Further, in order to meet the resource requirement of the container in real time and respond to the load change condition of the container quickly, the step S42 may include:
step S421: sequencing the resource demands of all the containers based on the resource demand time to obtain the resource demand sequence of all the containers;
step S422: constructing a capacity expansion sequence according to the resource demand sequence and the resource demand index;
step S423: and expanding and contracting the volume of each container based on the expansion and contraction sequence.
It can be understood that, in order to meet the resource requirements of the containers in different periods in the future and ensure the timeliness of the expansion and contraction of the containers, the expansion device of this embodiment sorts the resource requirement configuration sequence of each container based on the resource requirement time of each container, obtains the resource requirement sequence of each container, and constructs the expansion and contraction sequence based on the resource requirement sequence and the resource requirement index, thereby ensuring the orderly progress of the expansion and contraction to meet the resource requirements of the containers in different periods.
Further, in order to accurately construct the expansion and contraction sequence, the step S422 may include:
step S4221: acquiring service types and container performance parameters of each container;
step S4222: performing weight distribution on each container based on the resource demand index, the service type and the container performance parameter to obtain the expansion and contraction capacity weight corresponding to each container;
step S4223: and constructing a capacity expansion sequence according to the capacity expansion weight and the resource demand sequence.
It is understood that the capacity expansion device gathers service type, name, performance and load data for the container. The method comprises subsystem description, CPU utilization rate, memory usage amount and network flow, wherein importance degree evaluation is carried out on each container based on container service type and performance parameters, weight distribution is carried out on each container based on evaluation results, expansion and contraction capacity weights corresponding to each container are obtained, and expansion and contraction capacity sequences are built according to the expansion and contraction capacity weights and the resource demand sequence.
According to the embodiment, the resource demand index of each container and the resource demand time corresponding to the resource demand index are determined according to the occupation prediction information, and the expansion and contraction of each container are carried out based on the resource demand index and the resource demand time; according to the method, the device and the system, the containers are expanded and contracted according to the resource demand indexes and the resource demand time of the containers, which are obtained through model prediction, so that the containers are orderly allocated with resources, the containers are ensured to be satisfied with the resource demands in different time periods, and the load change of the containers is responded more quickly.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium is stored with a cluster container expansion and contraction program, and the cluster container expansion and contraction program realizes the steps of the cluster container expansion and contraction method when being executed by a processor.
Because the storage medium adopts all the technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are not described in detail herein.
Referring to fig. 5, fig. 5 is a block diagram of a first embodiment of a cluster container expansion and contraction device according to the present invention.
As shown in fig. 5, a cluster container expansion and contraction device according to an embodiment of the present invention includes:
a container monitoring module 10 for monitoring current performance parameters of each container in the target cluster;
the resource prediction module 20 is configured to input the current performance parameter to a pre-constructed resource prediction model to perform resource occupation prediction, so as to obtain occupation prediction information of each container;
and the expansion and contraction module 30 is used for expanding and contracting each container based on the occupancy prediction information.
Further, the cluster container expansion and contraction device further comprises:
a model training module 40, configured to obtain a resource occupation log of each container in the target cluster; acquiring historical operation information of each container based on the resource occupation log; modeling a training set according to the historical operation information; and constructing a resource prediction model based on the model training set.
Further, the model training module 40 is further configured to obtain historical resource occupation data and historical time data of each container in a historical operation process based on the historical operation information; associating the historical resource occupation data with the historical time data, and determining a mapping relation between the historical resource occupation data and the historical time data; and constructing a model training set according to the mapping relation.
Further, the expansion and contraction module 30 is further configured to determine a resource requirement index of each container and a resource requirement time corresponding to the resource requirement index according to the occupancy prediction information; and expanding and contracting the capacity of each container based on the resource demand index and the resource demand time.
Further, the expansion and contraction module 30 is further configured to sort the resource requirements of each container based on the resource requirement time, so as to obtain a resource requirement sequence of each container; constructing a capacity expansion sequence according to the resource demand sequence and the resource demand index; and expanding and contracting the volume of each container based on the expansion and contraction sequence.
Further, the expansion and contraction module 30 is further configured to obtain a service type and a container performance parameter of each container; performing weight distribution on each container based on the resource demand index, the service type and the container performance parameter to obtain the expansion and contraction capacity weight corresponding to each container; and constructing a capacity expansion sequence according to the capacity expansion weight and the resource demand sequence.
Further, the cluster container expansion and contraction device further comprises:
the model iteration module 50 is used for monitoring the expansion and contraction result of the target cluster; updating the model training set based on the expansion and contraction result; and iterating the resource prediction model based on the updated model training set.
According to the embodiment, the current performance parameters of all containers in a target cluster are monitored, the current performance parameters are input into a pre-constructed resource prediction model to perform resource occupation prediction, occupation prediction information of all the containers is obtained, and expansion and contraction of all the containers are performed based on the occupation prediction information; according to the method, the device and the system, the current performance parameters of all containers in the monitored target cluster are input into the pre-constructed resource prediction model to conduct resource occupation prediction, so that expansion and contraction of all the containers are conducted on the basis of predicted occupation prediction information of all the containers, instantaneity of load balancing is guaranteed, real-time resource allocation is effectively conducted in various business scenes, and resource requirements of all the containers are met.
It should be understood that the foregoing is illustrative only and is not limiting, and that in specific applications, those skilled in the art may set the invention as desired, and the invention is not limited thereto.
It should be noted that the above-described working procedure is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select part or all of them according to actual needs to achieve the purpose of the embodiment, which is not limited herein.
In addition, technical details not described in detail in this embodiment may refer to the method for expanding and shrinking the cluster container provided in any embodiment of the present invention, which is not described herein again.
Furthermore, it should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. Read Only Memory)/RAM, magnetic disk, optical disk) and including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. The cluster container expansion and contraction method is characterized by comprising the following steps of:
monitoring current performance parameters of all containers in the target cluster;
inputting the current performance parameters into a pre-constructed resource prediction model to perform resource occupation prediction, and obtaining occupation prediction information of each container;
and expanding and contracting the capacity of each container based on the occupancy prediction information.
2. The method for expanding and shrinking the cluster containers according to claim 1, wherein the step of inputting the current performance parameter into a pre-constructed resource prediction model to perform resource occupancy prediction, and before obtaining occupancy prediction information of each container, comprises the steps of:
acquiring a resource occupation log of each container in a target cluster;
acquiring historical operation information of each container based on the resource occupation log;
modeling a training set according to the historical operation information;
and constructing a resource prediction model based on the model training set.
3. The cluster container scaling method of claim 2, wherein modeling a training set based on the historical operating information, comprises:
acquiring historical resource occupation data and historical time data of each container in a historical operation process based on the historical operation information;
associating the historical resource occupation data with the historical time data, and determining a mapping relation between the historical resource occupation data and the historical time data;
and constructing a model training set according to the mapping relation.
4. The cluster container scaling method of claim 1, wherein scaling each container based on the occupancy prediction information comprises:
determining resource demand indexes of all containers and resource demand time corresponding to the resource demand indexes according to the occupation prediction information;
and expanding and contracting the capacity of each container based on the resource demand index and the resource demand time.
5. The method for scaling up and scaling down the cluster container according to claim 4, wherein scaling up and scaling down each container based on the resource demand index and the resource demand time comprises:
sequencing the resource demands of all the containers based on the resource demand time to obtain the resource demand sequence of all the containers;
constructing a capacity expansion sequence according to the resource demand sequence and the resource demand index;
and expanding and contracting the volume of each container based on the expansion and contraction sequence.
6. The method for scaling up and scaling down a cluster container according to claim 5, wherein said constructing a scaling up and scaling down sequence according to said resource demand sequence and said resource demand index comprises:
acquiring service types and container performance parameters of each container;
performing weight distribution on each container based on the resource demand index, the service type and the container performance parameter to obtain the expansion and contraction capacity weight corresponding to each container;
and constructing a capacity expansion sequence according to the capacity expansion weight and the resource demand sequence.
7. The cluster container scaling method of any one of claims 1 to 6, further comprising, after scaling each container based on the occupancy prediction information:
monitoring the expansion and contraction result of the target cluster;
updating the model training set based on the expansion and contraction result;
and iterating the resource prediction model based on the updated model training set.
8. The utility model provides a cluster container expands and contracts appearance device which characterized in that, cluster container expands and contracts appearance device includes:
the container monitoring module is used for monitoring the current performance parameters of each container in the target cluster;
the resource prediction module is used for inputting the current performance parameters into a pre-constructed resource prediction model to perform resource occupation prediction, so as to obtain occupation prediction information of each container;
and the expansion and contraction module is used for expanding and contracting the volume of each container based on the occupation prediction information.
9. A cluster container expansion and contraction device, characterized in that the cluster container expansion and contraction device comprises: a memory, a processor, and a cluster container scaling program stored on the memory and executable on the processor, the cluster container scaling program configured to implement the cluster container scaling method of any one of claims 1-7.
10. A storage medium, wherein a cluster container scaling program is stored on the storage medium, and the cluster container scaling program realizes the cluster container scaling method according to any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311448954.4A CN117435306A (en) | 2023-11-01 | 2023-11-01 | Cluster container expansion and contraction method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311448954.4A CN117435306A (en) | 2023-11-01 | 2023-11-01 | Cluster container expansion and contraction method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117435306A true CN117435306A (en) | 2024-01-23 |
Family
ID=89547768
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311448954.4A Pending CN117435306A (en) | 2023-11-01 | 2023-11-01 | Cluster container expansion and contraction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117435306A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117648173A (en) * | 2024-01-26 | 2024-03-05 | 杭州阿里云飞天信息技术有限公司 | Resource scheduling method and device |
| CN118051347A (en) * | 2024-04-16 | 2024-05-17 | 杭州冰特科技股份有限公司 | Server cluster management method and system |
-
2023
- 2023-11-01 CN CN202311448954.4A patent/CN117435306A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117648173A (en) * | 2024-01-26 | 2024-03-05 | 杭州阿里云飞天信息技术有限公司 | Resource scheduling method and device |
| CN117648173B (en) * | 2024-01-26 | 2024-05-14 | 杭州阿里云飞天信息技术有限公司 | Resource scheduling method and device |
| CN118051347A (en) * | 2024-04-16 | 2024-05-17 | 杭州冰特科技股份有限公司 | Server cluster management method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112000459B (en) | Method for expanding and shrinking capacity of service and related equipment | |
| CN117435306A (en) | Cluster container expansion and contraction method, device, equipment and storage medium | |
| CN1956457B (en) | Method and apparatus for arranging mesh work in mesh computing system | |
| US7752239B2 (en) | Risk-modulated proactive data migration for maximizing utility in storage systems | |
| CN111258767A (en) | Intelligent cloud computing resource allocation method and device for complex system simulation application | |
| CN112685170A (en) | Dynamic optimization of backup strategies | |
| CN104298550A (en) | Hadoop-oriented dynamic scheduling method | |
| CN114356587A (en) | Calculation power task cross-region scheduling method, system and equipment | |
| CN114661482B (en) | GPU (graphics processing Unit) computing power management method, medium, equipment and system | |
| US11651271B1 (en) | Artificial intelligence system incorporating automatic model updates based on change point detection using likelihood ratios | |
| CN117271100B (en) | Algorithm chip cluster scheduling method, device, computer equipment and storage medium | |
| CN117332970A (en) | Planning task management method, device, computer equipment and storage medium | |
| CN117290090A (en) | Node cooperative resource allocation method, device, equipment and storage medium | |
| US11636377B1 (en) | Artificial intelligence system incorporating automatic model updates based on change point detection using time series decomposing and clustering | |
| Ismaeel et al. | Real-time energy-conserving vm-provisioning framework for cloud-data centers | |
| CN111783487A (en) | Fault early warning method and device for card reader equipment | |
| Tuli et al. | Optimizing the performance of fog computing environments using ai and co-simulation | |
| US11782923B2 (en) | Optimizing breakeven points for enhancing system performance | |
| CN115525394A (en) | Method and device for adjusting number of containers | |
| CN112257977B (en) | Logistics project construction period optimization method and system with resource limitation under fuzzy man-hour | |
| Chahal et al. | iSeSA: Towards Migrating HPC and AI Workloads to Serverless Platform | |
| Adeyelu et al. | A model for coordinating jobs on mobile wireless computational grids | |
| Shrivastava et al. | A predictive energy saving technique for 5G network base stations | |
| Tournaire et al. | Reinforcement learning with model-based approaches for dynamic resource allocation in a tandem queue | |
| CN117112213A (en) | Resource scheduling method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |