CN116389491A - Cloud edge computing power resource self-adaptive computing system - Google Patents
Cloud edge computing power resource self-adaptive computing system Download PDFInfo
- Publication number
- CN116389491A CN116389491A CN202310344726.6A CN202310344726A CN116389491A CN 116389491 A CN116389491 A CN 116389491A CN 202310344726 A CN202310344726 A CN 202310344726A CN 116389491 A CN116389491 A CN 116389491A
- Authority
- CN
- China
- Prior art keywords
- resource
- cloud
- adaptive
- node
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002068 genetic effect Effects 0.000 claims abstract description 12
- 238000012544 monitoring process Methods 0.000 claims abstract description 11
- 230000008447 perception Effects 0.000 claims abstract description 9
- 230000006855 networking Effects 0.000 claims abstract description 8
- 230000001133 acceleration Effects 0.000 claims abstract description 7
- 230000003044 adaptive effect Effects 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000013507 mapping Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000005265 energy consumption Methods 0.000 claims description 3
- 238000011065 in-situ storage Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 25
- 230000008901 benefit Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000000034 method Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000013209 evaluation strategy Methods 0.000 description 1
- 230000007614 genetic variation Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Genetics & Genomics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Physiology (AREA)
- Supply And Distribution Of Alternating Current (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a cloud edge computing power resource self-adaptive computing system, which relates to the technical field of cloud edge computing power resource computing, wherein a resource network containerization management module comprises a hardware resource abstraction layer, a resource container networking layer and a resource cooperative supporting layer, and provides resource perception support, task arrangement support, heterogeneous resource acceleration support and mixed load balancing support for the system; the resource intelligent sensing and discovery module is used for realizing node registration, state sensing and resource monitoring based on the resource networked container management module to form a cloud edge node resource pool, and dynamically updating the distribution conditions of the edge and cloud resources; after receiving the collaborative computing request, the resource self-adaptive collaborative scheduling and flexible scheduling module estimates the required resources based on an improved genetic algorithm, and performs task scheduling and allocation by taking the side and cloud resource distribution conditions as resource constraints, so as to realize resource self-adaptive collaboration.
Description
Technical Field
The invention relates to the technical field of cloud edge computing power resource calculation, in particular to a cloud edge computing power resource self-adaptive calculation system.
Background
The cloud edge resource collaboration has high resource isomerism and large technology generation difference, and lacks a standardized abstract access frame; the edge resources are limited, cloud resources are relatively excessive, and the global resources cannot be uniformly scheduled; the cloud edge network environment has large difference, and can not meet the requirements of high real-time collaborative computing service on network service quality. In order to maximize benefits, the cloud data center generally divides cloud services to be calculated into a plurality of mutually independent sub-modules, maps virtual computing resources on the virtual server to each sub-module respectively, and outputs virtual cloud computing values as a whole after each sub-module responds to virtual computing. However, the virtual computing resource differentiation caused by the difference feature of the computing performance of each physical server in the cloud data center environment results in different cloud computing time costs of the cloud data center acceptance submodule. And the loss cost caused by calculating one sub-module by mapping virtual resources of the virtual server based on the difference characteristic in a specific time is different. Clearly, the idea behind the cloud computing provider's good yields is to have the virtual server provide virtual computing resources for multiple unrelated sub-modules simultaneously.
The related mainstream researches are based on genetic algorithm advantages, so that timeliness of cloud task calculation and resource balance are solved only on one side even if improvement is made, related discussion is not developed even for the situation of early convergence, and benefits of cloud computing providers are difficult to be improved substantially.
Therefore, the cloud edge computing power resource self-adaptive computing system is provided, resource networked container management, resource intelligent perception and discovery, resource self-adaptive collaborative arrangement and flexible scheduling and network intelligent routing are constructed, edge resource constraint is broken, cloud edge resource configuration is optimized, and the problem to be solved by the global resource collaborative scheduling is solved by the person skilled in the art.
Disclosure of Invention
In view of the above, the invention provides a cloud edge computing power resource adaptive computing system, which breaks edge resource constraint, optimizes cloud edge resource allocation and realizes global resource cooperative scheduling, and in order to achieve the above purpose, the invention adopts the following technical scheme:
a cloud computing power resource adaptive computing system, comprising: the system comprises a resource network containerization management module, a resource intelligent perception and discovery module and a resource self-adaptive collaborative arrangement and flexible scheduling module, wherein:
the resource network containerization management module comprises a hardware resource abstraction layer, a resource container networking layer and a resource cooperative supporting layer, and provides resource perception support, task arrangement support, heterogeneous resource acceleration support and hybrid load balancing support for a system; the resource intelligent sensing and discovery module is used for realizing node registration, state sensing and resource monitoring based on the resource networked container management module to form a cloud edge node resource pool, and dynamically updating the distribution conditions of the edge and cloud resources; after receiving the collaborative computing request, the resource self-adaptive collaborative scheduling and flexible scheduling module estimates the required resources based on an improved genetic algorithm, and performs task scheduling and allocation by taking the side and cloud resource distribution conditions as resource constraints, so as to realize resource self-adaptive collaboration.
Optionally, the estimating the required resources based on the improved genetic algorithm includes constructing a cost function to be responded by the cloud edge service, respectively representing t and p as time cost and loss cost required by the virtual server v to execute the sub-module m, and adding p to the virtual server v v Individual cost characterized by v, loss cost is noted as p=tp v ；
The sub-module scale of the cloud edge service is x, and the data length of the sub-module m in the c-th cloud edge service is t=L cm /C v The scale of the virtual server is y, and the virtual computing capacity provided by the v-th virtual server is C v M is more than or equal to 1 and less than or equal to x, v is more than or equal to 1 and less than or equal to y, and the cost of the virtual server for bearing cloud edge service is p vt The bearing capacity of the virtual server v is z v Bearing time delay t of cloud edge service on virtual server mv Is L cm /z v Cloud edge service acceptance duration is as follows
The cloud edge service to-be-responded cost function is +.>
Optionally, the loss cost universality objective function value calculated by the cloud edge service is accepted and inversely proportional to the loss cost of the population individuals, and the loss cost universality objective function value is
The maximum duration of the cloud edge service accepted by the system is T H The time cost universality objective function for accepting cloud edge service calculation is as follows:
wherein (1)>
And representing the degree of the virtual computing resource imposed by the cloud edge service to be computed.
Optionally, the specific steps of estimating the required resources based on the improved genetic algorithm are as follows: the resource self-adaptive collaborative arrangement and flexible scheduling module randomly generates N primary individual populations and analyzes whether the iterative calculation frequency of an algorithm reaches a limit, and if so, the optimal value is screened out in situ; if the limit is not reached, the function is referred to
And->
The first 3% of individuals with larger universality value are eliminated, then 5% of the individuals following the first 3% are selected as individuals of the next generation population, the rest individuals are screened by adopting a polling algorithm, and the highest-quality individuals are extracted as next generation.
Optionally, the hardware resource abstraction layer is based on an Overlay mapping tool and a virtualization framework mapping tool to realize the programmable logic architecture of heterogeneous resources and the virtualization of heterogeneous resources; the resource network containerization layer is based on a cloud container network frame of Kubernetes+Docker, and the containerization capability of Docker is utilized to realize the resource containerization processing of abstract hardware at the cloud end and the side end; the resource collaboration support layer provides container management capabilities including container management, container orchestration, API gateway services, container load balancing, and resource monitoring, and provides support for resource awareness and resource orchestration scheduling.
Optionally, the resource intelligent sensing and discovery module realizes state sensing or actively requests resource information to cloud edge nodes based on timing state updating, including node type, total resource amount, resource utilization rate, task running condition and task occupied resources;
when a node is requested for a resource as a target node without any response or the node requests a logoff from the resource adaptive orchestration and flexible scheduling module, the system removes the node from the resource pool.
Optionally, the task scheduling and allocation are performed by using the side and cloud resource distribution conditions as resource constraints, and the specific steps for realizing resource self-adaptive coordination are as follows:
after receiving the request, the resource self-adaptive collaborative scheduling and flexible scheduling module takes dynamic distribution information of a node resource management pool as resource constraint, takes time delay, deadline, energy consumption, characteristics and user experience of a task as an allocation target, solves single task allocation, multi-task allocation and node task processing priority, divides the task according to the task and resource characteristics and allocates the task to a target node, generates a resource reservation queue, a data task queue, a calculation task queue and an I/O task queue, determines task unloading time and unloading strategy, and applies resources to the target node.
Optionally, the network intelligent route is automatically deployed at the edge and the cloud, and perceives data such as delay, packet loss rate and the like of a network link in real time, and when the network link is abnormal, an optimal route is selected preferentially.
Optionally, the cloud computing power self-adaptive computing system comprises an edge and cloud interface module, wherein the edge and cloud interface module provides a calling interface, and the third-party service platform can be connected into the cloud computing power self-adaptive computing system through the interface.
The system comprises a standard resource cooperative call interface, wherein the standard resource cooperative call interface comprises a node registration interface, a node cancellation interface, a node state report interface, a node resource report interface, a node state query interface, a node resource request interface and a node task report interface.
Compared with the prior art, the cloud edge computing power resource self-adaptive computing system provided by the invention has the following beneficial effects:
the resource networked container management module provides unified hardware abstraction capability, networked container management capability, and upper layer application can use cloud and side heterogeneous resources in a mode of point access, one-time adaptation and consistent experience, so that upper layer development difficulty is simplified, and better compatibility is realized; the resource intelligent sensing and discovery module provides standardized cloud and side resource discovery, access and monitoring capabilities, senses the health condition, resource utilization rate and task running condition of the resource node in real time, forms an intelligent resource pool and provides a basis for resource self-adaptive collaborative arrangement; the resource self-adaptive collaborative arrangement and flexible scheduling module integrates the edge and cloud resources, and supports global resource scheduling, so that the resources can be dynamically switched according to the resource requirements of data, models and applications or flexible self-defined strategies, the node resource characteristics are combined, the heterogeneous resource collaborative acceleration technology is based, the limitation of the edge resources is overcome, and the resource utilization rate is effectively improved.
According to the invention, the genetic quality of the offspring individuals is improved according to the universality evaluation mechanism, the winner and winner eliminating mechanism and the preferential variation mechanism, so that the universality function value of the offspring individuals has good global property. When virtual cloud computing of mass services is deployed, the loss cost of a cloud data center can be effectively reduced, and the time cost of cloud computing is reduced. And through the planning benefit model, the cloud data center environment is involved in the procedural management of mass cloud service calculation from the cost and cost perspective. A series of evaluation strategies such as universality are introduced on the basis of fully using a classical algorithm to screen the accuracy of virtual cloud computing, and meanwhile, the complexity of the virtual cloud computing is continuously reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram provided by the present invention.
Fig. 2 is a schematic diagram of a resource networking container management module structure provided by the present invention.
Fig. 3 is a schematic diagram of a resource intelligent sensing and discovery module structure provided by the invention.
Fig. 4 is a schematic structural diagram of a resource adaptive collaborative scheduling and flexible scheduling module provided by the invention.
Fig. 5 is a schematic diagram of a network intelligent routing structure provided by the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention discloses a cloud edge computing power resource self-adaptive computing system, which comprises the following components: the system comprises a resource network containerization management module, a resource intelligent perception and discovery module and a resource self-adaptive collaborative arrangement and flexible scheduling module, wherein:
the resource network containerization management module comprises a hardware resource abstraction layer, a resource container networking layer and a resource cooperative supporting layer, and provides resource perception support, task arrangement support, heterogeneous resource acceleration support and hybrid load balancing support for a system; the resource intelligent sensing and discovery module is used for realizing node registration, state sensing and resource monitoring based on the resource networked container management module to form a cloud edge node resource pool, and dynamically updating the distribution conditions of the edge and cloud resources; after receiving the collaborative computing request, the resource self-adaptive collaborative scheduling and flexible scheduling module estimates the required resources based on an improved genetic algorithm, and performs task scheduling and allocation by taking the side and cloud resource distribution conditions as resource constraints, so as to realize resource self-adaptive collaboration.
Further, in embodiment 1, as shown in fig. 1, a cloud computing power resource adaptive computing system includes functions of cloud computing system for managing construction resource network, intelligent sensing and finding of resources, adaptive collaborative scheduling and flexible scheduling of resources, intelligent routing of network, etc., providing resource collaborative scheduling service for cloud and side system platforms, and providing adaptive resource support for upper layer data collaboration, model collaboration and service collaboration;
the resource networking container management provides unified hardware abstraction capability, the networking container management capability, and the upper layer application can use cloud and side heterogeneous resources in a mode of point access, one-time adaptation and consistent experience, so that the development difficulty of the upper layer is simplified, and better compatibility is realized;
the resource intelligent sensing and discovery provides standardized cloud and side resource discovery, access and monitoring capabilities, and senses the health condition, resource utilization rate and task running condition of resource nodes in real time to form an intelligent resource pool, thereby providing a basis for resource self-adaptive collaborative arrangement;
the edge and cloud resources are integrated through resource self-adaptive collaborative arrangement and flexible scheduling, global resource scheduling is supported, so that resources can be dynamically switched according to the resource requirements of data, models and applications or flexible self-defined strategies, the node resource characteristics are combined, heterogeneous resource collaborative acceleration technology is based, limitation of edge resources is overcome, and the resource utilization rate is effectively improved.
Further, in embodiment 2, a resource-networked container management module, as shown in fig. 2, includes a hardware resource abstraction layer, a resource-container networking layer, and a resource collaboration support layer. The hardware resource abstraction layer is based on an Overlay mapping tool and a virtualization framework mapping tool, and realizes the programmable logic architecture of heterogeneous resources and heterogeneous resource virtualization. The resource network containerization layer is based on a cloud container network frame of Kubernetes+Docker, and the containerization capability of Docker is utilized to realize the resource containerization processing of abstract hardware of a cloud end and a side end, so that a cloud container network for unified management of cloud side containers is constructed;
based on hardware virtualization and network containerization of cloud edge heterogeneous resources, a resource cooperative support layer provides container management capability: for example, container management, container arrangement, API gateway service, container load balancing, resource monitoring and the like, the hardware resource development efficiency is effectively improved, the resource logic utilization rate is improved, the large-scale distributed deployment and operation and maintenance of resources are facilitated, the consistency experience of cloud edge resource management is optimized, and support is provided for resource perception and resource arrangement and scheduling.
Further, in embodiment 3, as shown in fig. 3, the resource intelligent sensing and discovery module is based on resource networked container management, so as to realize node registration, state sensing and resource monitoring, form a cloud edge node resource pool, dynamically update the side and cloud resource distribution conditions, and provide basis for resource self-adaptive collaborative scheduling and flexible scheduling;
the cloud side and cloud side nodes register nodes to the cloud side intelligent cooperative system based on the unified container monitoring agent interface, and after the system performs list comparison, the node related information is newly added or updated to a resource pool;
the system realizes state sensing based on the timing state update, and the system can also actively request resource information to cloud edge nodes, including node types, total resource amount, resource utilization rate, task running conditions and resources occupied by tasks. In order to save network bandwidth in the interaction process of the cloud and side systems and the collaborative system, the system supports a user-defined state updating strategy. In an extreme case, the supporting cloud edge node updates the node state sporadically, and when the node is requested for resources as a target node without any response, or the node requests a log-off from the collaborative system, the system removes the node from the resource pool.
Further, in embodiment 4, a resource adaptive collaborative scheduling and flexible scheduling module, as shown in fig. 4, performs resource adaptive collaborative scheduling based on task scheduling and allocation under resource constraint;
and after receiving the collaborative computing requests of data collaboration, model collaboration and service collaboration, the intelligent collaborative service layer estimates resources required by data operation, model training operation and service operation and requests the resources to the networked container resource management layer. Meanwhile, part of cloud end and side end nodes can also directly request resources from a resource management layer;
the specific steps for estimating the required resources based on the improved genetic algorithm are as follows: the estimating of the required resources based on the improved genetic algorithm comprises constructing a cloud edge service to-be-responded cost function, respectively representing t and p as time cost and loss cost required by the virtual server v to execute the sub-module m, and p v Individual cost characterized by v, loss cost is noted as p=tp v ；
The sub-module scale of the cloud edge service is x, and the data length of the sub-module m in the c-th cloud edge service is L cm ，t＝L cm /C v The scale of the virtual server is y, and the virtual computing capacity provided by the v-th virtual server is C v M is not less than 1 and not more than x, v is not less than 1 and not more than y, and time cost t=L cm /C v The cost of the virtual server for bearing cloud edge service is p vt The bearing time delay is t cv The bearing capacity of the virtual server v is z v Bearing time delay t of cloud edge service on virtual server mv Is L cm /z v Cloud edge service acceptance duration is as follows
The cloud edge service to-be-responded cost function is +.>
Accepting the inverse proportion of the loss cost universality objective function value calculated by cloud edge service and the individual loss cost of the population, wherein the loss cost universality objective function value
The maximum duration of the cloud edge service accepted by the system is T H The time cost universality objective function for accepting cloud edge service calculation is as follows:
wherein (1)>
Representing the degree of virtual computing resources for the cloud edge service to be computed;
the resource self-adaptive collaborative arrangement and flexible scheduling module randomly generates N primary individual populations and analyzes whether the iterative calculation frequency of an algorithm reaches a limit, and if so, the optimal value is screened out in situ; if the limit is not reached, the function is referred to
And->
Eliminating the first 3% of individuals with larger universality value, selecting the next 5% of individuals as the next generation population individuals, screening the rest individuals by adopting a polling algorithm, and extracting the highest quality individuals as next generation;
the aim of crossing targets is to enable any two individuals in the population to obtain new generations through transposition. For extracted individual reference functions
And o jx ＝o 1 Randomly cross-over and re-reference function
And o by ＝o 3 Performing genetic variation wherein the population of two individuals is universally used for A 1 、A 2 、A m Average universality and maximum universality are A and A respectively H The individual normal crossing rate is set to o 1 、o 3 The individual adaptive crossing rate is set to o 2 、o 4 The probability of two individual startup crosses is noted o jx . Thereby generating brand new individuals with higher quality and truly realizing the superior and inferior elimination. Finally, the two parameter convergence thresholds p are compared th And->
And verifying whether the selected individuals meet the benefit requirements of the cloud computing service provider or not according to the relation of the values, if not, continuing to analyze whether the iterative computation frequency of the algorithm reaches the limit or not until the global optimal solution is searched and then outputting.
Heterogeneous acceleration comprises GPU, ASIC, FPGA, CPU, DSP and other high-performance processors, computing resources are selected according to model complexity, high-order operation, numerical symbol operation and iterative operation are calculated, and the solving process is accelerated;
after receiving the request, the module takes dynamic distribution information of the node resource management pool as resource constraint, takes time delay, deadline, energy consumption, characteristics and user experience of the task as distribution targets, solves single task distribution, multi-task distribution and node task processing priority, segments tasks according to the task and resource characteristics and distributes the tasks to target nodes, generates a resource reservation queue, a data task queue, a calculation task queue and an I/O task queue, determines task unloading time and unloading strategy, and applies resources to the target nodes. The cloud end and the side end abstract nodes can feed back task application states to the collaborative system according to the use condition of the resources;
the resource self-adaptive collaborative scheduling and flexible scheduling module provides cloud edge resource scheduling capability, and multi-node deployment and dynamic switching of the application are realized.
Further, the resource self-adaptive collaborative arrangement and flexible scheduling module receives resource requests of terminals such as cloud edge data real-time collaborative transmission and calculation interaction systems, mechanism model on-demand cloud edge collaborative training systems, cloud edge-crossing integrated collaborative service standard control consoles and the like.
Further, in embodiment 5, as shown in fig. 5, a cloud-edge intelligent cooperative system automatically monitors network links of an accessed edge and a cloud end, and automatically deploys gateway agents at the edge and the cloud end to realize real-time sensing of data such as delay, packet loss rate and the like of the network links, and when the network links are abnormal, an optimal route is selected preferentially, cloud-edge communication is ensured, and meanwhile, network link states are synchronized with a data scheduler and a model scheduler to realize dynamic adjustment of data and model transmission topology.
Furthermore, in embodiment 6, a resource interface of a cloud end and a cloud end provides standard resource collaborative call interfaces such as node registration, node cancellation, node status reporting, node resource reporting, node task reporting and the like, and a third party service platform can intelligently and collaborative manage registered resource nodes to a cloud edge through the interfaces and synchronize node resource status to a collaborative system so as to realize dynamic sensing and discovery of resources; meanwhile, the system provides standard interfaces such as node state inquiry, node resource request and the like, and combines a resource self-adaptive collaborative scheduling module to realize intelligent collaboration of resources.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A cloud computing power resource adaptive computing system, comprising: the system comprises a resource network containerization management module, a resource intelligent perception and discovery module and a resource self-adaptive collaborative arrangement and flexible scheduling module, wherein:
the resource network containerization management module comprises a hardware resource abstraction layer, a resource container networking layer and a resource cooperative supporting layer, and provides resource perception support, task arrangement support, heterogeneous resource acceleration support and hybrid load balancing support for a system; the resource intelligent sensing and discovery module is used for realizing node registration, state sensing and resource monitoring based on the resource networked container management module to form a cloud edge node resource pool, and dynamically updating the distribution conditions of the edge and cloud resources; after receiving the collaborative computing request, the resource self-adaptive collaborative scheduling and flexible scheduling module estimates the required resources based on an improved genetic algorithm, and performs task scheduling and allocation by taking the side and cloud resource distribution conditions as resource constraints, so as to realize resource self-adaptive collaboration.
2. The cloud computing power resource adaptive computing system of claim 1, wherein the estimating of the required resources based on the improved genetic algorithm comprises constructing a cloud-edge business to-be-responded cost function, representing t and p as time cost and loss cost required by the virtual server v to execute the sub-module m, respectively, and p v Individual cost characterized by v, loss cost is noted as p=tp v ；
The sub-module scale of the cloud edge service is x, and the data length of the sub-module m in the c-th cloud edge service is L cm ，t＝L cm /C v The scale of the virtual server is y, and the virtual computing capacity provided by the v-th virtual server is C v M is not less than 1 and not more than x, v is not less than 1 and not more than y, and time cost t=L cm /C v The cost of the virtual server for bearing cloud edge service is p vt The bearing time delay is t cv The bearing capacity of the virtual server v is z v Bearing time delay t of cloud edge service on virtual server mv Is L cm /z v Cloud edge service acceptance duration is as follows
The cloud edge service to-be-responded cost function is +.>
3. The cloud computing power resource adaptive computing system of claim 2, comprising accepting cloud computing loss-cost-universality objective function values inversely proportional to individual loss costs of a population, said loss-cost-universality objective function values being
The maximum duration of the cloud edge service accepted by the system is T H The time cost universality objective function for accepting cloud edge service calculation is as follows:
wherein (1)>
And representing the degree of the virtual computing resource imposed by the cloud edge service to be computed.
4. A cloud computing power resource adaptive computing system as recited in claim 3, wherein said estimating the required resources based on the improved genetic algorithm comprises the steps of:
the resource self-adaptive collaborative arrangement and flexible scheduling module randomly generates N primary individual populations and analyzes whether the iterative calculation frequency of an algorithm reaches a limit, and if so, the optimal value is screened out in situ; if the limit is not reached, the function is referred to
And->
The first 3% of individuals with larger universality value are eliminated, then 5% of the individuals following the first 3% are selected as individuals of the next generation population, the rest individuals are screened by adopting a polling algorithm, and the highest-quality individuals are extracted as next generation.
5. The cloud computing power resource adaptive computing system of claim 1, wherein the hardware resource abstraction layer is based on an Overlay mapping tool and a virtualization framework mapping tool to implement a programmable logic architecture and heterogeneous resource virtualization of heterogeneous resources; the resource network containerization layer is based on a cloud container network frame of Kubernetes+Docker, and the containerization capability of Docker is utilized to realize the resource containerization processing of abstract hardware at the cloud end and the side end; the resource collaboration support layer provides container management capabilities including container management, container orchestration, API gateway services, container load balancing, and resource monitoring, and provides support for resource awareness and resource orchestration scheduling.
6. The cloud computing power resource adaptive computing system according to claim 1, wherein the resource intelligent sensing and discovery module is configured to implement state sensing or actively request resource information to cloud nodes based on timing state update, including node type, total amount of resources, resource utilization, task running condition and task occupied resources;
when a node is requested for a resource as a target node without any response or the node requests a logoff from the resource adaptive orchestration and flexible scheduling module, the system removes the node from the resource pool.
7. The cloud computing power resource adaptive computing system according to claim 1, wherein the task scheduling and allocation are performed by using the side and cloud resource distribution condition as resource constraint, and the specific steps of realizing resource adaptive coordination are as follows:
after receiving the request, the resource self-adaptive collaborative scheduling and flexible scheduling module takes dynamic distribution information of a node resource management pool as resource constraint, takes time delay, deadline, energy consumption, characteristics and user experience of a task as allocation targets, solves single task allocation, multi-task allocation and node task processing priorities, divides the task according to the task and resource characteristics and allocates the task to a target node, generates a resource reservation queue, a data task queue, a calculation task queue and an I/O task queue, determines task unloading time and unloading strategy, and applies resources to the target node.
8. The cloud computing power resource adaptive computing system of claim 1, comprising network intelligent routes, wherein the network intelligent routes are automatically deployed at the edge and the cloud, and are perceived in real time by data such as delay, packet loss rate and the like of network links, and optimal routes are selected when the network links are abnormal.
9. The cloud computing power resource adaptive computing system of claim 1, comprising a side and cloud interface module, wherein the side and cloud interface module provides a call interface, and the third party service platform can access the cloud computing power resource adaptive computing system through the interface.
10. The cloud computing power resource adaptive computing system of claim 1, comprising a standard resource collaboration calling interface, wherein the standard resource collaboration calling interface comprises a standard resource collaboration calling interface for node registration, node cancellation, node status reporting, node resource reporting, node status query, node resource request, and node task reporting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310344726.6A CN116389491B (en) | 2023-04-03 | 2023-04-03 | Cloud edge computing power resource self-adaptive computing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310344726.6A CN116389491B (en) | 2023-04-03 | 2023-04-03 | Cloud edge computing power resource self-adaptive computing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116389491A true CN116389491A (en) | 2023-07-04 |
| CN116389491B CN116389491B (en) | 2023-11-07 |
Family
ID=86972655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310344726.6A Active CN116389491B (en) | 2023-04-03 | 2023-04-03 | Cloud edge computing power resource self-adaptive computing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116389491B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116627618A (en) * | 2023-07-21 | 2023-08-22 | 北京万界数据科技有限责任公司 | Computing resource pre-scheduling method and system |
| CN116887357A (en) * | 2023-09-08 | 2023-10-13 | 山东海博科技信息***股份有限公司 | Computing platform management system based on artificial intelligence |
| CN116996518A (en) * | 2023-09-26 | 2023-11-03 | 北京邮电大学 | Multistage heterogeneous computing power network scheduling method, device, system and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111258716A (en) * | 2020-01-14 | 2020-06-09 | 钛链智云(厦门)信息科技有限公司 | Container cloud platform-based edge machine learning system and method thereof |
| CN113703984A (en) * | 2021-09-02 | 2021-11-26 | 同济大学 | SOA (service oriented architecture) -based cloud task optimization strategy method under 5G cloud edge collaborative scene |
| CN114971574A (en) * | 2022-06-14 | 2022-08-30 | 北京航天长峰科技工业集团有限公司 | Multi-mode information composite sensing and fusion framework and method based on cloud edge cooperation |
| WO2022252496A1 (en) * | 2021-06-01 | 2022-12-08 | 苏州路之遥科技股份有限公司 | Software-defined cloud-side collaborative network energy consumption optimisation method and system |
| CN115858108A (en) * | 2022-10-14 | 2023-03-28 | 成都信息工程大学 | Cloud edge coordination system constructed based on Kubeedge edge computing framework |
| CN115866040A (en) * | 2022-11-22 | 2023-03-28 | 浙江省公众信息产业有限公司 | Cloud edge cooperative self-sensing resource allocation method and device |
-
2023
- 2023-04-03 CN CN202310344726.6A patent/CN116389491B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111258716A (en) * | 2020-01-14 | 2020-06-09 | 钛链智云(厦门)信息科技有限公司 | Container cloud platform-based edge machine learning system and method thereof |
| WO2022252496A1 (en) * | 2021-06-01 | 2022-12-08 | 苏州路之遥科技股份有限公司 | Software-defined cloud-side collaborative network energy consumption optimisation method and system |
| CN113703984A (en) * | 2021-09-02 | 2021-11-26 | 同济大学 | SOA (service oriented architecture) -based cloud task optimization strategy method under 5G cloud edge collaborative scene |
| CN114971574A (en) * | 2022-06-14 | 2022-08-30 | 北京航天长峰科技工业集团有限公司 | Multi-mode information composite sensing and fusion framework and method based on cloud edge cooperation |
| CN115858108A (en) * | 2022-10-14 | 2023-03-28 | 成都信息工程大学 | Cloud edge coordination system constructed based on Kubeedge edge computing framework |
| CN115866040A (en) * | 2022-11-22 | 2023-03-28 | 浙江省公众信息产业有限公司 | Cloud edge cooperative self-sensing resource allocation method and device |
Non-Patent Citations (1)
| Title |
|---|
| 邹萍;张华;马凯蒂;程仕通;: "面向边缘计算的制造资源感知接入与智能网关技术研究", 计算机集成制造***, no. 01, pages 40 - 48 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116627618A (en) * | 2023-07-21 | 2023-08-22 | 北京万界数据科技有限责任公司 | Computing resource pre-scheduling method and system |
| CN116627618B (en) * | 2023-07-21 | 2023-09-19 | 北京万界数据科技有限责任公司 | Computing resource pre-scheduling method and system |
| CN116887357A (en) * | 2023-09-08 | 2023-10-13 | 山东海博科技信息***股份有限公司 | Computing platform management system based on artificial intelligence |
| CN116887357B (en) * | 2023-09-08 | 2023-12-19 | 山东海博科技信息***股份有限公司 | Computing platform management system based on artificial intelligence |
| CN116996518A (en) * | 2023-09-26 | 2023-11-03 | 北京邮电大学 | Multistage heterogeneous computing power network scheduling method, device, system and storage medium |
| CN116996518B (en) * | 2023-09-26 | 2023-12-26 | 北京邮电大学 | Multistage heterogeneous computing power network scheduling method, device, system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116389491B (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116389491B (en) | Cloud edge computing power resource self-adaptive computing system | |
| CN113079218B (en) | Service-oriented power network system, working method and storage medium | |
| Belgaum et al. | A systematic review of load balancing techniques in software-defined networking | |
| Zhang et al. | Online adaptive interference-aware VNF deployment and migration for 5G network slice | |
| Hu et al. | BalanceFlow: Controller load balancing for OpenFlow networks | |
| US20220131945A1 (en) | Methods, systems, and computer readable media for rank processing for network function selection | |
| CN113448721A (en) | Network system for computing power processing and computing power processing method | |
| CN113709048A (en) | Routing information sending and receiving method, network element and node equipment | |
| Chiti et al. | Virtual functions placement with time constraints in fog computing: A matching theory perspective | |
| CN102035737A (en) | Adaptive load balancing method and device based on cognitive network | |
| CN113810205B (en) | Service computing power information reporting and receiving method, server and data center gateway | |
| Bu et al. | Enabling adaptive routing service customization via the integration of SDN and NFV | |
| CN104798356A (en) | Method and apparatus for controlling utilization in a horizontally scaled software application | |
| Ibrahim et al. | A multi-objective routing mechanism for energy management optimization in SDN multi-control architecture | |
| Kang et al. | Application of adaptive load balancing algorithm based on minimum traffic in cloud computing architecture | |
| Lei et al. | Computing power network: An interworking architecture of computing and network based on IP extension | |
| CN109600760A (en) | Network management, equipment and system | |
| Zhang et al. | Cost efficient and low-latency network service chain deployment across multiple domains for SDN | |
| KR20130060350A (en) | Method and apparatus for scheduling communication traffic in atca-based equipment | |
| Raith et al. | Mobility-aware serverless function adaptations across the edge-cloud continuum | |
| CN116684418B (en) | Calculation power arrangement scheduling method, calculation power network and device based on calculation power service gateway | |
| CN108933737A (en) | Load-balancing method and device | |
| CN116455817A (en) | Software-defined cloud network fusion architecture and route implementation method | |
| CN115883490A (en) | SDN-based distributed computing and communication integrated scheduling method and related components | |
| Hu et al. | An information transport dynamic load balancing policy based on software defined mobile networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |