CN117135156B - Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol - Google Patents

Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol Download PDF

Info

Publication number
CN117135156B
CN117135156B CN202311122427.4A CN202311122427A CN117135156B CN 117135156 B CN117135156 B CN 117135156B CN 202311122427 A CN202311122427 A CN 202311122427A CN 117135156 B CN117135156 B CN 117135156B
Authority
CN
China
Prior art keywords
cluster
resource
edge
data
edge cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311122427.4A
Other languages
Chinese (zh)
Other versions
CN117135156A (en
Inventor
梅强
蔡威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Daoke Network Technology Co ltd
Original Assignee
Shanghai Daoke Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Daoke Network Technology Co ltd filed Critical Shanghai Daoke Network Technology Co ltd
Priority to CN202311122427.4A priority Critical patent/CN117135156B/en
Publication of CN117135156A publication Critical patent/CN117135156A/en
Application granted granted Critical
Publication of CN117135156B publication Critical patent/CN117135156B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04Protocols for data compression, e.g. ROHC

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer And Data Communications (AREA)

Abstract

The application relates to the technical field of cloud protogenesis, and provides an edge cluster nano-tube method and system based on a publish/subscribe message protocol. In the scheme, an edge cluster management component and a multi-cluster management component are deployed in a central cluster, and the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to a target resource, and the target resource is any resource of the edge cluster; the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode; the edge cluster management component publishes the control message data packet based on a message transmission mechanism of a publish/subscribe mode, and the edge cluster receives the control message data packet in an asynchronous subscription mode and executes management operation on the target resource according to the control message data packet. Therefore, the center cluster can reliably transmit the control message to the edge cluster in the weak network environment, so that the nanotubes of the edge cluster are realized.

Description

Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol
Technical Field
The present application relates to the field of cloud native technologies, and in particular, to an edge cluster management method, system, computer readable storage medium, and electronic device based on a publish/subscribe message protocol.
Background
An edge cluster, also called edge cloud, is made up of multiple nodes at edge locations. The central cluster (central cloud) can perform unified management on a plurality of edge clusters, and the edge clusters are responsible for executing application programs or other computing tasks submitted by users.
However, because the edge location (such as a factory or a field) where the edge cluster is located often has unstable network connection and poor signal transmission quality, the conventional communication mode based on the HTTP or the gRPC protocol has insufficient reliability in a weak network environment, which results in difficulty in managing the edge cluster by the central cluster.
Accordingly, there is a need to provide an improved solution to the above-mentioned deficiencies of the prior art.
Disclosure of Invention
It is an object of the present application to provide a method, a system, a computer readable storage medium and an electronic device for edge cluster nanotubes based on a publish/subscribe message protocol, which solve or alleviate the above-mentioned problems in the prior art.
In order to achieve the above object, the present application provides the following technical solutions:
The application provides an edge cluster nano-management method based on a publish/subscribe message protocol, wherein an edge cluster management component and a multi-cluster management component are deployed in a center cluster, and the method comprises the following steps:
the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to a target resource, wherein the target resource is any resource of the edge cluster;
the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode;
the edge cluster management component publishes the control message data packet based on a message passing mechanism of a publish/subscribe mode, receives the control message data packet in an asynchronous subscription mode by the edge cluster, and executes the management operation on the target resource according to the control message data packet.
In the above solution, before sending an operation request to the edge cluster management component, the multi-cluster management component further includes:
invoking a data cache service deployed in the central cluster to acquire the latest state information of the target resource;
Determining legal operation behaviors and effective management contents of the target resources according to the latest state information;
generating the operation request based on legal operation behaviors of the target resource and effective management content;
the data caching service is used for storing cluster resource twin data of the edge cluster, and the cluster resource twin data is used for describing the latest state information of all cluster resources in the edge cluster.
In the above scheme, the cluster resource twin data is created by the following steps:
an edge cluster management agent in the edge cluster encapsulates cluster resource data of the edge cluster by using a message protocol of a publish/subscribe mode, and generates a registration message data packet corresponding to the cluster resource data of the edge cluster;
transmitting the registration message data packet to an edge cluster management component of the central cluster based on a message passing mechanism of a publish/subscribe mode;
the edge cluster management component of the central cluster invokes the data caching service to create, by the data caching service, the cluster resource twinning data based on the cluster resource data of the edge cluster contained in the registration message data packet.
In the above scheme, the edge cluster management agent in the edge cluster encapsulates the cluster resource data of the edge cluster by using a message protocol in a publish/subscribe mode, and generates a registration message data packet corresponding to the cluster resource data of the edge cluster, specifically:
the edge cluster management agent generates a message theme of the registration message data packet based on cluster resource data of the edge cluster;
analyzing the cluster resource data of the edge cluster to determine the resource type corresponding to the cluster resource data of the edge cluster;
based on the resource types, adopting different processing modes to compress the cluster resource data of the edge cluster to obtain the compressed cluster resource data of the edge cluster;
and packaging the cluster resource data of the compressed edge cluster as a message body of the registration message data packet.
In the above solution, the edge cluster management component of the central cluster invokes the data caching service to create, by the data caching service, the cluster resource twin data based on the cluster resource data of the edge cluster included in the registration message data packet, specifically:
The edge cluster management component of the center cluster calls the data cache service to determine whether cluster resource twin data corresponding to the cluster resource data of the edge cluster is stored in the data cache service;
responding to the cluster resource twin data corresponding to the cluster resource data of the edge cluster which is not stored in the data cache service, and generating a unique identifier of the cluster resource data of the edge cluster;
creating cluster resource twin data corresponding to the cluster resource data of the edge cluster;
and storing the cluster resource twin data into the data caching service in the form of key value pairs, and taking the unique identification as identification information of the cluster resource twin data.
In the above solution, the sending, by the multi-cluster management component, an operation request to the edge cluster management component is specifically:
the multi-cluster management component sends the operation request to a data gateway of the edge cluster management component in a connection-oriented communication mode;
the data gateway determines a unique identifier of the target resource and a type of an instruction carried by the operation request based on the access address information and the instruction parameters recorded by the operation request;
And responding to the instruction carried by the operation request as an operation instruction, and forwarding the operation request to a resource synchronizer of the edge cluster management component by the data gateway so as to package the operation request by using a message protocol of a publish/subscribe mode by the resource synchronizer to obtain a control message data packet based on the publish/subscribe mode.
In the above solution, before sending an operation request to the edge cluster management component, the multi-cluster management component further includes:
acquiring access address information of the target resource through accessing an edge cluster resource object, and writing the access address information into the operation request;
wherein the edge cluster resource object is registered in the multi-cluster management component by:
the edge cluster management component invokes the data gateway after generating the unique identifier of the cluster resource data of the edge cluster so as to generate the access address information corresponding to the cluster resource data of the edge cluster by the data gateway;
the edge cluster management component generates an edge cluster resource registration request based on the access address information;
and the multi-cluster management component responds to the edge cluster resource registration request and generates an edge cluster resource object corresponding to cluster resource data of the edge cluster.
The embodiment of the application provides an edge cluster nano-tube system based on a publish/subscribe message protocol, wherein the system is positioned in a central cluster, an edge cluster management component and a multi-cluster management component are deployed in the central cluster, and the system comprises:
the sending unit is configured to send an operation request to the edge cluster management component by the multi-cluster management component; the operation request carries management operation pointing to a target resource, wherein the target resource is any resource of the edge cluster;
the encapsulation unit is configured to encapsulate the operation request by the edge cluster management component by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode;
and the issuing unit is configured to issue the control message data packet by the edge cluster management component based on a message transmission mechanism of a publish/subscribe mode, receive the control message data packet in an asynchronous subscription mode by the edge cluster, and execute the management operation on the target resource according to the control message data packet.
Embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the edge cluster nanotube method based on the publish/subscribe message protocol as described in any of the embodiments above.
The embodiment of the application also provides electronic equipment, which comprises: a memory, a processor, and a program stored in the memory and executable on the processor, which when executed implements the publish/subscribe message protocol based edge cluster nanotube method as described in any of the embodiments above.
The beneficial effects are that:
in the technical scheme, an edge cluster management component and a multi-cluster management component are deployed in a central cluster, and when the central cluster needs to perform unified management on any resource (namely, target resource) of the edge cluster, the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to the target resource; the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode; the edge cluster management component publishes the control message data packet based on the message passing mechanism of the publish/subscribe mode, receives the control message data packet in an asynchronous subscription manner by the edge cluster, and performs management operation on the target resource according to the control message data packet. The working principle of the message transmission mode based on the publish/subscribe message protocol can be known that in the process of issuing the operation instruction, the operation instruction is firstly issued to a message queue or a message middleware, then the edge cluster acquires the operation instruction through subscribing the message, and even if the network connection is unstable and the signal transmission quality is low, the network is interrupted, the operation instruction is still stored in the message queue or the message middleware until the network is restored. When the network is restored, the operation instruction can be correctly transmitted without loss, thereby ensuring the reliability of message transmission. On the other hand, the message transfer mode based on the publish/subscribe message protocol is a non-connection-oriented communication mode, and the communication mode can decouple the center cluster from the edge cluster, allow the edge cluster management component of the center cluster to continuously execute other tasks after sending out an operation instruction, even if the network of the edge cluster is unstable, each related component of the center cluster can still continuously work without waiting for the response of the edge cluster, and avoid the occurrence of aggravated network blocking condition caused by the fact that the connection is maintained due to the waiting for the response in the weak network environment. Meanwhile, as the multi-cluster management component and the edge cluster management component are both deployed in the center cluster, the communication between the multi-cluster management component and the edge cluster management component belongs to the internal communication of the center cluster, so that the multi-cluster management component can still send an operation instruction to the edge cluster management component in an original communication mode, the multi-cluster management component is completely unaware of the change of a communication protocol between the center cluster and the edge cluster, when the multi-cluster management component of different types is required to be added, the newly added multi-cluster management component can realize unified management of the edge cluster without any adjustment, and the transparency, compatibility and usability of the scheme are improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. Wherein:
fig. 1 is a schematic structural diagram of multi-cloud management in an edge scenario provided in the related art;
FIG. 2 is a logical schematic diagram of an edge cluster nanotube approach based on publish/subscribe messaging protocol provided by some embodiments of the present application;
FIG. 3 is a flow chart of an edge cluster nanotube approach based on publish/subscribe messaging protocol provided by some embodiments of the present application;
fig. 4 is a schematic flow chart of reporting cluster resource data by an edge cluster according to some embodiments of the present application;
fig. 5 is a schematic diagram of a message packet according to some embodiments of the present application;
FIG. 6 is a schematic diagram of an edge cluster nanotube system based on a publish/subscribe message protocol provided by some embodiments of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 8 is a hardware configuration diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Before further describing embodiments of the present application in detail, the terms and terminology involved in the embodiments of the present application will be described, with the terms and terminology involved in the embodiments of the present application being suitable for the following explanation.
1. The cloud native application is an application which is built by the thought of cloud computing and is suitable for a cloud computing environment and is provided with cloud computing genes, and the cloud native application has the characteristics that: the method can be implemented through network access, remote deployment, scalable elastic scaling, sharing, on-demand use of autonomous services, high availability, remote monitoring of billing audits, standardized delivery independent of location, and the like.
2. Kubernetes, abbreviated as K8S, is a container orchestration engine of Google open source, and a Kubernetes management platform is generally referred to as a container cluster management system using Kubernetes as a core engine, and can provide a cloud native platform for deploying, scheduling resources, discovering services, dynamically expanding and contracting, and the like, which covers the whole life cycle operation and management of a distributed system for containerized applications.
A Kubernetes cluster is a cluster of nodes, including a plurality of nodes, deployed with a Kubernetes system.
The Kubernetes system is one of the most popular container organizers at present, and the technical solution is described by taking the Kubernetes system as an example in this embodiment.
3. An edge cluster, also called edge cloud, is composed of multiple nodes at edge locations, and a central cluster (central cloud) can uniformly manage multiple edge clusters, which are responsible for executing applications or other computing tasks submitted by users.
The application scenario and related technology of the present embodiment are described below.
The embodiment of the application provides an edge cluster nano-tube method and system based on a publish/subscribe message protocol.
The central cluster is deployed in a data center, the data center is a place for centralized calculation and data storage, and comprises high-performance nodes (servers), high-speed internet access bandwidth, reliable machine room environment and the like, is an important component of cloud computing, and can provide an infrastructure for the cloud computing and simultaneously provide centralized data storage and processing functions. In this embodiment, a cloud computing platform built on top of the infrastructure of a data center is referred to as a central cloud.
The concept corresponding to the center Cloud is Edge Cloud (Edge Cloud), which is related to Edge computing (Edge processing), which is a network technology of placing nodes near a sensor or a device, wherein the sensor or the device is responsible for collecting and generating data, a position near the sensor or the device is called an Edge position, a node at the Edge position is called an Edge node, when a plurality of Edge nodes exist at the same Edge position, a Cloud native platform represented by a Kubernetes system can be used for uniformly managing the Edge nodes to form an Edge cluster (Edge Cloud), and the Edge Cloud can process and store the data generated by the sensor and the device, so that the processing load of the system is reduced and the data transmission delay is solved.
In general, the central cloud processes all data on the cloud or at a central location, and edge computing is able to process and store data in the edge cloud near the sensor or device, compared to the central cloud, so that edge computing solves the problems related to insufficient network bandwidth and delay, and is able to process data in real time.
For example, in an automatic driving scenario, a vehicle is required to respond quickly to road conditions, if cloud computing is relied on to respond, traffic accidents may occur once data transmission or processing is delayed, and the use of edge computing to process detected data can ensure that the vehicle responds to road conditions in real time.
Fig. 1 shows an example of cloudiness management in an edge scene in the related art. As shown in fig. 1, in the edge scenario, in the multi-cloud management method, a central cluster is mainly configured, and monitors and manages workloads in multiple clouds (which may be central clouds or edge clouds) through a set of tools or management programs (also referred to as multi-cluster management components), so as to implement unified management on the multiple edge clusters, and in terms of a communication manner, an HTTP protocol or a gppc-based protobuf protocol is used to issue an operation instruction to the edge clusters, and the edge clusters perform corresponding operations according to the received operation instruction.
However, as described in the background art, because the situation that the network connection is unstable and the signal transmission quality is not high often occurs at the edge position (such as the factory or the field) where the edge cluster is located, the problem that the network delay is large and the data packet is seriously lost exists in the communication between the edge cluster and the center cluster is caused, so that the operation instruction sent by the center cluster cannot normally reach the edge cluster, the edge cluster cannot naturally perform corresponding operation according to the content of the operation instruction, and the management of the edge cluster by the center cluster is difficult.
The applicant has intensively studied the above problems and the prior art, and found that in the prior container cloud technology, a center cluster issues an operation instruction to an edge cluster by using an HTTP protocol or a protobuf protocol based on gRPC, and the two protocols are realized by adopting a connection-oriented communication mode based on a synchronous communication mechanism. Specifically, a data channel is established between a sending end and a destination end before data transmission, and a Socket is created at the system level of the sending end and the destination end, so that an upper layer application transmits data for use. The synchronous communication mechanism adopts a connection-oriented communication mode to enable the transmitting end and the destination end to have coupling, namely the transmitting end and the destination end can complete the communication process by being matched together, however, under the weak network environment of the edge position, the condition of network interruption sometimes occurs, and the transmission of application data is interrupted, so that the transmission data is lost.
On the other hand, the HTTP protocol or the protobuf protocol based on gRPC has no mechanism for checking the validity of the data channel in real time, when the intermediate link of the data channel fails, the transmitting end and the destination end of the data channel cannot sense that the data channel fails in real time, and only after receiving the information of overtime of the transmitted data or overtime of the socket connection, the socket connection can be interrupted, related system resources are recovered, the whole process needs a longer time, when more operation instructions are issued by a central cluster or more cluster resource data are reported by an edge cluster, a large number of sockets are created in a short time, related system resources cannot be recovered in time, network blocking is aggravated, and therefore, network communication components are crashed, and the stable operation of the system is affected.
In addition, when the operation instruction is sent in the form of HTTP protocol or the protobuf protocol of the gRPC, since the data format of the operation instruction is a text markup language, the packet body of the data packet of the operation instruction is larger, and when the number of edge clusters is more (for example, more than 100), a large number of operation instructions are often required to be issued at the same time, which makes the requirement on the communication bandwidth between the center cluster and the edge clusters very high, once the network shakes, the network is further blocked by the large number of operation instructions, and the service application deployed in the center cluster is affected.
In order to solve the above-mentioned problems, an embodiment of the present application provides an edge cluster nano-tube scheme based on a publish/subscribe message protocol, in which a central cluster is deployed with an edge cluster management component and a multi-cluster management component, the multi-cluster management component sends an operation request to the edge cluster management component, the edge cluster management component encapsulates the operation request by using a publish/subscribe (publish/subscribe) mode message protocol, and publishes a control message packet based on a publish/subscribe mode message delivery mechanism, and the edge cluster acquires an operation instruction through a subscribe message.
By improving the communication mode between the central cluster and the edge cluster, the existing HTTP protocol and protobuf protocol are replaced by a message protocol based on a publish/subscribe mode, and the message passing based on the publish/subscribe mode is that a publisher publishes a message to a message broker, a subscriber registers and subscribes in the message broker, and the message broker can store and forward the message and route the message from the publisher to the subscriber. Thus, even if the network connection is unstable and the signal transmission quality is low, which leads to the interruption of the network, the operation instruction is still stored in the message queue or the message middleware until the network is restored. When the network is restored, the operation instruction can be correctly transmitted without loss, thereby ensuring the reliability of message transmission. And meanwhile, the center cluster and the edge cluster are decoupled, so that the situation that the network blocking is aggravated due to the fact that connection is maintained due to waiting for response in a weak network environment is avoided.
In contrast to the traditional client-server model, both publishers and subscribers in the publish/subscribe model are clients and message intermediaries are message servers. Customer service side A sends information to server, customer side A is publisher, customer side B needs to subscribe information to server, customer side B is subscriber, message intermediary (message server) is located between publisher and subscriber, as information transmission hub, responsible for transmitting information sent by customer side A to customer side B. In addition, the message server is also responsible for managing the clients, ensuring smooth communication between the clients and ensuring that the message can be correctly received and accurately delivered.
It should be noted that, as known from the above description of the working principle of the message passing mode, the message passing based on the publish/subscribe mode is an asynchronous communication mode, information sent by a publisher can be stored in a message broker, and when the subscriber is connected with the message broker network, the message is transmitted to the subscriber, so that the reliability of the message passing is ensured.
In addition, the message data packet based on the publish/subscribe mode is smaller than the HTTP protocol, so that the network bandwidth consumption of the central cluster is lower, and the method is more suitable for a scene that the central cluster simultaneously sends a large number of operation instructions to the edge cluster. Thus, compared to the existing HTTP protocol and protobuf protocol, the message protocol based on the publish/subscribe mode can be more suitable for the situation that the network condition is bad, such as the communication between the central cluster and the edge cluster.
The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments. Various examples are provided by way of explanation of the present application and not limitation of the present application. Indeed, it will be apparent to those skilled in the art that modifications and variations can be made in the present application without departing from the scope or spirit of the application. For example, features illustrated or described as part of one embodiment can be used on another embodiment to yield still a further embodiment. Accordingly, it is intended that the present application include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
In the following description, the terms "first/second/third" are used merely to distinguish between similar objects and do not represent a particular ordering of the objects, it being understood that the "first/second/third" may be interchanged with a particular order or precedence where allowed, to enable embodiments of the present application described herein to be implemented in other than those illustrated or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of the present disclosure only and is not intended to be limiting of the present disclosure.
Exemplary method
The embodiment of the application provides an edge cluster nano-tube method based on a publish/subscribe message protocol, wherein an edge cluster management component and a multi-cluster management component are deployed in a center cluster, as shown in fig. 1-5, the method comprises the following steps:
step S101, the multi-cluster management component sends an operation request to the edge cluster management component.
In this embodiment, there are one or more edge clusters, where a container orchestration engine (i.e. a management system of a container cloud platform) is pre-deployed in each edge cluster, for example, may be a reduced version K3s of Kubernetes, and each edge cluster is connected to the central cluster network and accepts nanotubes of the central cluster. Here, the nano tube refers to a container cloud management platform which uniformly accesses the edge clusters distributed everywhere into the center cluster and enjoys the uniform management and monitoring functions provided by the container cloud management platform.
In order to realize unified management of the edge clusters, a multi-cluster management component is deployed in the center cluster and is used for managing a plurality of clouds formed by the center cluster and a plurality of edge clusters, so that application deployment, configuration and management of a multi-cloud scene are simplified and unified, and application arrangement and management of the cross-clusters are realized, so that a developer can manage application programs in the clouds more easily.
Illustratively, in the container cloud platform represented by the Kubernetes system, the multi-cluster management component may be Karmada, kubefed, clusterPedia or the like, and the type of the multi-cluster management component is not limited in this embodiment.
Further, the multi-cluster management component is typically deployed on a control node of the central cluster. It is to be appreciated that the multi-cluster management component can also be deployed on a working node of a central cluster, as this application is not limited in this regard.
The control node and the working node are divided based on different roles of the nodes in the cluster. In practice, from the view of the composition of nodes in the cluster, the container cloud platform represented by the Kubernetes system comprises a control node and a plurality of working nodes, wherein the control node is a management node of the Kubernetes cluster and is responsible for managing the state and configuration information of the whole cluster, and the working nodes are nodes running containerized application programs and bear a container group (Pod) and execute actual workload.
In this embodiment, the control node in the central cluster is responsible for managing the state and configuration information of the entire central cluster and the state and configuration information of the plurality of edge clusters.
In order to implement communication between the central cluster and the edge cluster based on the publish/subscribe message protocol, in this embodiment, an edge cluster management component is further disposed in the central cluster, as shown in fig. 2, where the edge cluster management component may be disposed on a control node of the central cluster or may be disposed on a working node of the central cluster, and this embodiment is not limited to this.
In this embodiment, when the multi-cluster management component manages the edge clusters, the multi-cluster management component (for example Karmada) needs to send an operation request to the edge cluster management component, and the edge cluster management component receives the operation request sent by the multi-cluster management component, encapsulates the operation request into a form of a publish/subscribe message data packet, and then sends the operation request to each edge cluster.
The operation request carries management operation pointing to a target resource, and the target resource is any resource of the edge cluster.
In this embodiment, the target resource is any resource of the edge cluster. The resource is an abstraction of managed content by the container cloud platform, and illustratively, the resources of Kubernetes may include: pod, service, deployment, job, statetulSet, secret, configMap, node, etc. Therefore, the target resource in this embodiment may be any one Pod, service, deployment, job, statetulSet, secret, configMap of the edge clusters, or Node, and the central cluster needs to operate these resources in the edge clusters by sending an operation request, so as to achieve the purpose of managing the entire edge clusters.
In order to implement management of the edge cluster, the operation request carries a management operation pointing to the target resource, where the management operation is a specific action or behavior performed on the target resource, and is an operation behavior implementing a management function, for example, the operation behavior may be a GET operation for reading the target resource, a LIST operation for listing or observing a target resource type, and a CREATE/UPDATE/DELETE operation for creating/updating/deleting the target resource.
In this embodiment, the multi-cluster management component may send the operation request to the edge cluster management component in multiple manners, for example, may send the operation request to the edge cluster management component using HTTP protocol, may send the operation request to the edge cluster management component using gppc protobuf protocol, and may send the operation request to the edge cluster management component using other communication protocols.
To ensure the legitimacy and effectiveness of the management of the target resources, in some embodiments, the multi-cluster management component, prior to sending the operation request to the edge cluster management component, further comprises: invoking a data cache service deployed in a central cluster to acquire the latest state information of the target resource; determining legal operation behaviors and effective management contents of the target resources according to the latest state information; generating an operation request based on legal operation behaviors of the target resource and effective management content; the data caching service is used for storing cluster resource twin data of the edge cluster, and the cluster resource twin data is used for describing the latest state information of all cluster resources in the edge cluster.
In this embodiment, cluster resource twin data is cached in advance in the data caching service of the central cluster, where the cluster resource twin data is used to describe the latest state information of all cluster resources in the edge cluster. Wherein the latest state information may include a resource state or Event (Event). The content of the resource state contained in the latest state information may also be different according to the different cluster resource types, for example, the resource state of the Node may include: access address information, operating conditions, status of available resources of the node, operating systems running on the node, kernel versions, kubernetes versions, etc.; for another example, the resource status of Pod may include: start time of Pod, lifecycle location (e.g., pending, running, succeeded, failed), node location, etc. An Event (Event) is a report of an Event in a cluster that generally indicates that certain actions have caused certain state changes of the system, typically including time of Event occurrence (eventTime), action taken (action), action result (note), action cause (reason), etc., and by storing events for all cluster resources of the respective edge clusters, the evolution process of the cluster resources can be traced back over time. It will be appreciated that the latest state information may also include other state information related to the edge cluster, which is not limited in this application.
In this embodiment, as shown in fig. 2, the data cache service is deployed on a control node of the central cluster, and before sending an operation request to the edge cluster management component, the multi-cluster management component invokes the data cache service to obtain the latest state information of the target resource, and determines legal operation behaviors and effective management contents of the target resource according to the latest state information.
The legal operation behavior of the target resource may be an operation behavior executed on the target resource and obtaining a preset operation result. For example, for a Pod in a Pending (Pending) or Failed (Failed) state, it may be deleted by sending a DELETE operation, i.e., the DELETE operation is a legal operation behavior. However, since the suspended (Pending) or Failed (Failed) state is actually that the Pod is in an abnormal state, some operation behaviors may not obtain a preset operation result for the Pod in the abnormal state, for example, the GET operation may not be able to read the Pod, and the UPDATE operation may not be able to UPDATE the Pod, at which time the GET operation, the UPDATE operation may be considered an illegal operation behavior for the Pod. For another example, a Pod is deleted at an edge cluster, and a DELETE (DELETE) instruction is issued to the Pod, so that a preset operation result cannot be obtained.
The effective management content of the target resource can be understood as whether the configuration information of the target resource has differential configuration in the edge cluster. Here, the differentiated configuration is a personalized setting made by the edge cluster for the target resource, and corresponds to a uniform resource template issued by the multi-cluster management component in management. In multi-cloud management, a multi-cluster management component, as a control plane component of a multi-cluster, performs unified management on cluster resources in an edge cluster through a series of data structures (such as resource objects) related to multi-cluster management. Taking Karmada as an example, the uniform distribution of the multi-cluster resources is realized through a resource template (resource template) and a delivery policy (PropagationPolicy). Here, resource template (resource template) is an abstraction of the resource that Karmada can distribute, where the resource can contain all supported types of resources in Kubernetes, such as deviyment, service, pod, lngress, PVC, PV, job, etc., while also supporting custom resource CRD. The issue policy (PropagationPolicy) defines the policy to which resource templates (resource templates) are issued, for example, to which edge clusters, and when issued to those edge clusters, determines the number of copies of each cluster resource, and how each copy is allocated.
In addition to unified management through resource templates (resource templates) and issue policies (provisioning policies), different edge clusters can also perform differentiated configuration on cluster resources according to their own needs in the running process of service application, when an edge cluster performs differentiated configuration on a certain cluster resource (for example, a target resource), it is generally desirable that a multi-cluster management component can keep the differentiated configuration in the unified management process, but not be covered by the unified resource templates (resource templates), so that effective management content of the target resource needs to be confirmed before issuing an operation request, that is, whether the target resource has differentiated configuration on the edge cluster. In this embodiment, since the cluster resource twin data can describe the latest state information of all cluster resources in the edge cluster, it can be known whether the target resources have differential configuration, if so, the configuration information of the target resources provided by the cluster resource twin data is used as effective management content, otherwise, the unified resource template provided by the multi-cluster management component is used as effective management content of the target resources.
It should be noted that, the operation instruction issued by the multi-cluster management component includes an operation behavior or a management content that cannot obtain a preset operation result, and there are various processing manners in the industry at present, for example, one possible processing manner is that the correctness and the validity of the content included in the operation request are not verified, and after the edge cluster receives the operation request, the content of the operation request is verified by the edge cluster according to the latest state information of the cluster resource of the cluster where the edge cluster is located. Another approach is for the multi-cluster management component to verify before issuing an operation request, however, the multi-cluster management component needs to obtain the latest state information of the target resource in the edge cluster before issuing an operation request, which undoubtedly increases the network burden. In this embodiment, cluster resource twin data is cached in advance in a central cluster, and before an operation request is generated, the latest state information of a target resource can be obtained from the pre-established cluster resource twin data, so that legal operation behavior and effective management content of the target resource are determined, the operation request is generated based on the legal operation behavior and the effective management content, the validity, accuracy and rationality of the operation request are ensured, management operation failure caused by that the operation behavior or the management content does not conform to the actual situation is avoided, and meanwhile, network load is avoided being increased.
In this embodiment of the present application, the cluster resource twin data may be created in advance in a plurality of manners, for example, the latest state information of all cluster resources of each edge cluster may be registered or synchronized to the central cluster periodically by using a synchronization mechanism provided by some multi-cluster management components, or probes (probes) may be set in each edge cluster (for example, a survival Probe, a ready Probe, and a start Probe) for monitoring and periodically reporting the latest state information of all cluster resources of each edge cluster. However, due to the development history of the container cloud platform, most of the synchronization mechanisms provided by the existing multi-cluster management component are based on the HTTP protocol for communication, and the probes adopt a communication mode of the protobuf protocol based on the gRPC, so that the problem of network blocking is also caused in the reporting of cluster resource data of the edge cluster.
For this reason, some embodiments of the present application improve the communication protocol of the reporting process of the cluster resources of the edge cluster, and use the message protocol of the publish/subscribe mode to implement the reporting of the cluster resource data of the edge cluster. Specifically, cluster resource twinning data is created by: an edge cluster management agent in the edge cluster encapsulates cluster resource data of the edge cluster by using a message protocol of a publish/subscribe mode, and generates a registration message data packet corresponding to the cluster resource data of the edge cluster; transmitting the registration message data packet to an edge cluster management component of the central cluster based on a message delivery mechanism of a publish/subscribe mode; the edge cluster management component of the central cluster invokes the data caching service to create cluster resource twinned data by the data caching service based on the cluster resource data of the edge cluster contained in the registration message data packet.
According to different reporting contents, reporting of cluster resource data can be divided into two scenes of registering cluster resources and updating the cluster resources, wherein registering of the cluster resources refers to that an edge cluster is firstly connected with a central cluster, and all cluster resources of the cluster are registered to the central cluster through reporting of the cluster resource data to construct cluster resource twin data of the edge cluster; the updating of the cluster resources means that part or all of the cluster resources in the edge clusters are increased or decreased, and the latest state information of the cluster resources which are changed is sent to the center cluster through the reporting of the cluster resource data so as to update the cluster resource twin data. Since the communication principle of the edge cluster and the center cluster is basically the same in the two scenarios, the following description will specifically refer to the registration process of the cluster resource.
In order to implement registration of cluster resources by using a message protocol in a publish/subscribe mode, as shown in fig. 2, in this embodiment, an edge cluster agent is deployed in each edge cluster, and is configured to collect cluster resource data (such as a resource status, an event, etc.) of the edge cluster, and report the cluster resource data to an edge cluster management component in a central cluster based on the publish/subscribe message protocol.
In this embodiment, the message protocol of the publish/subscribe mode may be any communication protocol capable of implementing the publish/subscribe message transfer mode, such as an advanced message queue protocol (Advanced Message Queuing Protocol, AMQP) protocol, a message queue telemetry transport protocol (Message Queuing Telemetry Transport, MQTT), etc., and the embodiment is not limited to a specific protocol type.
The MQTT protocol is used as a lightweight communication protocol based on a publish/subscribe (publish/subscribe) mode, is constructed on a TCP/IP protocol, can provide real-time reliable message service for remote connection equipment with very little code and limited bandwidth, and is used as an instant communication protocol with low cost and low bandwidth occupation, and has wider application in the aspects of Internet of things, small-sized equipment, mobile application and the like. Thus, as a preferred implementation, the present embodiment may use the MQTT protocol instead of the HTTP protocol and the protobuf protocol as a protocol for transmitting operation instructions and cluster resource data between the central cluster and the edge cluster.
As a specific implementation of the publish/subscribe mode based messaging mechanism, the edge cluster management agent includes a publish/subscribe message Client (Client) and a data transfer unit. Accordingly, the edge cluster management component of the central cluster includes a publish/subscribe message middleware (Broker) and a publish/subscribe message client. The publish/subscribe message client is used for publishing or subscribing messages, and the publish/subscribe message middleware is a message server which is used as a message intermediary to be responsible for forwarding the messages, and assists the edge cluster and the center cluster to complete message transmission.
The data transmission unit is used for accessing the API-Server component of the edge cluster through a List-watch mechanism to acquire cluster resource data of the edge cluster, and sending the cluster resource data to the publishing/subscribing message client of the edge cluster. And the publishing/subscribing message client of the edge cluster receives the cluster resource data, encapsulates the cluster resource data, generates a registration message data packet corresponding to the cluster resource data of the edge cluster, and publishes the registration message data packet to the publishing/subscribing message middleware of the center cluster by using a message protocol of a publishing/subscribing mode.
The encapsulation of the registration message data packet mainly relates to the generation of a message theme (Topic) and a message body (body), wherein the message theme is used as an identifier for classifying the message data packet transmitted between the publish/subscribe message client and the publish/subscribe message middleware, the message body is the specific content of the message, and the message theme and the message body are written into the message data packet (also called as a publish/subscribe message data packet) conforming to the standard of the message protocol of the publish/subscribe mode, so that the encapsulation can be completed.
As an example, fig. 5 shows the structure of a message packet conforming to the MQTT protocol. As shown in fig. 5, the publish/subscribe message packet is composed of three parts, fixed Header (Fixed Header), variable Header (Variable Header), and message body (Payload).
The fixed header is used for starting a message data packet of the MQTT protocol, and the structure of the fixed header comprises a control message type, a flag bit and a residual length, wherein the control message type is used for indicating the message type, such as publishing, subscribing, connecting request and the like; the flag bits, i.e. some specific flag bits, may have different meanings according to different message types; the remaining length is the remaining length of the data packet represented by the variable length code. The variable header is located after the fixed header and may have different variable headers depending on the message type. For example, when a message is published, the variable header includes the message subject, a message identifier, and the like. The message body is the actual content of the message, and can be any format data.
When the registration message data packet is encapsulated by the MQTT protocol, there are various encapsulation modes, for example, one possible implementation mode is to directly encapsulate the cluster resource data as a message body of the registration message data packet and the randomly generated unique topic identifier as a message topic, and such encapsulation mode is simple to implement and easy to operate.
However, the above-mentioned encapsulation method has the problems of large transmission amount and ambiguous message subject, so in some embodiments, the edge cluster management agent in the edge cluster encapsulates the cluster resource data of the edge cluster by using the message protocol of the publish/subscribe mode, and the generation of the registration message packet corresponding to the cluster resource data of the edge cluster specifically includes: the edge cluster management agent generates a message theme of a registration message data packet based on cluster resource data of the edge cluster; analyzing cluster resource data of the edge cluster to determine a resource type corresponding to the cluster resource data of the edge cluster; based on the resource types, adopting different processing modes to compress the cluster resource data of the edge cluster to obtain the compressed cluster resource data of the edge cluster; and packaging the compressed cluster resource data of the edge cluster as a message body of the registration message data packet. Therefore, the message theme is closely related to the cluster resources of the edge cluster, so that the message theme of the encapsulated registration message data packet can effectively distinguish different cluster resources, the characteristics of the cluster resources are clear, the subsequent use and processing are facilitated, meanwhile, different processing modes are selected according to the resource types to compress the cluster resource data, the network transmission quantity can be further reduced, and the possibility of network blocking in a weak network environment is reduced.
In this embodiment, the edge cluster management agent generates a message theme of the registration message packet based on cluster resource data of the edge cluster, so that the message theme can reflect characteristics of related cluster resources, and define key information of the cluster resources. The specific generation of the message theme can be realized by the following modes: obtaining information (namely, cluster name+GVR+resource name) of the edge cluster, resource name of the cluster resource, GVR (Group, version, resource) information of the cluster resource and the like from cluster resource data of the edge cluster; the cluster name+GVR+resource name is combined to generate the message theme of the registration message data packet corresponding to the cluster resource.
Here, the combination may be divided by a slash "/", or by comma "," division ", or by other conforming division, and the specific combination adopted may be determined as needed, which is not limited in this embodiment.
Further, the structure of the message theme may be customized according to the actual requirement, and as an example, the message theme may be designed into a 4-level theme structure, and the specific Topic theme specification is as follows: cluster- { Cluster name/{ built-in/CRD object/Cluster information }/{ resource type }/{ resource instance name }, whose detailed description is shown in Table 1, table 1 is as follows:
TABLE 1 Topic subject specification
After determining the message theme of the registration message data packet, in order to reduce the volume of the message body of the registration message data packet and reduce the data transmission amount in the network, in this embodiment, before writing the cluster resource data of the edge cluster into the message body, the edge cluster management agent first analyzes the cluster resource data of the edge cluster to determine the resource type corresponding to the cluster resource data of the edge cluster. Then, based on the resource types, adopting different processing modes to compress the cluster resource data of the edge cluster to obtain the compressed cluster resource data of the edge cluster; and packaging the compressed cluster resource data of the edge cluster as a message body of the registration message data packet.
In particular, the resource type is used to distinguish whether the cluster resource belongs to a native resource built in the container cloud platform or an external custom resource (Custom Resource Definition, CRD). Typically, the resource type is recorded in a configuration file (yaml file) of the cluster resource of the edge cluster, and the edge cluster management agent can determine the resource type of the cluster resource of the edge cluster by analyzing the configuration file and reading the content of the bond field in the file.
Different types of resources can be compressed in different processing modes. For example, if the cluster resource of the edge cluster is cluster resource data built in by Kubernetes, the access API-Server component requests that the cluster resource data be exported in a data stream of a protobuf coding mode, so as to implement coding of the cluster resource data by using the protobuf coding mode based on the gRPC supported by the Kubernetes system. It should be noted that, the cluster resource data requested to be acquired by the access API-Server component is exported by default in a data stream of a Unicode coding mode, and the Unicode coding mode has lower coding efficiency compared with a protobuf coding mode. That is to say, the data code length after Unicode coding is longer and more network resources are required to be consumed during transmission, so that the data stream in the protobuf coding mode is used for deriving the cluster resource data built in the Kubernetes, which is equivalent to data compression of the cluster resource data. On this basis, the edge cluster management agent may further perform data compression on the cluster resource data encoded by protobuf by using a compression algorithm (specifically, a compression scheme based on a DEFLATE algorithm may be adopted, for example, gzip or zlib). If the CRD resource data is the external CRD resource data of the Kubernetes, the data can only be exported in a Unicode coding mode, and then the data is compressed by an edge cluster management agent by using a compression algorithm.
After the compression of the cluster resource data is completed, the compressed cluster resource data is written into the message body of the registration message data packet to obtain the registration message data packet, and the publishing/subscribing message client in the edge cluster agent publishes the registration message data packet to the center cluster. The size of the data packet can be reduced by compressing the content carried by the message body, so that the bandwidth consumption of the central cluster is further reduced, and the network blocking of the central cluster is avoided.
The publishing/subscribing message middleware in the central cluster receives the registration message data packet published by each edge cluster based on the message protocol of the publishing/subscribing mode, and forwards the registration message data packet to the publishing/subscribing message client of the controller according to the subscription of the publishing/subscribing message client of the controller. The controller is a core control module of the edge cluster management component, and is used for implementing the technical logic of the embodiment. The publish/subscribe message client of the controller is used for connecting the publish/subscribe message middleware to acquire subscribed messages forwarded by the publish/subscribe message middleware.
Further, the controller also includes a resource registry and a resource synchronizer. The resource register is used for calling a publishing/subscribing message client in the controller, acquiring registration message data packages of all edge clusters from the publishing/subscribing message middleware in an asynchronous message subscription mode, analyzing and decoding the registration message data packages to acquire cluster resource data of the edge clusters, then calling a data caching service, and creating cluster resource twin data based on the cluster resource data of the edge clusters by the data caching service.
In this embodiment, the data caching service is used for storing cluster resource twin data of an edge cluster, which essentially belongs to a micro service, and has the main function of providing a unified interface for managing a cluster resource twin data caching pool externally, and other applications/components implement operations of adding, deleting, modifying, searching and the like on the cluster resource twin data in the caching pool by calling the interfaces. The data caching service can concentrate on providing relevant operations of the cluster resource twin data, other applications/components can operate the cluster resource twin data only by calling a unified interface provided by the application/component, and the possible change of the data structure of the cluster resource twin data is not required to be concerned, so that the coupling of the cluster resource twin data and other applications/components is reduced, and the flexibility of the scheme is improved.
Based on the foregoing description, the cluster resource data reported by the edge cluster may be sent to the central cluster for the first time by the edge cluster, or may be sent again after the cluster resource in the edge cluster changes, and for these two situations, the resource register and the data cache service need to be processed differently to complete the creation of the cluster resource twin data. To this end, in some embodiments, an edge cluster management component of the central cluster invokes the data caching service to create cluster resource twinned data by the data caching service based on cluster resource data of the edge cluster contained in the registration message data packet, specifically: the edge cluster management component of the center cluster calls a data cache service to determine whether cluster resource twin data corresponding to cluster resource data of the edge cluster is stored in the data cache service; responding to cluster resource twin data corresponding to cluster resource data of an edge cluster which is not stored in the data caching service, and generating a unique identifier of the cluster resource data of the edge cluster; creating cluster resource twin data corresponding to cluster resource data of the edge cluster; storing the cluster resource twin data into a data caching service in the form of key value pairs, and taking the unique identification as identification information of the cluster resource twin data.
It should be appreciated that for the scenario where the cluster resources of the edge cluster change, the data caching service already stores cluster resource twinning data corresponding to the cluster resources of the edge cluster, whereas for the scenario where the edge cluster first establishes a connection with the central cluster, the data caching service does not store cluster resource twinning data corresponding to the cluster resources of the edge cluster. After the cluster resource data of the edge cluster is acquired by the resource registry in the edge cluster management component, the data cache service is invoked, and the cluster resource of the edge cluster is searched in the cluster resource twin data cache pool by the data cache service so as to determine whether the corresponding cluster resource twin data exists in the cluster resource twin data cache pool.
If the search finds that the cluster resource twin data corresponding to the cluster resource of the edge cluster already exists in the cluster resource twin data cache pool, the registration message data packet is actually an update message data packet, and the content carried in the message body is the latest state information after the sending change of the cluster resource of the edge cluster. The resource registry further calls an API provided by the multi-cluster management component to confirm whether the registration information of the cluster resource of the edge cluster exists in the multi-cluster management component again, and confirms the validity of the registration information in the multi-cluster management component so as to ensure that the registration information of the cluster resource of the edge cluster in the multi-cluster management component is in the latest state.
If the search finds that the cluster resource twin data corresponding to the cluster resources of the edge cluster is not stored in the cluster resource twin data cache pool, the cluster resource twin data needs to be created. At this time, in order to ensure uniqueness of cluster resource twin data corresponding to cluster resources of each edge cluster in a multi-cluster range, a corresponding unique identifier needs to be generated.
One possible implementation manner is that after the resource registry receives the search result of the data cache service and determines that a corresponding unique identifier needs to be generated, the unique identifier is set for the cluster resource data reported by each edge cluster according to the cluster name of the edge cluster and the GVR (Group, version, resource) information of the cluster resource.
Specifically, the resource registry analyzes the registration message data packet, obtains the cluster name of the edge cluster and GVR information of the cluster resource, combines the cluster resource names ("cluster name+GVR+resource name"), and generates a unique identifier of the cluster resource of the edge cluster. The unique identifier is generated by adopting the cluster name of the edge cluster and the GVR information of the cluster resource, so that the unique identifier and the message theme have a one-to-one correspondence, the unique identifier of the cluster resource can be extracted quickly according to the message theme in the message transmission process, and the cached cluster resource twin data can be searched by using the unique identifier so as to be called by the multi-cluster management component.
It should be noted that, without considering the correspondence between the unique identifier and the message theme, the unique identifier of the cluster resource of the edge cluster may be obtained in other manners, for example, the current timestamp, the ID or the name of the cluster resource, the random number and other information may be used to generate the global unique identifier conforming to the UUID identifier, or the cluster resource of the edge cluster may be used to generate a string of unique, fixed-length character sequences as identifiers through a hash function.
Because the cluster resource data reported by each edge cluster exists in a data stream form derived by an API-Server component, in order to facilitate subsequent management and use, cluster resource twin data corresponding to the cluster resource data of the edge cluster is also required to be created, namely, the cluster resource data of the edge cluster in the data stream form is converted into a resource object form of a container cloud platform. It should be appreciated that the clustered resource twinning data generated by the above process only completes the conversion of the data format in memory. In order to persist the resource object, the resource register uses the generated unique identifier as a Key, uses cluster resource twin data corresponding to cluster resource data of the edge cluster as Value, and writes the cluster resource twin data into the data cache service in a data format of a Key Value pair.
To further improve the flexibility of the solution, the edge cluster management component in the central cluster further includes a data gateway, and for this purpose, in some embodiments, the multi-cluster management component sends an operation request to the edge cluster management component, specifically: the multi-cluster management component sends an operation request to a data gateway of the edge cluster management component in a connection-oriented communication mode; the data gateway determines a unique identifier of a target resource and a type of an instruction carried by the operation request based on access address information and instruction parameters recorded by the operation request; and responding the instruction carried by the operation request as the operation instruction, and forwarding the operation request to a resource synchronizer of the edge cluster management component by the data gateway so as to package the operation request by using a message protocol of a publish/subscribe mode by the resource synchronizer to obtain a control message data packet based on the publish/subscribe mode.
In this embodiment, the access address information described in the operation request is the destination address of the operation request, and mainly provides the following functions: (1) Providing control access capability of cluster resources, and particularly providing access addresses for issuing operation instructions to a multi-cluster management component; (2) And providing cluster resource twin data access capability for acquiring cluster resource twin data cached in the data caching service. The access address information is first directed to the data gateway and then forwarded by the data gateway to the actual destination according to the instruction type. Depending on the instruction type, the actual destination may be each edge cluster or may be a data cache service, so that the access address recorded by the operation request may be regarded as a reverse routing address, which achieves the purpose of changing the transmission direction of the operation request of the multi-cluster management component by registering in advance to the multi-cluster management component.
To obtain access address information, in some embodiments, the multi-cluster management component, prior to sending the operation request to the edge cluster management component, further comprises: the method comprises the steps of obtaining access address information of a target resource through accessing an edge cluster resource object, and writing the access address information into a request; wherein the edge cluster resource object is registered in the multi-cluster management component by: the edge cluster management component calls the data gateway after generating the unique identifier of the cluster resource data of the edge cluster so as to generate access address information corresponding to the cluster resource data of the edge cluster by the data gateway; the edge cluster management component generates an edge cluster resource registration request based on the access address information; the multi-cluster management component responds to the edge cluster resource registration request to generate an edge cluster resource object corresponding to the cluster resource data of the edge cluster.
In this embodiment, the edge cluster resource object is an object registered in the multi-cluster management component, and is directly managed by the multi-cluster management component. The edge cluster resource object corresponds to the cluster resource data reported by each edge cluster, and access address information of all cluster resources (including target resources) of each edge cluster is recorded, that is, the cluster resources of each edge cluster are registered with corresponding edge cluster resource objects in the multi-cluster management component, so that when an operation request for the target resources is issued, the multi-cluster management component can acquire the access address information of the edge cluster resource object by accessing the edge cluster resource object, and write the access address information into the operation request.
In order to register the edge cluster resource object to the multi-cluster management component, in this embodiment, after generating the unique identifier of the cluster resource data of the edge cluster, the edge cluster management component invokes the data gateway, generates corresponding access address information according to the unique identifier of the cluster resource data of the edge cluster by the data gateway, records the mapping relationship between the unique identifier of the cluster resource data of the edge cluster and the corresponding access address information, and then returns the access address information to the edge cluster management component.
Further, after the resource register in the edge cluster management component calls the data caching service and successfully creates cluster resource twin data of the edge cluster, an API interface provided by the data gateway is called, and access address information is dynamically added for cluster resources of the edge cluster by the API interface.
After the cluster resources dynamically add the access address information, the resource registrar generates an edge cluster resource registration request based on the access address information and sends the edge cluster resource registration request to the multi-cluster management component.
And after receiving the edge cluster resource registration request, the multi-cluster management component responds to the edge cluster resource registration request and registers access address information associated with cluster resources of the edge cluster into the multi-cluster management component. The multi-cluster management component generates an edge cluster resource object, and access address information corresponding to cluster resources of the edge cluster is recorded in the edge cluster resource object.
When the multi-cluster resource management component calls a data gateway to request to operate the cluster resources of the edge cluster, the request comprises access address information and an operation instruction of the cluster resources, the data gateway analyzes the access address information, and then the unique identification of the cluster resources corresponding to the access address information can be determined, so that the corresponding operation of the cluster resources in the edge cluster is realized.
Under the condition that cluster resource twin data corresponding to each edge cluster are cached, each multi-cluster management component in the central cluster can acquire the latest state information of all cluster resources in each edge cluster from the data caching service without accessing the edge cluster. In order to fully utilize the cluster resource twin data and further reduce network burden, instructions carried in an operation request sent by a multi-cluster management component can be divided into operation instructions and search instructions, wherein the operation instructions refer to instructions such as adding, updating and deleting cluster resources of an edge cluster, and the instructions are essentially actions which are executed on the cluster resources of the edge cluster and possibly cause the cluster resources to change; the search instruction is to query the cluster resources of the edge cluster, and the cluster resources are not changed.
In this embodiment, the type of the instruction carried by the operation request sent by the multi-cluster management component is identified through the data gateway, if the instruction carried by the operation request is an operation instruction, the operation request needs to be issued to the edge cluster, and if the instruction carried by the operation request is a search instruction, the operation request is directly forwarded to the data cache service, and the data cache service obtains the search result from the cluster resource twin data cache pool and returns the search result.
In this embodiment, the data gateway is preset with multiple communication modes for supporting different communication protocols. The multi-cluster management component can send the operation request to the data gateway of the edge cluster management component by using a communication mode facing to the connection such as HTTP protocol or the protobuf protocol based on gRPC, and the like, and no change is needed to be made to the operation request.
After receiving the operation request sent by the multi-cluster management component, the data gateway analyzes the operation request, identifies the type of the instruction carried by the operation request based on the access address information and the instruction parameters recorded by the operation request, and forwards the operation request according to the type of the instruction.
In this embodiment, the instruction parameters recorded in the operation request determine the type of the instruction, and the type of the instruction determines the forwarding direction of the data gateway, thereby determining the transmission path of the operation request. Determining the type of the instruction according to the instruction parameter may be implemented in various manners, in one possible manner, the instruction parameter may be a type of a multi-cluster resource management component, where the data gateway determines the type of the instruction according to the type of the multi-cluster management component that issues the operation instruction, for example, the type of the multi-cluster management component is a management component (e.g. Karmada), the data gateway determines the type of the instruction carried by the operation request as the operation instruction according to the type of the multi-cluster management component is a resource retrieval component (e.g. clusteridia), and the data gateway determines the type of the instruction carried by the operation request as the retrieval instruction according to the type of the instruction. In another implementation manner, the instruction parameter may be a specific operation behavior, where the data gateway determines the type of the instruction according to the content executed by the operation behavior, for example, the operation behavior is specifically CREATE, DELETE operation, and the data gateway determines the type of the instruction carried by the operation request as the operation instruction according to the operation behavior; when the operation behavior is specifically a GET operation, the data gateway determines the type of the instruction carried by the operation request as a retrieval instruction according to the operation behavior. Of course, other ways of determining the type of instruction may be used, which is not limited in this application.
After determining the type of the instruction, if the instruction is a search instruction, the data gateway sends the search instruction and the unique identifier of the target resource to the data caching service, and the data caching service can query cluster resource twin data cached in the data caching service based on the unique identifier of the target resource and return the cluster resource twin data to the multi-cluster management component. Therefore, the search process is completed through the inside of the central cluster, and the search result can be obtained without accessing the edge cluster, so that the network load is further reduced.
If the operation instruction is the operation instruction, the data gateway forwards the operation request to a resource synchronizer of the edge cluster management component for further processing. Here, the data gateway may forward the unique identifier of the target resource and the operation instruction to the resource synchronizer of the edge cluster management component, or may forward the operation request directly to the resource synchronizer of the edge cluster management component.
Step S102, the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode.
It will be appreciated that the generation of control message packets also involves both the subject matter of the message and the body of the message.
The resource synchronizer of the edge cluster management component receives the operation request of the multi-cluster management component forwarded by the data gateway, analyzes the content of the operation request, acquires the unique identification of the target resource, and simultaneously acquires information related to the operation instruction, such as operation behavior and management content.
Based on the foregoing description, in this embodiment, the unique identifier of the target resource includes, but is not limited to, a form of "cluster name+gvr+resource name", and when the unique identifier of the cluster resource is in a form of "cluster name+gvr+resource name", the name, GVR, and the target resource name of the edge cluster where the target resource is located can be determined by analyzing the unique identifier of the target resource, and then a message theme corresponding to the control message packet is generated according to the three information of the name, GVR, and the target resource name of the edge cluster where the target resource is located.
In another embodiment, the message theme may further include content such as a resource type, a resource object definition, and the specific specification may refer to the content of table 1.
It should be understood that, in order to implement unified management, the structure of the message theme of the control message data packet is identical to the structure adopted by the message theme of the registration message data packet, and the structure and specific generation process of the message theme of the control message data packet may refer to the structure and specific generation process adopted by the message theme of the registration message data packet when the edge cluster reports the cluster resource, which are not described in detail herein.
In this embodiment, since the corresponding message topic is generated according to the information such as "cluster name+gvr+resource name", each cluster resource corresponds to a different message topic. In the registration stage of the cluster resource, the publishing/subscribing message client of each edge cluster publishes the cluster resource data to the publishing/subscribing message middleware in the form of a registration message data packet, and subscribes the message theme corresponding to the cluster resource.
In this embodiment, information such as operation behavior and management content is written into a message body of a control message packet including a message theme on the basis of generating the message theme.
As shown in fig. 5, taking the MQTT protocol as an example, the resource synchronizer writes the operation instruction into the message body of the control message packet according to the operation behavior and the specific management content. Specifically, the operation behavior is written into an action field of the message body, the specific management content is correspondingly encoded and data compressed according to different resource types, and then the content field of the message body is written into the content field of the message body, so that a control message data packet based on the MQTT protocol is obtained.
Step S103, the edge cluster management component publishes the control message data packet based on the message transmission mechanism of the publish/subscribe mode, so that the edge cluster receives the control message data packet in an asynchronous subscription mode, and performs management operation on the target resource according to the control message data packet.
Specifically, the resource synchronizer of the edge cluster management component invokes the publish/subscribe message client in the central cluster to publish the control message data package to the publish/subscribe message middleware of the central cluster.
Based on the foregoing description, it can be seen that, in the process of reporting the cluster resources of the edge cluster, the publish/subscribe message client in the edge cluster has subscribed to the message topics related to all the cluster resources belonging to the edge cluster in an asynchronous subscription manner. Therefore, after receiving the control message data packet, the publish/subscribe message middleware forwards the control message data packet to the publish/subscribe message clients in each edge cluster according to the subscription of each edge cluster.
After receiving the control message data packet, the publish/subscribe message client in the edge cluster analyzes the control message data packet by the edge cluster management agent to obtain an operation instruction carried in the control message data packet, converts the operation instruction into a Kubernetes declaration API which can be identified by the Kubernetes system, and finally calls an API-Server component of the edge cluster to execute corresponding management operation on the target resource.
Further, since the execution of the operation instruction may cause a change in the state of the target resource, in order to ensure that the cluster resource twin data in the central cluster can reflect the latest state information of all cluster resources in the edge cluster, in this embodiment, the edge cluster management agent monitors the change condition of the target resource while executing the management operation, that is: the change condition of the cluster resource data of the edge cluster is obtained through a List-watch mechanism, after the cluster resource data of the edge cluster is changed, the cluster resource data of the edge cluster is obtained, the changed cluster resource data is packaged into a registration message data packet, and a publishing/subscribing message client in the edge cluster transmits the registration message data packet to an edge cluster management component of the center cluster based on a message transmission mechanism of a publishing/subscribing mode, so that the update of the cluster resource is realized.
The process of transmitting the registration message data packet to the edge cluster management component of the central cluster by the message delivery mechanism based on the publish/subscribe mode is the same as the registration process of the cluster resource described in any of the foregoing embodiments, and related steps are not repeated herein.
Based on the foregoing description, for the situation that the cluster resources of the edge cluster change, the data cache service already stores cluster resource twin data corresponding to the cluster resources of the edge cluster, the registration message data packet is actually an update message data packet, and the content carried in the message body is the latest state information after the sending change of the cluster resources of the edge cluster, at this time, the edge cluster management component in the central cluster receives the registration message data packet, and still the resource register retrieves the registration message data packet by calling the data cache service to confirm the content of the registration message data packet. After the data caching service searches, it finds that cluster resource twin data corresponding to the cluster resources of the edge cluster already exist in the cluster resource twin data caching pool, and can confirm that the content actually carried by the registration message data packet is the latest state information after the sending change of the cluster resources of the edge cluster, and then starts to execute the updating operation of the cluster resources.
The registration process of the cluster resources can be known, the data cache service of the central cluster manages the cluster resource twin data of each edge cluster, and meanwhile, the multi-cluster management component of the central cluster registers the edge cluster resource objects corresponding to all the cluster resources of each edge cluster, so that the update of the cluster resources comprises two aspects of updating the cluster resource twin data and updating the edge cluster resource objects, and the two aspects of updating are respectively described in detail below.
For the edge cluster resource object in the multi-cluster management component, since access address information of all cluster resources (including target resources) of each edge cluster is mainly recorded, updating of the edge cluster resource object is performed through the resource registry. After receiving the search result returned by the data caching service, the resource registry calls an API provided by the multi-cluster management component to confirm whether the registration information of the cluster resource of the edge cluster exists in the multi-cluster management component again if the cluster resource of the edge cluster is found to be cached in the cluster resource twin data caching pool, namely, confirms whether the edge cluster resource object is registered in the multi-cluster management component, and if so, further uses the access address information contained in the edge cluster resource object to request the data gateway for inquiring the cluster resource data so as to confirm the validity of the access address information. If the access address is found to be wrong, the access address of the cluster resource in the multi-cluster management component is updated, namely a new access address is obtained from the data gateway and written into the corresponding edge cluster resource object of the cluster resource.
The updating of the cluster resource twinning data is performed by a resource synchronizer, comprising the steps of: the resource synchronizer extracts the registration message data packet to acquire the content of the message body; and then processing content fields in the message body, generating cluster resource twin data corresponding to the changed cluster resources, and finally calling the data caching service to write the cluster resource twin data corresponding to the changed cluster resources into the data caching service.
The update of cluster resource twinning data will be described, illustratively, with the message packet shown in fig. 5 as a registration message packet. Referring to fig. 5, the resource synchronizer reads the content of the payload field in the message body from the registration message packet, and performs a deserialization operation on the cluster resource data written in the payload field:
1. reading the content of the proto field, if the content of the proto field is pb, the protobuf decoding strategy is adopted, otherwise, the proto field is a default unicode decoding strategy;
2. reading the content of the action field, and acquiring the operation behavior of the resource, such as GET, PUT, POST, DELETE and other common Restful operation types;
3. according to the resource type definition in the message Topic definition (i.e. message theme) in the variable header of the registration message data packet, creating a non-assigned object of the type in the memory, decoding the content in the field content according to the decoding strategy determined by the previous step, assigning the successfully decoded content to the object for initialization assignment operation, generating cluster resource twin data corresponding to the changed cluster resource, and thus completing the reverse-serialization operation of the whole cluster resource data synchronization information.
The resource synchronizer acquires information such as an edge cluster where the target resource is located, the target resource and the like from the registration message data packet, and invokes the data caching service to inquire whether corresponding cluster resource twin data exist. And if the cluster resource twin data exists, carrying out corresponding operation on the corresponding cluster resource twin data in the data caching service according to the operation behavior and the specific execution content.
Specifically, the resource synchronizer acquires a 'cluster name+GVR+resource name' from a message theme of a registration message data packet, combines the 'cluster name+GVR+resource name' into a unique identifier of a cluster resource, calls a data caching service by taking the unique identifier as a parameter, and compares the unique identifier with unique identifiers of all cluster resource twin data in a cluster resource twin data caching pool after the data caching service receives the unique identifier so as to determine whether corresponding cluster resource twin data exists.
If the cluster name is GET, PUT, POST and the resource operation behavior acquired from the content of the action field is GET, PUT, POST, the cluster name+GVR+resource name is used as a key, an interface of the data caching service is called, and the cluster resource twinning data corresponding to the changed cluster resource obtained by the reverse serialization operation is used for updating the corresponding cluster resource twinning data in the data caching service.
If the cluster name is not available and the resource operation behavior obtained from the content of the action field is GET, PUT, POST, calling an interface of the data caching service by taking the cluster name+GVR+resource name as a key, and directly writing the cluster resource twin data corresponding to the changed cluster resource obtained by the reverse serialization operation into the data caching service.
If the operation is present and the resource operation behavior obtained from the content of the action field is DELETE, the interface of the data caching service is called by taking the cluster name+GVR+resource name as a key, and the corresponding cluster resource twin data in the data caching service is deleted.
If not, and the resource operation behavior obtained in the content of the action field is DELETE, no processing is done.
The above is the update process of cluster resource twin data. By timely updating the cluster resource twin data, the data cached by the cluster resource twin data can be ensured to describe the latest state of the cluster resources of the edge cluster. In addition, cluster resources of the edge clusters are monitored in the process of executing management operation, the cluster resource twin data are updated only after the cluster resources of the edge clusters are monitored to change, and the cluster resource twin data are not updated when the cluster resource twin data are not changed, so that the data cached in the center clusters are guaranteed to be in the latest state, network transmission quantity between the center clusters and the edge clusters is further reduced, and the possibility of network blocking in a weak network environment is reduced.
In summary, in the technical solution of the present application, the edge cluster management component and the multi-cluster management component are deployed in the central cluster, and when the central cluster needs to perform unified management on any resource (i.e., target resource) of the edge cluster, the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to the target resource; the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode; the edge cluster management component publishes the control message data packet based on the message passing mechanism of the publish/subscribe mode, receives the control message data packet in an asynchronous subscription manner by the edge cluster, and performs management operation on the target resource according to the control message data packet. The working principle of the message transmission mode based on the publish/subscribe message protocol can be known that in the process of issuing the operation instruction, the operation instruction is firstly issued to a message queue or a message middleware, then the edge cluster acquires the operation instruction through subscribing the message, and even if the network connection is unstable and the signal transmission quality is low, the network is interrupted, the operation instruction is still stored in the message queue or the message middleware until the network is restored. When the network is restored, the operation instruction can be correctly transmitted without loss, thereby ensuring the reliability of message transmission. On the other hand, the message transfer mode based on the publish/subscribe message protocol does not adopt a connection-oriented communication mode, and the communication mode can decouple the center cluster from the edge cluster, so that the edge cluster management component of the center cluster can continuously execute other tasks after sending out an operation instruction, even if the network of the edge cluster is unstable, each related component of the center cluster can still continuously work without waiting for the response of the edge cluster, and the situation that the network congestion is aggravated due to the fact that the connection is maintained due to the waiting for the response in the weak network environment is avoided. Meanwhile, as the multi-cluster management component and the edge cluster management component are both deployed in the center cluster, the communication between the multi-cluster management component and the edge cluster management component belongs to the internal communication of the center cluster, so that the multi-cluster management component can still send an operation instruction to the edge cluster management component in an original communication mode, the multi-cluster management component is completely unaware of the change of a communication protocol between the center cluster and the edge cluster, when the multi-cluster management component of different types is required to be added, the newly added multi-cluster management component can realize unified management of the edge cluster without any adjustment, and the transparency, compatibility and usability of the scheme are improved.
According to the embodiment, through improving the communication mode between the center cluster and the edge cluster, the message protocol based on the publish/subscribe mode is used for replacing the conventional HTTP protocol and the protobuf protocol, and reliable transmission of information can be ensured under the scene of unstable network connection and low signal transmission quality.
The message data packet based on the publish/subscribe mode has low network bandwidth consumption on the central cluster, reduces the occupation of the network bandwidth of the central cluster, and avoids the network congestion of the central cluster.
Based on the use of publish/subscribe message packets, the data is compressed in the encoding stage prior to data transmission, further reducing the volume of the packets, and further reducing the bandwidth throughput of data communication between the central cluster and the edge clusters.
The time delay of issuing the operation request by the multi-cluster management component is reduced, the response time is shortened, and the management efficiency and the retrieval efficiency of the multi-cluster resources are obviously improved.
Exemplary System
The embodiment of the application provides an edge cluster nano-tube system based on a publish/subscribe message protocol, the system is located in a central cluster, an edge cluster management component and a multi-cluster management component are deployed in the central cluster, as shown in fig. 6, the system comprises: a transmitting unit 601, an encapsulating unit 602, and a transmitting unit 603. Wherein:
A sending unit 601 configured to send an operation request to the edge cluster management component by the multi-cluster management component; the operation request carries a management operation directed to a target resource, which is any resource of the edge cluster.
And the encapsulation unit 602 is configured to encapsulate the operation request by using a message protocol of a publish/subscribe mode by using the edge cluster management component, so as to obtain a control message data packet based on the publish/subscribe mode.
A issuing unit 603 configured for the edge cluster management component to issue control message data packets based on the message passing mechanism of the publish/subscribe mode, to receive the control message data packets by the edge cluster in an asynchronous subscription manner, and to perform management operations on the target resources according to the control message data packets.
The edge cluster nano-tube system based on the publish/subscribe message protocol provided by the embodiment of the application can realize the steps and the flow of the edge cluster nano-tube method based on the publish/subscribe message protocol provided by any embodiment, and achieve the same technical effects, and are not described in detail herein.
Exemplary apparatus
Fig. 7 is a schematic structural diagram of an electronic device provided according to some embodiments of the present application; the electronic device is located in a central cluster, in which an edge cluster management component and a multi-cluster management component are deployed, as shown in fig. 7, and the electronic device includes:
One or more processors 701;
a computer readable storage medium may be configured to store one or more programs 702, the one or more processors 701, when executing the one or more programs 702, implement the steps of:
the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to a target resource, and the target resource is any resource of the edge cluster; the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode; the edge cluster management component publishes the control message data packet based on the message passing mechanism of the publish/subscribe mode, receives the control message data packet in an asynchronous subscription manner by the edge cluster, and performs management operation on the target resource according to the control message data packet.
FIG. 8 is a hardware architecture of an electronic device provided in accordance with some embodiments of the present application; as shown in fig. 8, the hardware structure of the electronic device may include: a processor 801, a communication interface 802, a computer readable storage medium (also referred to as memory) 803, and a communication bus 804.
Wherein the processor 801, the communication interface 802, and the computer-readable storage medium 803 communicate with each other via a communication bus 804.
The electronic device is located in a central cluster, and an edge cluster management component and a multi-cluster management component are deployed in the central cluster.
The computer-readable storage medium 803 may be configured to store one or more programs.
Alternatively, the communication interface 802 may be an interface of a communication module, such as an interface of a GSM module.
Wherein the processor 801 executes one or more programs that implement the steps of: the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to a target resource, and the target resource is any resource of the edge cluster; the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode; the edge cluster management component publishes the control message data packet based on the message passing mechanism of the publish/subscribe mode, receives the control message data packet in an asynchronous subscription manner by the edge cluster, and performs management operation on the target resource according to the control message data packet.
The processor 801 may be a general purpose processor including a central processing unit (central processing unit, CPU for short), a network processor (Network Processor, NP for short), etc., or may be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The electronic device of the embodiments of the present application exist in a variety of forms including, but not limited to:
(1) A mobile communication device: such devices are characterized by mobile communication capabilities and are primarily aimed at providing voice, data communications. Such terminals include: smart phones (e.g., iPhone), multimedia phones, functional phones, and low-end phones, etc.
(2) Ultra mobile personal computer device: such devices are in the category of personal computers, having computing and processing functions, and generally also having mobile internet access characteristics. Such terminals include: PDA, MID, and UMPC devices, etc., such as iPad.
(3) Portable entertainment device: such devices may display and play multimedia content. The device comprises: audio, video players (e.g., iPod), palm game consoles, electronic books, and smart toys and portable car navigation devices.
(4) And (3) a server: the configuration of the server includes a processor, a hard disk, a memory, a system bus, and the like, and the server is similar to a general computer architecture, but is required to provide highly reliable services, and thus has high requirements in terms of processing capacity, stability, reliability, security, scalability, manageability, and the like.
(5) Other electronic devices with data interaction function.
It should be noted that, according to implementation requirements, each component/step described in the embodiments of the present application may be split into more components/steps, and two or more components/steps or part of operations of the components/steps may be combined into new components/steps, so as to achieve the purposes of the embodiments of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine storage medium and to be stored in a local recording medium downloaded through a network, so that the methods described herein may be stored on such software processes on a recording medium using a general purpose computer, a special purpose processor, or programmable or dedicated hardware such as an ASIC or FPGA. It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the publish/subscribe message protocol based edge cluster nanotube approach described herein. Furthermore, when a general purpose computer accesses code for implementing the methods illustrated herein, execution of the code converts the general purpose computer into a special purpose computer for performing the methods illustrated herein.
Those of ordinary skill in the art will appreciate that the elements and method steps of the examples described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment is mainly described in a different point from other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, with reference to the description of the method embodiments in part.
The above-described apparatus and system embodiments are merely illustrative, in which elements illustrated as separate elements may or may not be physically separate, and elements illustrated as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims (9)

1. An edge cluster nano-tube method based on a publish/subscribe message protocol, wherein an edge cluster management component and a multi-cluster management component are deployed in a central cluster, and the method is characterized by comprising the following steps:
the multi-cluster management component sends an operation request to the edge cluster management component; the operation request carries management operation pointing to a target resource, wherein the target resource is any resource of the edge cluster;
the edge cluster management component encapsulates the operation request by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode;
the edge cluster management component publishes the control message data packet based on a message transmission mechanism of a publish/subscribe mode, receives the control message data packet in an asynchronous subscription mode by the edge cluster, and executes the management operation on the target resource according to the control message data packet;
Before the multi-cluster management component sends the operation request to the edge cluster management component, the method further comprises:
invoking a data cache service deployed in the central cluster to acquire the latest state information of the target resource;
determining legal operation behaviors and effective management contents of the target resources according to the latest state information;
generating the operation request based on legal operation behaviors of the target resource and effective management content;
the data caching service is used for storing cluster resource twin data of the edge cluster, the cluster resource twin data is used for describing the latest state information of all cluster resources in the edge cluster, legal operation behaviors of the target resources are operation behaviors which are executed on the target resources and obtain a preset operation result, and effective management contents of the target resources are whether configuration information of the target resources is configured differently in the edge cluster or not.
2. The publish/subscribe message protocol based edge cluster nano tube method according to claim 1, wherein the cluster resource twin data is created by:
an edge cluster management agent in the edge cluster encapsulates cluster resource data of the edge cluster by using a message protocol of a publish/subscribe mode, and generates a registration message data packet corresponding to the cluster resource data of the edge cluster;
Transmitting the registration message data packet to an edge cluster management component of the central cluster based on a message passing mechanism of a publish/subscribe mode;
the edge cluster management component of the central cluster invokes the data caching service to create, by the data caching service, the cluster resource twinning data based on the cluster resource data of the edge cluster contained in the registration message data packet.
3. The edge cluster nano-tube method based on the publish/subscribe message protocol according to claim 2, wherein the edge cluster management agent in the edge cluster encapsulates the cluster resource data of the edge cluster by using the message protocol of the publish/subscribe mode, and generates a registration message data packet corresponding to the cluster resource data of the edge cluster, specifically:
the edge cluster management agent generates a message theme of the registration message data packet based on cluster resource data of the edge cluster;
analyzing the cluster resource data of the edge cluster to determine the resource type corresponding to the cluster resource data of the edge cluster;
based on the resource types, adopting different processing modes to compress the cluster resource data of the edge cluster to obtain the compressed cluster resource data of the edge cluster;
And packaging the cluster resource data of the compressed edge cluster as a message body of the registration message data packet.
4. The edge cluster nano-tube method based on the publish/subscribe message protocol according to claim 2, wherein the edge cluster management component of the central cluster invokes the data caching service to create the cluster resource twinning data by the data caching service based on the cluster resource data of the edge cluster contained in the registration message data packet, in particular:
the edge cluster management component of the center cluster calls the data cache service to determine whether cluster resource twin data corresponding to the cluster resource data of the edge cluster is stored in the data cache service;
responding to the cluster resource twin data corresponding to the cluster resource data of the edge cluster which is not stored in the data cache service, and generating a unique identifier of the cluster resource data of the edge cluster;
creating cluster resource twin data corresponding to the cluster resource data of the edge cluster;
and storing the cluster resource twin data into the data caching service in the form of key value pairs, and taking the unique identification as identification information of the cluster resource twin data.
5. The edge cluster nano-tube method based on the publish/subscribe message protocol according to claim 4, wherein the multi-cluster management component sends an operation request to the edge cluster management component, specifically:
the multi-cluster management component sends the operation request to a data gateway of the edge cluster management component in a connection-oriented communication mode;
the data gateway determines a unique identifier of the target resource and a type of an instruction carried by the operation request based on the access address information and the instruction parameters recorded by the operation request;
and responding to the instruction carried by the operation request as an operation instruction, and forwarding the operation request to a resource synchronizer of the edge cluster management component by the data gateway so as to package the operation request by using a message protocol of a publish/subscribe mode by the resource synchronizer to obtain a control message data packet based on the publish/subscribe mode.
6. The publish/subscribe message protocol based edge cluster nano tube method according to claim 5, wherein the multi-cluster management component further comprises, prior to sending an operation request to the edge cluster management component:
Acquiring access address information of the target resource through accessing an edge cluster resource object, and writing the access address information into the operation request;
wherein the edge cluster resource object is registered in the multi-cluster management component by:
the edge cluster management component invokes the data gateway after generating the unique identifier of the cluster resource data of the edge cluster so as to generate the access address information corresponding to the cluster resource data of the edge cluster by the data gateway;
the edge cluster management component generates an edge cluster resource registration request based on the access address information;
and the multi-cluster management component responds to the edge cluster resource registration request and generates an edge cluster resource object corresponding to cluster resource data of the edge cluster.
7. An edge cluster nanotube system based on a publish/subscribe message protocol, the system being located in a central cluster having an edge cluster management component and a multi-cluster management component deployed therein, comprising:
the sending unit is configured to send an operation request to the edge cluster management component by the multi-cluster management component; the operation request carries management operation pointing to a target resource, wherein the target resource is any resource of the edge cluster;
The encapsulation unit is configured to encapsulate the operation request by the edge cluster management component by using a message protocol of a publish/subscribe mode to obtain a control message data packet based on the publish/subscribe mode;
a issuing unit configured to issue the control message data packet by the edge cluster management component based on a message transmission mechanism of a publish/subscribe mode, to receive the control message data packet by the edge cluster in an asynchronous subscription manner, and to execute the management operation on the target resource according to the control message data packet;
the system further comprises:
the acquisition unit is configured to call a data cache service deployed in the center cluster to acquire the latest state information of the target resource;
a determining unit configured to determine legal operation behavior and effective management content of the target resource according to the latest state information;
a generation unit configured to generate the operation request based on legal operation behavior of the target resource and effective management content;
the data caching service is used for storing cluster resource twin data of the edge cluster, the cluster resource twin data is used for describing the latest state information of all cluster resources in the edge cluster, legal operation behaviors of the target resources are operation behaviors which are executed on the target resources and obtain a preset operation result, and effective management contents of the target resources are whether configuration information of the target resources is configured differently in the edge cluster or not.
8. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the publish/subscribe message protocol based edge cluster nanotube method according to any of claims 1 to 6.
9. An electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor, the processor implementing the publish/subscribe message protocol based edge cluster nanotube method according to any of claims 1 to 6 when the program is executed.
CN202311122427.4A 2023-08-31 2023-08-31 Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol Active CN117135156B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311122427.4A CN117135156B (en) 2023-08-31 2023-08-31 Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311122427.4A CN117135156B (en) 2023-08-31 2023-08-31 Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol

Publications (2)

Publication Number Publication Date
CN117135156A CN117135156A (en) 2023-11-28
CN117135156B true CN117135156B (en) 2024-03-26

Family

ID=88852500

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311122427.4A Active CN117135156B (en) 2023-08-31 2023-08-31 Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol

Country Status (1)

Country Link
CN (1) CN117135156B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114637599A (en) * 2022-02-28 2022-06-17 阿里巴巴(中国)有限公司 Cloud resource management method and device, electronic equipment and readable storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220116755A1 (en) * 2021-06-07 2022-04-14 Miltiadis Filippou Multi-access edge computing (mec) vehicle-to-everything (v2x) interoperability support for multiple v2x message brokers

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114637599A (en) * 2022-02-28 2022-06-17 阿里巴巴(中国)有限公司 Cloud resource management method and device, electronic equipment and readable storage medium

Also Published As

Publication number Publication date
CN117135156A (en) 2023-11-28

Similar Documents

Publication Publication Date Title
US11411897B2 (en) Communication method and communication apparatus for message queue telemetry transport
US10645181B2 (en) Meta broker for publish-subscribe-based messaging
CN108028833B (en) NAS data access method, system and related equipment
US7640357B2 (en) Transmitting enterprise messages based on buffer sizes
US7519669B2 (en) Prioritizing producers and consumers of an enterprise messaging system
US8095598B2 (en) Methods and apparatus for subscribing/publishing messages in an enterprising computing environment
WO2020207091A1 (en) Management client, and device monitoring system and method
US20170257280A1 (en) Reliably Updating a Messaging System
US8719780B2 (en) Application server with a protocol-neutral programming model for developing telecommunications-based applications
US20030182464A1 (en) Management of message queues
CN106663033B (en) System and method for supporting a wraparound domain and proxy model and updating service information for cross-domain messaging in a transactional middleware machine environment
US20050262205A1 (en) Delivering messages in an enterprise messaging system using message selector hierarchy
US10303529B2 (en) Protocol for communication of data structures
US20020046304A1 (en) Dynamic class loading
US10536560B2 (en) System and method for implementing augmented object members for remote procedure call
CN113556359A (en) Communication protocol conversion method, device, system and gateway device
CN113794652A (en) Data processing method and device, electronic equipment and storage medium
US8316083B2 (en) System and method for client interoperability
CN116684468B (en) Data processing method, device, equipment and storage medium
CN117135156B (en) Edge cluster nano-tube method, system, computer readable storage medium and electronic equipment based on publish/subscribe message protocol
CN116405547A (en) Message pushing method and device, processor, electronic equipment and storage medium
US20030149771A1 (en) Remote services system back-channel multicasting
US11861386B1 (en) Application gateways in an on-demand network code execution system
CN113973135A (en) Data caching processing method and device, caching grid platform and storage medium
US20230315543A1 (en) Tightly coupled parallel applications on a serverless computing system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant