WO2024103983A1

WO2024103983A1 - Distributed dispatch system deployment method, system and apparatus, distributed dispatch system dispatch method, system and apparatus, and medium

Info

Publication number: WO2024103983A1
Application number: PCT/CN2023/122225
Authority: WO
Inventors: 颜浩
Original assignee: 中兴通讯股份有限公司
Priority date: 2022-11-15
Filing date: 2023-09-27
Publication date: 2024-05-23
Also published as: CN118101757A

Abstract

Provided in the embodiments of the present disclosure are a distributed dispatch system deployment method, system and apparatus, a distributed dispatch system dispatch method, system and apparatus, and a medium. By means of the present disclosure, a target node is deployed for a target host in a service cluster, a real-time message dispatch parameter is configured for the target node, and the target node, which is configured with the real-time message dispatch parameter, is taken as a forwarding node, such that the target host in the service cluster performs real-time message dispatch by means of the forwarding node.

Description

一种分布式分发***部署、分发方法、***、装置及介质A distributed distribution system deployment, distribution method, system, device and medium

相关公开的交叉引用Cross-references to related publications

本公开基于2022年11月15日提交的发明名称为“一种分布式分发***部署、分发方法、***、装置及介质”的中国专利公开202211428702.0，并且要求该专利公开的优先权，通过引用将其所公开的内容全部并入本公开。This disclosure is based on Chinese Patent Publication 202211428702.0, filed on November 15, 2022, with the invention name “A distributed distribution system deployment, distribution method, system, device and medium”, and claims the priority of the patent disclosure, and all its disclosed contents are incorporated into this disclosure by reference.

技术领域Technical Field

本公开实施例涉及通信领域，具体而言，涉及一种分布式分发***部署、分发方法、***、装置及介质。The embodiments of the present disclosure relate to the field of communications, and in particular, to a distributed distribution system deployment, distribution method, system, device and medium.

背景技术Background technique

海量数据的实时流式业务***，一般是采用流水线式的组网架构，实时数据将按照流水线，从一个上游计算集群，分发到一个或多个下游计算集群继续进行处理，并最终完成数据的分析统计。在计算框架中，两个计算集群可以同时位于一个局域网内部，或者位于两个通过广域网互联的异地网络之间。Real-time streaming business systems for massive data generally adopt a pipeline-style networking architecture. Real-time data will be distributed from an upstream computing cluster to one or more downstream computing clusters for further processing according to the pipeline, and finally complete the data analysis and statistics. In the computing framework, two computing clusters can be located in a local area network at the same time, or between two remote networks connected by a wide area network.

目前主流的消息分发***，kafka等***组网需要依托于kafka集群的中心化部署，束缚了分布式业务集群的部署。RabbitMQ等***组网，虽然有联邦等组网形式，但缺少联邦组网统一部署管理的功能，在应对大规模组网场景时，缺少统一的部署和管理的手段，分布式组网能力不足。The current mainstream message distribution system, Kafka and other system networking needs to rely on the centralized deployment of Kafka clusters, which restricts the deployment of distributed business clusters. RabbitMQ and other system networking, although there are federation and other networking forms, lack the function of unified deployment and management of federation networking. When dealing with large-scale networking scenarios, there is a lack of unified deployment and management methods, and the distributed networking capabilities are insufficient.

发明内容Summary of the invention

本公开实施例提供一种分布式分发***部署、分发方法、***、装置及介质，以至少解决相关技术中分布式组网能力不足的问题。The embodiments of the present disclosure provide a distributed distribution system deployment, distribution method, system, device and medium to at least solve the problem of insufficient distributed networking capability in related technologies.

根据本公开的一个实施例，提供了一种分布式分发***部署方法，包括：According to an embodiment of the present disclosure, a distributed distribution system deployment method is provided, including:

获取实时消息分发参数；Get real-time message distribution parameters;

给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发；Deploy a target node to a target host in a service cluster, configure the real-time message distribution parameter on the target node, and use the target node configured with the real-time message distribution parameter as a forwarding node, so that the target host in the service cluster performs real-time message distribution through the forwarding node;

根据本公开的另一个实施例，提供了一种分布式实时消息分发方法，包括：According to another embodiment of the present disclosure, a distributed real-time message distribution method is provided. include:

获取待分发的实时消息；Get real-time messages to be distributed;

确定部署了转发节点的业务集群；所述转发节点配置有实时消息分发参数；Determining a service cluster in which a forwarding node is deployed; the forwarding node is configured with real-time message distribution parameters;

通过所述转发节点按照所述实时消息分发参数对所述实时消息进行分发。The real-time message is distributed by the forwarding node according to the real-time message distribution parameter.

根据本公开的另一个实施例，提供了一种分布式实时消息分发***，包括：转发节点、业务集群和拓扑管理***；According to another embodiment of the present disclosure, there is provided a distributed real-time message distribution system, including: a forwarding node, a service cluster, and a topology management system;

所述拓扑管理***，设置为获取实时消息分发参数；给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发；The topology management system is configured to obtain real-time message distribution parameters; deploy a target node to a target host in a service cluster, configure the real-time message distribution parameters on the target node, and use the target node configured with the real-time message distribution parameters as a forwarding node, so that the target host in the service cluster performs real-time message distribution through the forwarding node;

根据本公开的另一个实施例，提供了一种分布式分发***部署装置，包括：According to another embodiment of the present disclosure, a distributed distribution system deployment device is provided, including:

第一获取模块，设置为获取实时消息分发参数；A first acquisition module, configured to acquire real-time message distribution parameters;

配置模块，设置为给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发。The configuration module is configured to deploy a target node to a target host in a business cluster, configure the real-time message distribution parameters on the target node, and use the target node configured with the real-time message distribution parameters as a forwarding node, so that the target host in the business cluster performs real-time message distribution through the forwarding node.

根据本公开的另一个实施例，提供了一种分布式实时消息分发装置，包括：According to another embodiment of the present disclosure, a distributed real-time message distribution device is provided, comprising:

第二获取模块，设置为获取待分发的实时消息；A second acquisition module is configured to acquire real-time messages to be distributed;

确定模块，设置为确定部署了转发节点的业务集群；所述转发节点配置有实时消息分发参数；A determination module, configured to determine a service cluster in which a forwarding node is deployed; the forwarding node is configured with real-time message distribution parameters;

分发模块，设置为通过所述转发节点按照所述实时消息分发参数对所述实时消息进行分发。The distribution module is configured to distribute the real-time message through the forwarding node according to the real-time message distribution parameter.

根据本公开的又一个实施例，还提供了一种存储介质，所述存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。According to another embodiment of the present disclosure, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

根据本公开的又一个实施例，还提供了一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行上述任一项方法实施例中的步骤。 According to another embodiment of the present disclosure, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

此处所说明的附图用来提供对本公开的进一步理解，构成本公开的一部分，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。在附图中：The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

图1是本公开实施例提供的电子装置结构示意图；FIG1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

图2是本公开实施例提供的一种分布式分发***的***架构图；FIG2 is a system architecture diagram of a distributed distribution system provided by an embodiment of the present disclosure;

图3是本公开实施例提供的一种分布式分发***部署方法的流程示意图；FIG3 is a flow chart of a distributed distribution system deployment method provided by an embodiment of the present disclosure;

图4是本公开实施例提供的一种分布式实时消息分发方法的流程示意图；FIG4 is a flow chart of a distributed real-time message distribution method provided by an embodiment of the present disclosure;

图5是本公开实施例提供的另一种分布式分发***的***架构图；FIG5 is a system architecture diagram of another distributed distribution system provided by an embodiment of the present disclosure;

图6是本公开实施例提供的一种转发节点的结构示意图；FIG6 is a schematic diagram of the structure of a forwarding node provided by an embodiment of the present disclosure;

图7是本公开实施例提供的另一种分布式分发***的***架构图；FIG7 is a system architecture diagram of another distributed distribution system provided by an embodiment of the present disclosure;

图8是本公开实施例提供的一种分布式分发***的***架构图；FIG8 is a system architecture diagram of a distributed distribution system provided by an embodiment of the present disclosure;

图9是本公开实施例提供的另一种分布式分发***部署方法的流程示意图；FIG9 is a flow chart of another distributed distribution system deployment method provided by an embodiment of the present disclosure;

图10是本公开实施例提供的一种集群拓扑展示的示意图；FIG10 is a schematic diagram of a cluster topology display provided by an embodiment of the present disclosure;

图11是本公开实施例提供的一种非集群拓扑展示的示意图；FIG11 is a schematic diagram of a non-cluster topology display provided by an embodiment of the present disclosure;

图12是本公开实施例提供的一种拓扑管理节点展开示意图；FIG12 is a schematic diagram of a topology management node deployment provided by an embodiment of the present disclosure;

图13是本公开实施例提供的一种拓扑管理节点配置修改示意图；FIG13 is a schematic diagram of a topology management node configuration modification provided by an embodiment of the present disclosure;

图14是本公开实施例提供的一种分布式分发***部署装置的结构示意图；FIG14 is a schematic diagram of the structure of a distributed distribution system deployment device provided by an embodiment of the present disclosure;

图15是本公开实施例提供的一种分布式实时消息分发装置的结构示意图。FIG. 15 is a schematic diagram of the structure of a distributed real-time message distribution device provided in an embodiment of the present disclosure.

具体实施方式Detailed ways

下文中将参考附图并结合实施例来详细说明本公开。需要说明的是，在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互组合。The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present disclosure may be combined with each other without conflict.

需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

在目前，kafka是一种常见的消息分发***，它是一套高性能、低延时的消息分发***，但是，当前的kafka集群在实际应用中，存在如下不足：At present, Kafka is a common message distribution system. It is a high-performance, low-latency However, the current Kafka cluster has the following shortcomings in practical applications:

kafka集群不能很好的应对跨网段多计算节点的组网方式，考虑到多计算节点之间网络环境、计算容量等的限制，并不能支持统一的kafka/zookeeper集群的组网条件。The Kafka cluster cannot cope well with the networking mode of multiple computing nodes across network segments. Considering the limitations of the network environment and computing capacity between multiple computing nodes, it cannot support the unified networking conditions of the Kafka/Zookeeper cluster.

生产者(producer)采用向kafka中推送数据的方式发送数据，而kafka的处理能力和topic分区数量有直接关系。当kafka对应topic分区数量较少时，producer推送能力可能超过当前topic的接收能力，导致producer发送拥塞或数据丢弃的情况。Producers send data by pushing data to Kafka, and Kafka's processing capacity is directly related to the number of topic partitions. When the number of Kafka corresponding topic partitions is small, the producer's push capacity may exceed the current topic's receiving capacity, resulting in producer sending congestion or data discard.

消费者(consumer)采用从kafka中拉取数据的方式接收数据，如果consumer数量较少，必然会造成kafka中未取消息的堆积。Consumers receive data by pulling data from Kafka. If the number of consumers is small, unretrieved messages will inevitably accumulate in Kafka.

kafka中的转发节点(broker)集群在接收到数据时，需要先写成文件。在发送数据时，需要读取文件。虽然kafka在读写文件时采用了一些优化性能的处理方式，比如：顺序写、零拷贝读、页缓存等，但数据落盘后再读取始终存在性能消耗。所以，对kafka性能会有影响，也影响了集群的实时性。When the forwarding node (broker) cluster in Kafka receives data, it needs to write it into a file first. When sending data, it needs to read the file. Although Kafka uses some processing methods to optimize performance when reading and writing files, such as sequential writing, zero-copy reading, page cache, etc., there is always performance consumption when reading data after it is written to the disk. Therefore, it will affect the performance of Kafka and the real-time performance of the cluster.

kafka需要搭建broker集群，而且如果生产数据较多时，集群规模不能太小，否则，性能很难保障。这必然造成需要构建庞大、臃肿的broker集群和zookeeper集群。Kafka requires the construction of a broker cluster, and if there is a lot of production data, the cluster size cannot be too small, otherwise, performance is difficult to guarantee. This inevitably leads to the need to build a large and bloated broker cluster and zookeeper cluster.

kafka向consumer分发数据时，分发方式比较单一，在业务分流要求复杂的场景中，需要应用***自己提前规划好流量，再依次导入到相应的kafka topic队列中，才能实现数据业务的复杂分流。When Kafka distributes data to consumers, the distribution method is relatively simple. In scenarios with complex business diversion requirements, the application system needs to plan the traffic in advance and then import it into the corresponding Kafka topic queues in sequence to achieve complex data diversion.

kafka的主题(topic)和分区(patition)规划比较复杂，面对复杂组网场景，管理维护成本比较高，需要有丰富经验的人员才能实现。kafka集群扩容成本也较高，无法弹性扩容。The planning of Kafka topics and partitions is relatively complex. In complex networking scenarios, the management and maintenance costs are relatively high, and only experienced personnel are required to implement them. The cost of expanding the Kafka cluster is also high, and it cannot be expanded elastically.

RabbitMQ是另外一种比较常见的消息分发***，构建在高级消息队列协议(AMQP)协议基础上。该分发***存在以下一些缺点：RabbitMQ is another common message distribution system, built on the Advanced Message Queuing Protocol (AMQP). This distribution system has the following disadvantages:

RabbitMQ中的规则定义比较简单，只能实现比较简单的消息分发功能，如上所述。在复杂业务分流场景下，功能和性能上都有很大局限。The rule definition in RabbitMQ is relatively simple, and can only implement relatively simple message distribution functions, as mentioned above. In complex business diversion scenarios, there are great limitations in terms of function and performance.

RabbitMQ集群根据队列是否实际部署在某个broker节点，会将broker节点分为宿主节点和非宿主节点。客户端在与RabbitMQ集群进行通信时，可能链接的是非宿主节点，会触发节点之间创建临时通道和内部节点间消息路由，降低消息传输的效率。RabbitMQ cluster divides broker nodes into host nodes and non-host nodes according to whether the queue is actually deployed on a broker node. When the client communicates with the RabbitMQ cluster, it may connect to a non-host node, which will trigger the creation of temporary channels between nodes and message routing between internal nodes, reducing the efficiency of message transmission.

上述部署方式实际也会造成单点故障，RabbitMQ一般采用镜像模式来应对。镜像模式将队列分为主队列(Master)和若干从队列(Slave)，如果是同步方式，将会进一步引入时延。如果是异步方式，其实也存在数据丢失的风险。The above deployment method will actually cause a single point of failure. RabbitMQ generally uses the mirror mode. To deal with it. The mirror mode divides the queue into a master queue (Master) and several slave queues (Slave). If it is a synchronous mode, it will further introduce latency. If it is an asynchronous mode, there is actually a risk of data loss.

RabbitMQ单机、集群、联邦等组网模式中的交换器、队列、规则等规划管理复杂，特别在大的集群组网时，管理维护成本会很高。The planning and management of switches, queues, rules, etc. in RabbitMQ stand-alone, cluster, federation and other networking modes are complex, especially in large cluster networking, the management and maintenance costs will be very high.

RabbitMQ集群将broker节点分为：磁盘节点和内存节点，其中磁盘节点实现了元数据的持久化，内存节点有更高的性能，一个rabbitMQ集群至少要有一个磁盘节点。如果磁盘节点宕机，集群将只能路由消息，不能再进行业务调整。RabbitMQ cluster divides broker nodes into disk nodes and memory nodes. Disk nodes implement metadata persistence, and memory nodes have higher performance. A RabbitMQ cluster must have at least one disk node. If the disk node goes down, the cluster can only route messages and can no longer adjust services.

如前所述，当前消息分发***普遍存在如下问题：As mentioned above, the current message distribution system generally has the following problems:

分布式组网能力不足，目前kafka等***组网，需要依托于kafka集群的中心化部署，束缚了分布式业务集群的部署。RabbitMQ等***组网，虽然有联邦等组网形式，但缺少联邦组网统一部署管理的功能，在应对大规模组网场景时，缺少统一的部署和管理的手段。Insufficient distributed networking capabilities. Currently, the networking of systems such as Kafka needs to rely on the centralized deployment of Kafka clusters, which restricts the deployment of distributed business clusters. Although RabbitMQ and other system networking have federation and other networking forms, they lack the function of unified deployment and management of federation networking. When dealing with large-scale networking scenarios, there is a lack of unified deployment and management methods.

消息分发***业务处理能力比较单一，目前主流的kafka、RabbitMQ等***自身只能实现比较简单的数据分流功能。The business processing capabilities of the message distribution system are relatively simple. Currently, mainstream systems such as Kafka and RabbitMQ can only implement relatively simple data diversion functions.

消息分发***管理维护成本高，主流的kafka或RabbitMQ等***，虽然有自己的web管理维护界面。对管理维护人员要求比较高，复杂组网场景中，部署和维护成本很高。The management and maintenance costs of message distribution systems are high. Although mainstream systems such as Kafka or RabbitMQ have their own web management and maintenance interfaces, they have high requirements for management and maintenance personnel. In complex networking scenarios, the deployment and maintenance costs are very high.

目前主流的消息分发***，对于实时流的处理，还是存在一定局限。特别是在组网场景复杂的情况下，需要通过专用服务器的堆砌来实现性能的提升，往往还需要依托比较丰富且有技巧的业务部署方式，才能达到性能提升的效果，成本和收益不成比例。The current mainstream message distribution system still has certain limitations in processing real-time streams. Especially in complex networking scenarios, it is necessary to stack dedicated servers to achieve performance improvement, and often it is necessary to rely on relatively rich and skillful business deployment methods to achieve performance improvement, and the cost and benefits are disproportionate.

本公开至少部分的解决了上述问题，下面从具体的实施例对本公开作出详细说明。The present disclosure at least partially solves the above problems, and the present disclosure is described in detail below based on specific embodiments.

实施例1Example 1

本公开实施例一所提供的方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在移动终端上为例，图1是本公开实施例的一种分布式分发***部署方法或分布式实时消息分发方法的移动终端的硬件结构框图。如图1所示，移动终端10可以包括一个或多个(图1中仅示出一个)处理器102(处理器102可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)和设置为存储数据的存储器104，可选地，上述移动终端还可以包括设置为通信功能的传输设备106以及输入输出设备108。本领域普通技术人员可以理解，图1所示的结构仅为示意，其并不对上述移动终端的结构造成限定。例如，移动终端10还可包括比图1中所示更多或者更少的组件，或者具有与图1所示不同的配置。The method embodiment provided in the first embodiment of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, FIG1 is a hardware structure block diagram of a mobile terminal of a distributed distribution system deployment method or a distributed real-time message distribution method of an embodiment of the present disclosure. As shown in FIG1 , the mobile terminal 10 may include one or more (only one is shown in FIG1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 configured to store data. Optionally, the mobile terminal may also include a transmission device 106 configured to have a communication function and an input/output device 108. It will be understood by those skilled in the art that the structure shown in FIG1 is only for illustration purposes. For example, the mobile terminal 10 may include more or fewer components than those shown in FIG. 1 , or may have a configuration different from that shown in FIG. 1 .

存储器104可设置为存储计算机程序，例如，应用软件的软件程序以及模块，如本公开实施例中的分布式分发***部署方法或分布式实时消息分发方法对应的计算机程序，处理器102通过运行存储在存储器104内的计算机程序，从而执行各种功能应用以及数据处理，即实现上述的方法。存储器104可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器104可进一步包括相对于处理器102远程设置的存储器，这些远程存储器可以通过网络连接至移动终端10。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 104 may be configured to store computer programs, for example, software programs and modules of application software, such as the computer programs corresponding to the distributed distribution system deployment method or the distributed real-time message distribution method in the embodiment of the present disclosure. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, to implement the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

传输装置106设置为经由一个网络接收或者发送数据。上述的网络具体实例可包括移动终端10的通信供应商提供的无线网络。在一个实例中，传输装置106包括一个网络适配器(Network Interface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置106可以为射频(Radio Frequency，简称为RF)模块，其设置为通过无线方式与互联网进行通讯。The transmission device 106 is configured to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is configured to communicate with the Internet wirelessly.

本公开实施例可以运行于图2所示的***架构上，如图2所示，该***架构包括：拓扑管理***201、业务集群202和转发节点203，其中，拓扑管理***201，设置为获取实时消息分发参数；给业务集群202中的目标主机部署目标节点，将实时消息分发参数配置于目标节点，并将配置了实时消息分发参数的目标节点作为转发节点203，以使业务集群202中的目标主机通过转发节点203进行实时消息分发。The embodiment of the present disclosure can run on the system architecture shown in Figure 2. As shown in Figure 2, the system architecture includes: a topology management system 201, a business cluster 202 and a forwarding node 203, wherein the topology management system 201 is configured to obtain real-time message distribution parameters; deploy target nodes to target hosts in the business cluster 202, configure the real-time message distribution parameters on the target nodes, and use the target nodes configured with the real-time message distribution parameters as forwarding nodes 203, so that the target hosts in the business cluster 202 perform real-time message distribution through the forwarding nodes 203.

在本实施例中，业务集群可以是核心网，也可以是核心网下的基站子***或网络子***，不作限制。业务集群可以对核心网话单进行消息分发，其中，话单种类可以不作限制，比如，可以是原始话单，可以是原始话单的关联话单，还可以是基于关联话单生成的话单，还可以是数据过滤后的话单等等。In this embodiment, the service cluster may be a core network, or a base station subsystem or a network subsystem under the core network, without limitation. The service cluster may distribute messages to the core network call records, wherein the types of call records may not be limited, for example, the original call record, the associated call record of the original call record, the call record generated based on the associated call record, the call record after data filtering, and so on.

该***架构可以是分布式实时消息分发***，也可简称为分布式分发***，该***架构可以应用在分布式消息总线***，通过在分布式消息总线***中，结合web和拓扑化管理方式，可以实现分布式消息总线***的高效管理和使用。该***架构也可以应用在分布式多集群业务***，通过方案的扩展，理论上可以将涵盖业务***和分布式消息总线***在内的分布式多集群***，都可结合web和拓扑化管理方式，提升分布式多集群***的管理效率。对该***架构，在下述实施例中还会进行详细说明。 The system architecture can be a distributed real-time message distribution system, or simply a distributed distribution system. The system architecture can be applied to a distributed message bus system. By combining the web and topological management methods in the distributed message bus system, efficient management and use of the distributed message bus system can be achieved. The system architecture can also be applied to a distributed multi-cluster business system. Through the expansion of the solution, in theory, the distributed multi-cluster system including the business system and the distributed message bus system can be combined with the web and topological management methods to improve the management efficiency of the distributed multi-cluster system. The system architecture will be described in detail in the following embodiments.

在本实施例中提供了一种运行于上述移动终端或***架构的分布式分发***部署方法，图3是根据本公开实施例的分布式分发***部署方法的流程图，如图3所示，该流程包括如下步骤：In this embodiment, a distributed distribution system deployment method running on the above mobile terminal or system architecture is provided. FIG3 is a flow chart of the distributed distribution system deployment method according to an embodiment of the present disclosure. As shown in FIG3, the process includes the following steps:

步骤S302，获取实时消息分发参数；Step S302, obtaining real-time message distribution parameters;

步骤S304，给业务集群中的目标主机部署目标节点，将实时消息分发参数配置于目标节点，并将配置了实时消息分发参数的目标节点作为转发节点，以使业务集群中的目标主机通过转发节点进行实时消息分发。Step S304, deploying target nodes to target hosts in the service cluster, configuring real-time message distribution parameters on the target nodes, and using the target nodes configured with real-time message distribution parameters as forwarding nodes, so that the target hosts in the service cluster perform real-time message distribution through the forwarding nodes.

通过上述步骤，给业务集群中的目标主机部署目标节点，给目标节点配置实时消息分发参数，以使部署了转发节点的各个业务集群通过转发节点进行实时消息分发。本公开将转发节点与业务集群的主机共机部署，使分布式消息分发***可以和业务集群统一部署，不需要部署专门的消息分发集群，解决了分布式组网能力不足的问题，真正实现分布式的组网，可避免***出现局部网络带宽、局部计算存储瓶颈，进而达到了提升分布式分发***性能的效果。Through the above steps, the target node is deployed to the target host in the business cluster, and the real-time message distribution parameters are configured for the target node, so that each business cluster deployed with the forwarding node can perform real-time message distribution through the forwarding node. The present invention deploys the forwarding node and the host of the business cluster on the same machine, so that the distributed message distribution system can be deployed uniformly with the business cluster, and there is no need to deploy a special message distribution cluster, which solves the problem of insufficient distributed networking capabilities and truly realizes distributed networking, which can avoid local network bandwidth and local computing and storage bottlenecks in the system, thereby achieving the effect of improving the performance of the distributed distribution system.

需要说明的是，转发节点即为配置了实时消息分发参数的目标节点，或者说，配置实时消息分发参数的目标节点可作为转发节点。如无特殊说明，下文中的转发节点既可指未配置实时消息分发参数的目标节点，也可指配置了实时消息分发参数的转发节点。It should be noted that the forwarding node is the target node configured with the real-time message distribution parameters, or in other words, the target node configured with the real-time message distribution parameters can be used as a forwarding node. Unless otherwise specified, the forwarding node in the following text can refer to both the target node not configured with the real-time message distribution parameters and the forwarding node configured with the real-time message distribution parameters.

可选地，上述步骤的执行主体可以为***架构中的拓扑管理***，但不限于此。Optionally, the execution entity of the above steps may be a topology management system in the system architecture, but is not limited thereto.

部署模式可以为集群部署，也可以为非集群部署。The deployment mode can be cluster deployment or non-cluster deployment.

若部署模式为集群部署，则获取实时消息分发参数，包括：If the deployment mode is cluster deployment, obtain the real-time message distribution parameters, including:

从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型；目标配置模型设置为给业务集群待部署的目标节点配置实时消息分发参数；Determine a target configuration model corresponding to the business cluster to be deployed in the cluster from the preset configuration model; the target configuration model is set to configure real-time message distribution parameters for the target node to be deployed in the business cluster;

相应地，给业务集群中的目标主机部署目标节点，将实时消息分发参数配置于目标节点，并将配置了实时消息分发参数的目标节点作为转发节点，以使业务集群中的目标主机通过转发节点进行实时消息分发，包括：Accordingly, a target node is deployed to a target host in the service cluster, a real-time message distribution parameter is configured on the target node, and the target node configured with the real-time message distribution parameter is used as a forwarding node, so that the target host in the service cluster performs real-time message distribution through the forwarding node, including:

给业务集群中的各个目标主机分别部署目标节点，根据目标配置模型，给各个目标节点分别配置实时消息分发参数，并将配置了实时消息分发参数的目标节点作为转发节点，以使业务集群通过转发节点进行实时消息分发。Deploy target nodes for each target host in the business cluster, configure real-time message distribution parameters for each target node according to the target configuration model, and use the target node configured with real-time message distribution parameters as a forwarding node so that the business cluster can perform real-time message distribution through the forwarding node.

在集群部署模式下，可通过目标配置模型，对相同业务集群下的目标主机快速的配置实时消息分发参数，提高转发节点的部署速度及部署效率。 In cluster deployment mode, the target configuration model can be used to quickly configure real-time message distribution parameters for target hosts in the same business cluster, thereby improving the deployment speed and efficiency of forwarding nodes.

可选地，从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型之前，方法还包括：确定待集群部署的至少一个业务集群。Optionally, before determining the target configuration model corresponding to the service cluster to be cluster-deployed from the preset configuration model, the method further includes: determining at least one service cluster to be cluster-deployed.

可选地，从预设的配置模型中，分别确定每个业务集群对应的目标配置模型之前，方法还包括：获取配置模型。Optionally, before respectively determining the target configuration model corresponding to each service cluster from the preset configuration models, the method further includes: acquiring the configuration model.

其中，配置模型的生成过程包括：获取所有业务集群的组网关系；根据组网关系，确定每一个业务集群对应的目标配置模型，将所有目标配置模型的集合作为配置模型。The generation process of the configuration model includes: obtaining the networking relationship of all business clusters; determining the target configuration model corresponding to each business cluster according to the networking relationship, and taking the set of all target configuration models as the configuration model.

配置模型，也可称为配置模板，通过配置模型可以对每一个业务集群统一配置该业务集群对应的目标配置模型。每一个业务集群对应一个目标配置模型，该目标配置模型中包括给该业务集群中的主机部署的转发节点配置的实时消息分发参数。The configuration model, also known as a configuration template, can be used to uniformly configure the target configuration model corresponding to each business cluster for each business cluster. Each business cluster corresponds to a target configuration model, which includes real-time message distribution parameters configured for the forwarding nodes deployed on the hosts in the business cluster.

实时消息分发参数，可以包括集群名、集群的上下游关系和转发节点的公共配置信息，公共配置信息包括服务名、服务端口和路由规则等，路由规则包括接收缓存大小、接收文件缓存大小、发送缓存、发送缓存大小、路由分发规则等相关的配置参数。Real-time message distribution parameters may include cluster name, upstream and downstream relationships of clusters, and public configuration information of forwarding nodes. Public configuration information includes service name, service port, and routing rules. Routing rules include receiving cache size, receiving file cache size, sending cache, sending cache size, routing distribution rules and other related configuration parameters.

其中，集群名可以是业务集群名，由于转发节点是配置到业务集群的各个主机当中，该集群名也可以表示转发节点集群名。The cluster name may be a business cluster name. Since the forwarding nodes are configured in each host of the business cluster, the cluster name may also represent the forwarding node cluster name.

可选地，给业务集群中的各个目标主机分别部署目标节点，并根据目标配置模型，给各个目标节点分别配置实时消息分发参数之后，方法还包括：获取部署调整方案；部署调整方案包括扩容或缩容。Optionally, after deploying target nodes to each target host in the business cluster and configuring real-time message distribution parameters to each target node according to the target configuration model, the method further includes: obtaining a deployment adjustment plan; the deployment adjustment plan includes expansion or reduction.

若部署调整方案为扩容，具体的扩容方案包括：获取待扩容的目标业务集群的目标配置模型；给第一目标主机部署第一目标节点，根据待扩容的目标业务集群的目标配置模型对第一目标节点配置实时消息分发参数，将配置了实时消息分发参数的第一目标节点作为第一目标主机的转发节点，以将第一目标主机添加到待扩容的目标业务集群。If the deployment adjustment plan is expansion, the specific expansion plan includes: obtaining the target configuration model of the target business cluster to be expanded; deploying the first target node to the first target host, configuring the real-time message distribution parameters for the first target node according to the target configuration model of the target business cluster to be expanded, and using the first target node configured with the real-time message distribution parameters as a forwarding node of the first target host to add the first target host to the target business cluster to be expanded.

若部署调整方案为缩容，具体的缩容方案包括：确定待缩容的目标业务集群对应的待移除的第二目标主机；将第二目标主机从待缩容的目标业务集群中移除。If the deployment adjustment plan is scaling down, the specific scaling down plan includes: determining a second target host to be removed corresponding to the target business cluster to be scaled down; and removing the second target host from the target business cluster to be scaled down.

可选地，根据目标配置模型，给各个转发节点分别配置实时消息分发参数，包括：根据目标配置模型，给各个转发节点批量配置实时消息分发参数。Optionally, according to the target configuration model, real-time message distribution parameters are configured for each forwarding node respectively, including: according to the target configuration model, real-time message distribution parameters are configured for each forwarding node in batches.

本实施例中，在集群部署模式下，可以批量的对该业务集群中部署的转发节点批量配置实时消息分发参数，提高配置效率。In this embodiment, in the cluster deployment mode, the real-time message distribution parameters can be configured in batches for the forwarding nodes deployed in the service cluster, thereby improving the configuration efficiency.

若部署模式为非集群部署，则获取实时消息分发参数，包括：确定进行非集群部署的第三目标主机，以及获取第三目标主机待配置的第一实时消息分发参数；If the deployment mode is non-cluster deployment, obtain the real-time message distribution parameters, including: A third target host that is non-cluster deployed is configured, and a first real-time message distribution parameter to be configured on the third target host is obtained;

相应地，给业务集群中的目标主机部署目标节点，将实时消息分发参数配置于目标节点，并将配置了实时消息分发参数的目标节点作为转发节点，以使业务集群中的目标主机通过转发节点进行实时消息分发，包括：给第三目标主机部署第二目标节点，将第一实时消息分发参数配置于第二目标节点，并将配置了第一实时消息分发参数的第二目标节点作为第三目标主机的转发节点，以使第三目标主机通过转发节点进行实时消息分发。Correspondingly, a target node is deployed to a target host in a business cluster, a real-time message distribution parameter is configured on the target node, and the target node configured with the real-time message distribution parameter is used as a forwarding node, so that the target host in the business cluster performs real-time message distribution through the forwarding node, including: a second target node is deployed to a third target host, a first real-time message distribution parameter is configured on the second target node, and the second target node configured with the first real-time message distribution parameter is used as a forwarding node of the third target host, so that the third target host performs real-time message distribution through the forwarding node.

本实施例中，在非集群部署模式下，可以针对该主机进行单独配置第一实时消息分发参数。In this embodiment, in the non-cluster deployment mode, the first real-time message distribution parameter may be configured individually for the host.

通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本公开各个实施例的方法。Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.

实施例2Example 2

在本实施例中提供了一种运行于上述移动终端或***架构的分布式实时消息分发方法，图4是根据本公开实施例的一种分布式实时消息分发方法的流程图，如图4所示，该流程包括如下步骤：In this embodiment, a distributed real-time message distribution method running on the above mobile terminal or system architecture is provided. FIG4 is a flow chart of a distributed real-time message distribution method according to an embodiment of the present disclosure. As shown in FIG4, the process includes the following steps:

步骤S402，获取待分发的实时消息；Step S402, obtaining the real-time message to be distributed;

步骤S404，确定部署了转发节点的业务集群；转发节点配置有实时消息分发参数；Step S404, determining a service cluster in which a forwarding node is deployed; the forwarding node is configured with real-time message distribution parameters;

步骤S406，通过转发节点按照实时消息分发参数对实时消息进行分发。Step S406: distribute the real-time message according to the real-time message distribution parameter through the forwarding node.

通过上述步骤，该分布式实时消息分发方法，通过业务集群中部署的转发节点，按照实时消息分发参数对实时消息进行分发，解决了分布式消息分发的问题，提高了实时消息的分发效率。Through the above steps, the distributed real-time message distribution method distributes real-time messages according to real-time message distribution parameters through forwarding nodes deployed in the business cluster, solves the problem of distributed message distribution, and improves the distribution efficiency of real-time messages.

可选地，步骤S402和步骤S404的执行顺序是可以互换的，即可以先执行步骤S404，然后再执行S402。Optionally, the execution order of step S402 and step S404 may be interchangeable, that is, step S404 may be executed first, and then step S402.

实施例3 Example 3

在本实施例中对图2的***架构进行详细说明，该***架构，也就是分布式实时消息分发***，可以如图5所示。In this embodiment, the system architecture of FIG. 2 is described in detail. The system architecture, that is, the distributed real-time message distribution system, can be shown in FIG. 5 .

***包括转发节点501、业务集群502和拓扑管理***503，还包括网络管理***504(web管理***)；业务集群包括至少一个生产者505和至少一个消费者506。The system includes a forwarding node 501 , a service cluster 502 , a topology management system 503 , and a network management system 504 (web management system); the service cluster includes at least one producer 505 and at least one consumer 506 .

生产者505设置为向转发节点501发送消息，消费者506设置为从转发节点501接收消息，网络管理***504设置为对转发节点501部署配置规则，转发节点501设置为根据配置规则向消费者506推送消息。The producer 505 is configured to send messages to the forwarding node 501 , the consumer 506 is configured to receive messages from the forwarding node 501 , the network management system 504 is configured to deploy configuration rules for the forwarding node 501 , and the forwarding node 501 is configured to push messages to the consumer 506 according to the configuration rules.

若部署模式为集群部署，转发节点包括级联转发模式。If the deployment mode is cluster deployment, the forwarding node includes the cascade forwarding mode.

拓扑管理***，具体设置为从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型；目标配置模型设置为给业务集群待部署的目标节点配置实时消息分发参数；给业务集群中的各个目标主机分别部署目标节点，根据目标配置模型，给各个目标节点分别配置实时消息分发参数，并将配置了实时消息分发参数的目标节点作为转发节点，以使业务集群通过转发节点进行实时消息分发；The topology management system is specifically configured to determine the target configuration model corresponding to the business cluster to be deployed in the cluster from a preset configuration model; the target configuration model is configured to configure real-time message distribution parameters for the target node to be deployed in the business cluster; target nodes are deployed to each target host in the business cluster respectively, and real-time message distribution parameters are configured for each target node according to the target configuration model, and the target node configured with the real-time message distribution parameters is used as a forwarding node, so that the business cluster performs real-time message distribution through the forwarding node;

在级联转发模式下，转发节点，设置为根据配置规则向下一级转发节点501或者消费者506推送消息。In the cascade forwarding mode, the forwarding node is configured to push messages to the next level forwarding node 501 or consumer 506 according to the configuration rules.

对于单个转发节点(broker)，如图6所示，转发节点包括：接收队列、路由转发引擎和发送队列。For a single forwarding node (broker), as shown in FIG6 , the forwarding node includes: a receiving queue, a routing forwarding engine, and a sending queue.

接收队列设置为接收生产者推送的消息；路由转发引擎设置为将接收队列中的消息转发至发送队列；发送队列设置为将消息推送至消费者或者下一级转发节点。The receiving queue is set to receive messages pushed by the producer; the routing forwarding engine is set to forward the messages in the receiving queue to the sending queue; the sending queue is set to push the messages to the consumer or the next-level forwarding node.

可选地，转发节点还包括：接收文件缓存和发送文件缓存。Optionally, the forwarding node further includes: a receiving file cache and a sending file cache.

接收文件缓存设置为在接收队列接收的消息达到第一预设阈值时，缓存生产者或者上一级转发节点推送的消息，第一预设阈值小于或等于接收队列允许接收消息的最大值。发送文件缓存设置为在发送队列推送的消息达到第二预设阈值时，缓存路由转发引擎转发的消息，第二预设阈值小于或等于发送队列允许发送消息的最大值。The receiving file cache is set to cache the messages pushed by the producer or the upper-level forwarding node when the messages received by the receiving queue reach a first preset threshold, and the first preset threshold is less than or equal to the maximum value of the messages allowed to be received by the receiving queue. The sending file cache is set to cache the messages forwarded by the routing forwarding engine when the messages pushed by the sending queue reach a second preset threshold, and the second preset threshold is less than or equal to the maximum value of the messages allowed to be sent by the sending queue.

本实施例中，在转发节点中，producer负责将业务***中的消息以一定格式，推送(push)给broker进行路由转发。每一个producer都会创建一条producer链接，设置为给broker发送消息。In this embodiment, in the forwarding node, the producer is responsible for pushing the message in the business system to the broker for routing and forwarding in a certain format. Each producer will create a producer link, which is set to send messages to the broker.

consumer负责从broker以固定格式的方式接收消息。每一个consumer都会创建一条consumer链接，设置为从broker接收消息。 The consumer is responsible for receiving messages from the broker in a fixed format. Each consumer creates a consumer connection and is set to receive messages from the broker.

broker为每一条producer链路会配置一定数量的接收队列缓存，以避免在broker路由转发出现瓶颈时，流量反压到producer的情况。可选地，broker也可以支持配置一定数量的接收文件缓存功能，在接收队列缓存拥塞时，可以将消息继续缓存到文件缓存当中。The broker configures a certain number of receive queue caches for each producer link to avoid traffic back pressure to the producer when the broker routing forwarding encounters a bottleneck. Optionally, the broker can also support the configuration of a certain number of receive file caches. When the receive queue cache is congested, the message can continue to be cached in the file cache.

本实施例中，可支持数据可靠传输，由于broker上提供了一定数量的接收和发送缓存队列，并且可以支持一定数量的文件缓存，在***局部拥塞的情况下，有一定抗拥塞的能力。集群组网中的broker全网状互联，可以保证在某条或某些链路中断时，消息也能正常转发。本实施例中，还可以支持链路镜像功能，可以为重要集群业务，提供额外的镜像broker和镜像链路。可以在个别主机宕机的情况下，保证数据不丢失。In this embodiment, reliable data transmission can be supported. Since a certain number of receiving and sending cache queues are provided on the broker, and a certain number of file caches can be supported, there is a certain ability to resist congestion in the case of local congestion in the system. The brokers in the cluster network are fully meshed to ensure that messages can be forwarded normally when one or some links are interrupted. In this embodiment, link mirroring function can also be supported, and additional mirror brokers and mirror links can be provided for important cluster services. In the event of a crash of an individual host, data can be guaranteed not to be lost.

broker还会负责消息的路由转发操作，broker为每一条producer链路提供了路由转发引擎。broker在接收消息后，可直接调用路由转发引擎，按照该消息的路由转发规则，将其路由转发给相应的consumer。The broker is also responsible for the routing and forwarding of messages. The broker provides a routing and forwarding engine for each producer link. After receiving a message, the broker can directly call the routing and forwarding engine and forward it to the corresponding consumer according to the routing and forwarding rules of the message.

broker为每一条consumer链路也准备了一个发送队列，设置为平衡broker和consumer之间的速度差异。broker会首先将消息转发到相应consumer的发送队列当中。然后，broker再会将发送队列中的消息，推送(push)给consumer。可选地，broker也可以在发送队列拥塞时，将该队列的数据缓存到文件缓存中，避免数据丢失。The broker also prepares a send queue for each consumer link, which is set to balance the speed difference between the broker and the consumer. The broker will first forward the message to the send queue of the corresponding consumer. Then, the broker will push the message in the send queue to the consumer. Optionally, the broker can also cache the data in the send queue into the file cache when the send queue is congested to avoid data loss.

本实施例涉及到的broker除了可以对接producer和consumer之外，还可以直接对接实现级联组网，从而实现跨主机、跨网段、跨集群的业务互联和流量分发。broker之间级联组网如图6所示。In addition to being able to connect to producers and consumers, the brokers involved in this embodiment can also directly connect to realize cascade networking, thereby realizing cross-host, cross-network segment, and cross-cluster business interconnection and traffic distribution. The cascade networking between brokers is shown in Figure 6.

broker级联时，可以根据主机与主机之间、集群与集群之间的业务要求，在上游broker上启用不同的数据过滤和路由策略，实现跨主机和跨集群的业务策略分发。broker级联时，还可以根据跨网段之间网络时延、抖动、带宽等情况，统一实现跨网段的链路参数适配和流控策略。When brokers are cascaded, different data filtering and routing strategies can be enabled on the upstream broker according to the business requirements between hosts and clusters, to achieve cross-host and cross-cluster business strategy distribution. When brokers are cascaded, link parameter adaptation and flow control strategies can be uniformly implemented across network segments according to network latency, jitter, bandwidth, etc.

在图6中，还示意了拓扑管理***与分布式消息分发***的逻辑关系，如下：FIG6 also illustrates the logical relationship between the topology management system and the distributed message distribution system, as follows:

web管理***主要实现对单个broker节点和业务节点的配置管理功能，比较适合非集群部署模式下的单个broker节点和业务节点的管理。包括：每个broker安装主机、服务端口、接收缓存大小、接收文件缓存大小、发送缓存、发送缓存大小、路由分发规则以及业务节点计算逻辑等配置。The web management system mainly implements the configuration management function of a single broker node and a business node, and is more suitable for the management of a single broker node and a business node in a non-cluster deployment mode. It includes: each broker installation host, service port, receiving cache size, receiving file cache size, sending cache, sending cache size, routing distribution rules, and business node calculation logic configuration.

拓扑管理***构建在web管理***之上，主要实现集群化的broker节点和业务节点的配置管理功能。拓扑管理***将broker节点和业务节点按业务集群进行划分，并对同一集群中相同位置的broker节点和业务节点进行统一抽象和展示。实现broker节点按照业务集群的快速配置、部署、扩容、缩容、展示等功能，并可结合拓扑关系图，对集群中的broker节点和业务节点的接收和发送速率、接收和发送缓存等运行状态进行及时监控和告警，实现分布式总线的可视化管理和高效运维。The topology management system is built on the web management system, and mainly implements the configuration management function of clustered broker nodes and business nodes. The topology management system divides broker nodes and business nodes according to business clusters, and manages broker nodes and business nodes in the same location in the same cluster. Unified abstraction and display. Realize the functions of rapid configuration, deployment, expansion, reduction, and display of broker nodes according to business clusters, and timely monitor and warn the receiving and sending rates, receiving and sending caches, and other operating states of broker nodes and business nodes in the cluster in combination with the topology diagram, so as to realize visual management and efficient operation and maintenance of distributed buses.

本实施例中，broker可以支持加载灵活的路由过滤分发的规则。规则可以深入消息中的指定片段进行分析，按照片段数据的特征，对消息进行相应的路由分发或过滤，真正实现了业务流量的灵活路由分发。In this embodiment, the broker can support loading flexible routing filtering and distribution rules. The rules can analyze the specified fragments in the message, and perform corresponding routing distribution or filtering on the message according to the characteristics of the fragment data, thus truly realizing flexible routing distribution of business traffic.

producer/consumer一般和业务***绑定在一起部署，producer/consumer与broker之间的注册/注销操作，由producer/consumer与broker之间自组织完成，无需web/拓扑管理***配置干预。而broker在***中部署，则需要web管理***和拓扑管理***的配合。Producers/consumers are usually deployed together with business systems. The registration/deregistration operations between producers/consumers and brokers are completed by the producers/consumers and brokers themselves, without the need for configuration intervention by the web/topology management system. If brokers are deployed in the system, the cooperation of the web management system and the topology management system is required.

broker的部署，主要分为以下两种模式：非集群部署模式和集群部署模式。Broker deployment is mainly divided into the following two modes: non-cluster deployment mode and cluster deployment mode.

broker非集群模式进行部署时，broker作为一个单独的个体，可以直接在web管理页面上进行部署，并完成安装主机、服务端口、接收缓存大小、接收文件缓存大小、发送缓存、发送缓存大小、路由分发规则等相关的配置。在拓扑管理***中，可以导入和生成非集群部署的broker的配置拓扑，并在该拓扑上完成对非集群部署的broker的配置展示和状态监控。When broker is deployed in non-cluster mode, broker can be deployed directly on the web management page as a single individual, and complete the configuration of installation host, service port, receiving buffer size, receiving file buffer size, sending buffer, sending buffer size, routing distribution rules, etc. In the topology management system, you can import and generate the configuration topology of broker deployed in non-cluster, and complete the configuration display and status monitoring of broker deployed in non-cluster on the topology.

broker作为集群模式进行部署时，broker将和业务集群配合组网。broker节点在业务集群的每台主机上都会部署，且在一进一出两方向都部署相应的broker节点，对于集群broker组网模式如图7所示。When broker is deployed as a cluster mode, broker will cooperate with the business cluster to form a network. Broker nodes will be deployed on each host of the business cluster, and corresponding broker nodes will be deployed in both the inbound and outbound directions. The cluster broker networking mode is shown in Figure 7.

业务集群在各个主机上通过consumer和入口broker节点建链，并从该节点中订阅消息。相应地，业务集群在各个主机上通过producer和出口broker节点建链，并向该broker节点中推送数据。The business cluster builds a link through the consumer and the entry broker node on each host and subscribes to messages from the node. Correspondingly, the business cluster builds a link through the producer and the exit broker node on each host and pushes data to the broker node.

业务集群之间的消息，将统一由broker来负责转发。上游业务集群出口broker和下游业务集群入口broker之间，将创建全网状的链路链接关系。上游业务集群出口broker将按照配置的路由规则，将上游业务集群中的消息，按需转发给下游业务集群入口broker。业务集群每台主机上一般会有大量consumer，这些consumer将统一从本地入口broker节点中订阅所需的消息。Messages between business clusters will be forwarded by brokers. A fully meshed link relationship will be created between the upstream business cluster egress broker and the downstream business cluster ingress broker. The upstream business cluster egress broker will forward messages in the upstream business cluster to the downstream business cluster ingress broker on demand according to the configured routing rules. There are generally a large number of consumers on each host in the business cluster, and these consumers will subscribe to the required messages from the local ingress broker node.

上述统一的集群组网方式，如果用常规web配置方式，将会非常繁琐，管理维护也会很困难。而本实施例这种组网方式十分有利于拓扑管理***对集群中broker的统一配置和管理。The above unified cluster networking method, if configured using a conventional web configuration method, will be very cumbersome and difficult to manage and maintain. However, the networking method of this embodiment is very conducive to the unified configuration and management of brokers in the cluster by the topology management system.

本实施例中，producer、consumer和broker之间可以实现自动建链、拆链，接收和发送队列自动创建。broker可以根据producer和consumer注册/注销信息，在broker中自动实现路由转发表的更新。不需要人工和同步集群***的干预，实现broker和管理节点之间尽量少的交互，提高***运行效率，减少管理开销，降低维护和管理成本。In this embodiment, the producer, consumer and broker can automatically establish a link. The chain is disconnected, and the receiving and sending queues are automatically created. The broker can automatically update the routing table in the broker according to the producer and consumer registration/deregistration information. No manual intervention or synchronization cluster system intervention is required, and the interaction between the broker and the management node is minimized, which improves the system operation efficiency, reduces management overhead, and reduces maintenance and management costs.

为此，可以将业务集群之间的组网关系进行抽象，并形成拓扑化的配置模版。在实际消息总线配置时，只需将主机和模版进行匹配，即可完成分布式消息集群的配置。比如：根据上述业务集群之间的组网关系，将其抽象为拓扑化的分布式消息集群配置模版，如图8所示。To this end, the networking relationship between business clusters can be abstracted and a topological configuration template can be formed. When configuring the actual message bus, you only need to match the host and the template to complete the configuration of the distributed message cluster. For example, based on the networking relationship between the above business clusters, it is abstracted into a topological distributed message cluster configuration template, as shown in Figure 8.

相应地，分布式分发***部署方法，可以如图9所示，包括：Accordingly, the distributed distribution system deployment method may be as shown in FIG9 , including:

步骤902，开始集群broker部署操作；Step 902, start cluster broker deployment operation;

步骤904，通过拓扑管理***，提前规划好broker集群的配置组网模型。模型包括：broker集群名、broker集群的上下游关系、每个broker集群中入口和出口broker的公共配置信息，比如：服务名、服务端口、路由规则等；Step 904: Plan the configuration networking model of the broker cluster in advance through the topology management system. The model includes: broker cluster name, upstream and downstream relationships of broker clusters, and common configuration information of the entry and exit brokers in each broker cluster, such as: service name, service port, routing rules, etc.

步骤906，通过拓扑管理***，将相关主机加入到相应broker集群当中；Step 906, adding the relevant host to the corresponding broker cluster through the topology management system;

步骤908，通过拓扑管理***，按照配置模型，调用web管理***的配置接口，在已加入主机的broker集群中进行统一部署实施；Step 908, through the topology management system, according to the configuration model, calling the configuration interface of the web management system, and performing unified deployment and implementation in the broker cluster that has joined the host;

步骤910，完成集群broker部署操作。Step 910, completing the cluster broker deployment operation.

本实施例中，分布式消息分发***可以和业务集群统一部署，不需要部署专门的消息分发集群，解决了分布式组网能力不足的问题，真正实现分布式的组网，可避免***出现局部网络带宽、局部计算存储瓶颈，进而达到了提升分布式分发***性能的效果。In this embodiment, the distributed message distribution system can be deployed uniformly with the business cluster, and there is no need to deploy a dedicated message distribution cluster. This solves the problem of insufficient distributed networking capabilities and truly realizes distributed networking. It can avoid local network bandwidth and local computing and storage bottlenecks in the system, thereby achieving the effect of improving the performance of the distributed distribution system.

broker在***运维阶段，涉及到调整转发节点数量的操作主要包括：***扩容或***缩容。During the system operation and maintenance phase, broker operations involving adjusting the number of forwarding nodes mainly include: system expansion or system reduction.

***扩容时，只需通过拓扑管理***，将相关主机加入到相关broker集群中，并同步一下相关broker集群的模版配置，即可完成相关broker集群扩容操作。When expanding the system, you only need to add the relevant hosts to the relevant broker clusters through the topology management system and synchronize the template configuration of the relevant broker clusters to complete the expansion of the relevant broker clusters.

同样地，***缩容时，也只通过拓扑管理***，将相关主机退出相关broker集群，即可完成broker集群缩容操作。Similarly, when the system is scaled down, the broker cluster scaling operation can be completed by simply removing the relevant hosts from the relevant broker cluster through the topology management system.

一个实施例中，在***运维时，拓扑管理***，还设置为对部署了转发节点的各个业务集群的组网情况进行展示，以及对各个业务集群的运行状态进行监控。 In one embodiment, during system operation and maintenance, the topology management system is further configured to display the networking status of each service cluster where the forwarding node is deployed, and monitor the operating status of each service cluster.

对各个业务集群的组网情况进行展示，可以是进行拓扑展示，包括集群拓扑展示和非集群拓扑展示。The networking conditions of each service cluster are displayed, which may be a topology display, including a cluster topology display and a non-cluster topology display.

集群拓扑展示：如图10所示，broker集群可以方便地通过拓扑管理***进行运维管理，每个broker集群中入口或出口位置的broker，完成的功能相同，会被抽象成一个拓扑管理节点即集群broker节点，进行展示。从抽象管理节点之间的上下游关系，即可完整的展示broker集群的组网情况。Cluster topology display: As shown in Figure 10, the broker cluster can be easily managed and operated through the topology management system. The brokers at the entry or exit of each broker cluster perform the same functions and are abstracted into a topology management node, i.e., the cluster broker node, for display. The networking status of the broker cluster can be fully displayed from the upstream and downstream relationships between the abstract management nodes.

非集群拓扑展示：如图11所示，如果消息分发***后续在运维的过程中，未完全按照集群组网模式进行扩容。此时，集群组网可能退化成非集群组网，对应的组网情况也会在拓扑管理***中反映。非集群broker节点以及与其他broker之间的链接关系，会用特殊图标标识以示区别。Non-cluster topology display: As shown in Figure 11, if the message distribution system is not expanded completely according to the cluster networking mode during the subsequent operation and maintenance process, the cluster networking may degenerate into a non-cluster networking, and the corresponding networking situation will also be reflected in the topology management system. Non-cluster broker nodes and the link relationship with other brokers will be marked with special icons to distinguish them.

拓扑管理***可以对各个业务集群的运行状态进行监控，包括对抽象的拓扑管理节点进行监控，以及对展开的各个主机broker节点进行监控。The topology management system can monitor the operating status of each business cluster, including monitoring the abstract topology management node and monitoring each expanded host broker node.

其中，抽象的拓扑管理节点还可以被展开成各个主机broker节点，在展开展示后，可以方便对各个主机broker节点进行监控，在不展开时，对该拓扑管理节点整体进行监控。如图12，在拓扑管理***中，当被抽象的拓扑管理节点获取到展开指令时，该拓扑管理节点可以展开为其包含的各个主机broker节点，展开指令可以是对拓扑管理节点的鼠标点击指令或对拓扑管理节点的触摸操作，也可以是代码指令，不作限制。The abstract topology management node can also be expanded into each host broker node. After the expansion is displayed, it is convenient to monitor each host broker node. When it is not expanded, the topology management node is monitored as a whole. As shown in Figure 12, in the topology management system, when the abstract topology management node obtains the expansion instruction, the topology management node can be expanded into each host broker node it contains. The expansion instruction can be a mouse click instruction on the topology management node or a touch operation on the topology management node, or a code instruction, without limitation.

其中，通过各个展开的主机broker节点可以方便查询各个主机broker节点的运行状态(包括但不限于输入流量、输出流量、输入队列、输出队列情况)，并接收主机broker节点上报的告警信息。查询方式可以根据需要设置，可以是获取查询指令后查询，比如获取到鼠标点击指令、鼠标悬停指令或者触摸操作等形式，具体的查询方式不作限制，还可以在各个主机broker的展示图标周围或者展示图标上直接显示各个运行状态，方便查看，不作限制。Among them, through each expanded host broker node, it is convenient to query the operating status of each host broker node (including but not limited to input traffic, output traffic, input queue, output queue status), and receive the alarm information reported by the host broker node. The query method can be set as needed, and it can be queried after obtaining the query instruction, such as obtaining the form of mouse click instruction, mouse hover instruction or touch operation, etc. The specific query method is not limited, and each operating status can also be directly displayed around the display icon of each host broker or on the display icon, which is convenient for viewing without restriction.

需要说明的是，拓扑管理***对各个业务集群的运行状态进行监控，是对各个业务集群的实时监控，不限于业务集群在拓扑管理界面处于抽象的拓扑管理节点还是展开的各个主机broker节点，当其在抽象的拓扑管理节点展示时，各个主机broker节点也处于拓扑管理***的监控之中，只是参数没有在拓扑管理***的界面展示而已，也可以根据需要对其展示，不作限制。It should be noted that the topology management system monitors the operating status of each business cluster, which is real-time monitoring of each business cluster. It is not limited to whether the business cluster is in the abstract topology management node or the expanded host broker nodes in the topology management interface. When it is displayed in the abstract topology management node, each host broker node is also under the monitoring of the topology management system, but the parameters are not displayed in the interface of the topology management system. It can also be displayed as needed without restriction.

同样地，被抽象为非集群的broker节点，也可以通过展开成各个主机broker节点，实现对其中主机broker节点的运行状态监控。 Similarly, the broker nodes that are abstracted as non-clustered nodes can also be expanded into various host broker nodes to monitor the operating status of the host broker nodes.

通过拓扑管理***，可以方便的对现有集群broker的组网方式进行链路连接关系、配置参数等的统一修改和配置下发功能。只需在现有集群broker节点图标中，将相关配置进行可视化的修改操作，并在拓扑管理界面上，将配置保存下发，即可完成集群broker配置修改操作，十分高效和便捷。如图13，在拓扑管理***中，可以对节点与节点之间的连线进行不同状态的标记，比如，以①代表当前配置，以②代表修改的配置(或待执行的配置)，以节点“集群broker节点V入”举例，当前配置为：服务端口A，输入队列大小B，输出队列大小C，路由规则D。在修改链路关系后，可以从“集群broker节点III入”直接连接到“集群broker节点V入”，并且，对该条新增链路进行配置修改，待修改或者待执行的配置为：服务端口A′，输入队列大小B′，输出队列大小C′，路由规则D′。Through the topology management system, the existing cluster broker networking mode can be easily modified and configured in a unified manner, such as link connection relationships, configuration parameters, etc. It can also be configured and issued. You only need to modify the relevant configurations in a visual way in the existing cluster broker node icon, and save and issue the configurations on the topology management interface to complete the cluster broker configuration modification operation, which is very efficient and convenient. As shown in Figure 13, in the topology management system, the connections between nodes can be marked with different states. For example, ① represents the current configuration, and ② represents the modified configuration (or the configuration to be executed). Taking the node "cluster broker node V in" as an example, the current configuration is: service port A, input queue size B, output queue size C, and routing rule D. After modifying the link relationship, you can directly connect from "cluster broker node III in" to "cluster broker node V in", and modify the configuration of the newly added link. The configuration to be modified or executed is: service port A', input queue size B', output queue size C', and routing rule D'.

需要说明的是，不同状态的标记可以有很多种形式，能进行区分即可，比如可以是线条的形状不同或者颜色不同等，具体表现形式不作限制。It should be noted that the markings of different states can take many forms as long as they can be distinguished, such as different line shapes or colors, etc., and the specific form of expression is not limited.

同样地，非集群broker节点也可以在拓扑管理***上，实现配置修改和配置下发功能。可选地，通过在配置模版中新增集群，将若干非集群broker节点，加入到新增集群，或者直接将若干非集群broker节点，加入到现有配置模版集群的方式，将非集群broker节点改造成集群broker节点。Similarly, non-cluster broker nodes can also implement configuration modification and configuration delivery functions on the topology management system. Optionally, non-cluster broker nodes can be transformed into cluster broker nodes by adding a new cluster in the configuration template and adding several non-cluster broker nodes to the new cluster, or by directly adding several non-cluster broker nodes to the existing configuration template cluster.

本公开提供了一种分布式实时消息分发***，该***主要满足分布式组网场景中，对海量数据、实时性要求较高的情况下，消息快速高效分发的要求。依托于拓扑管理***，可以很好的和分布式消息总线***结合，使分布式消息***具备快速部署和高效维护的能力。***部署和运行状态可以直观、高效的呈现在运维人员面前，极大减少运维投入。The present disclosure provides a distributed real-time message distribution system, which mainly meets the requirements of fast and efficient message distribution in distributed networking scenarios with high requirements for massive data and real-time performance. Relying on the topology management system, it can be well combined with the distributed message bus system, so that the distributed message system has the ability of rapid deployment and efficient maintenance. The system deployment and operation status can be presented to the operation and maintenance personnel intuitively and efficiently, greatly reducing the operation and maintenance investment.

本公开涉及的基于拓扑管理的分布式实时流消息分发***，主要由producer、consumer、broker、web管理***和拓扑管理***几个部分组成。***主要具有以下特点：The distributed real-time stream message distribution system based on topology management involved in this disclosure is mainly composed of producer, consumer, broker, web management system and topology management system. The system has the following main features:

producer/consumer负责向broker发送和接收消息，而broker是本次消息总线***的核心消息分发组件。Producers/consumers are responsible for sending and receiving messages to brokers, which are the core message distribution components of this message bus system.

在实际部署时，推荐broker和业务***共机部署，以最大限度适配和利用业务组网环境资源，实现全分布式的组网，并通过拓扑管理***统一部署和管理。依托于分布式的部署架构，充分利用分布式集群***的计算和存储资源，可以避免流量、计算资源和存储资源过分集中在***中的某个区域，不会存在比较明显的业务处理瓶颈点。In actual deployment, it is recommended to deploy brokers and business systems on the same machine to maximize the adaptation and utilization of business networking environment resources, realize fully distributed networking, and deploy and manage them uniformly through the topology management system. Relying on the distributed deployment architecture and making full use of the computing and storage resources of the distributed cluster system, it can avoid excessive concentration of traffic, computing resources, and storage resources in a certain area of the system, and there will be no obvious business processing bottlenecks.

***部署完成后，***各个组件具有一定的自适应性，可以实现自主的链路注册/注销、自动的路由计算、流量调节和链路备份等功能。 After the system is deployed, each component of the system has a certain degree of adaptability and can realize functions such as autonomous link registration/deregistration, automatic route calculation, traffic regulation and link backup.

***运维过程中的扩容、缩容、配置状态和运行状态监控等操作，都可以通过拓扑管理***比较好的完成，从而极大地降低运维管理成本。During the system operation and maintenance process, operations such as expansion, reduction, configuration status and operation status monitoring can be completed relatively well through the topology management system, thereby greatly reducing the operation and maintenance management costs.

实施例4Example 4

在本实施例中还提供了一种分布式分发***部署装置，该装置设置为实现上述实施例及优选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。In the present embodiment, a distributed distribution system deployment device is also provided, and the device is configured to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and/or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

图14是根据本公开实施例的一种分布式分发***部署装置的结构框图，如图14所示，该装置包括：FIG. 14 is a structural block diagram of a distributed distribution system deployment device according to an embodiment of the present disclosure. As shown in FIG. 14 , the device includes:

第一获取模块1402，设置为获取实时消息分发参数；A first acquisition module 1402 is configured to acquire real-time message distribution parameters;

配置模块1404，设置为给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发。Configuration module 1404 is configured to deploy a target node for a target host in a business cluster, configure the real-time message distribution parameters on the target node, and use the target node configured with the real-time message distribution parameters as a forwarding node, so that the target host in the business cluster performs real-time message distribution through the forwarding node.

该装置，给业务集群中的目标主机部署目标节点，给目标节点配置实时消息分发参数，以使部署了转发节点的各个业务集群通过转发节点进行实时消息分发。该装置将转发节点与业务集群的主机共机部署，使分布式消息分发***可以和业务集群统一部署，不需要部署专门的消息分发集群，解决了分布式组网能力不足的问题，真正实现分布式的组网，可避免***出现局部网络带宽、局部计算存储瓶颈，进而达到了提升分布式分发***性能的效果。The device deploys a target node to a target host in a business cluster, and configures real-time message distribution parameters for the target node, so that each business cluster that has deployed a forwarding node can perform real-time message distribution through the forwarding node. The device deploys the forwarding node and the host of the business cluster on the same machine, so that the distributed message distribution system can be deployed uniformly with the business cluster, without the need to deploy a dedicated message distribution cluster, solving the problem of insufficient distributed networking capabilities, truly realizing distributed networking, and avoiding local network bandwidth and local computing storage bottlenecks in the system, thereby achieving the effect of improving the performance of the distributed distribution system.

需要说明的是，上述各个模块是可以通过软件或硬件来实现的，对于后者，可以通过以下方式实现，但不限于此：上述模块均位于同一处理器中；或者，上述各个模块以任意组合的形式分别位于不同的处理器中。It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

实施例5Example 5

在本实施例中还提供了一种分布式实时消息分发装置，该装置设置为实现上述实施例及优选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。In the present embodiment, a distributed real-time message distribution device is also provided, and the device is configured to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and/or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

图15是根据本公开实施例的一种分布式实时消息分发装置的结构框图，如图15所示，该装置包括：FIG. 15 is a structural block diagram of a distributed real-time message distribution device according to an embodiment of the present disclosure. As shown in FIG. 15 , the device includes:

获取模块1502，设置为获取待分发的实时消息； An acquisition module 1502 is configured to acquire a real-time message to be distributed;

第三确定模块1504，设置为确定部署了转发节点的业务集群；所述转发节点配置有实时消息分发参数；The third determination module 1504 is configured to determine a service cluster in which a forwarding node is deployed; the forwarding node is configured with a real-time message distribution parameter;

分发模块1506，设置为通过所述转发节点按照所述实时消息分发参数对所述实时消息进行分发。The distribution module 1506 is configured to distribute the real-time message through the forwarding node according to the real-time message distribution parameter.

该分布式实时消息分发装置通过业务集群中部署的转发节点，按照实时消息分发参数对实时消息进行分发，解决了分布式消息分发的问题，提高了实时消息的分发效率。The distributed real-time message distribution device distributes real-time messages according to real-time message distribution parameters through forwarding nodes deployed in a service cluster, thereby solving the problem of distributed message distribution and improving the distribution efficiency of real-time messages.

实施例6Example 6

本公开的实施例还提供了一种存储介质，该存储介质中存储有计算机程序，其中，该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。An embodiment of the present disclosure further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

可选地，在本实施例中，上述存储介质可以被设置为存储设置为执行以下步骤的计算机程序：Optionally, in this embodiment, the storage medium may be configured to store a computer program configured to perform the following steps:

获取实时消息分发参数；Get real-time message distribution parameters;

给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发。Deploy a target node to a target host in a business cluster, configure the real-time message distribution parameters on the target node, and use the target node configured with the real-time message distribution parameters as a forwarding node so that the target host in the business cluster performs real-time message distribution through the forwarding node.

可选地，存储介质还被设置为存储设置为执行以下步骤的计算机程序：Optionally, the storage medium is further configured to store a computer program configured to perform the following steps:

获取待分发的实时消息；Get real-time messages to be distributed;

可选地，在本实施例中，上述存储介质可以包括但不限于：U盘、只读存储器(Read-Only Memory，简称为ROM)、随机存取存储器(Random Access Memory，简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

实施例7Example 7

本公开的实施例还提供了一种电子装置，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。将图1中的存储器104作为该电子装置中的一种存储器的一种示例。The embodiment of the present disclosure also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments. The memory 104 in FIG1 is taken as an example of a memory in the electronic device.

可选地，上述电子装置还可以包括传输设备以及输入输出设备，其中，该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。Optionally, the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.

可选地，在本实施例中，上述处理器可以被设置为通过计算机程序执行以下步骤：Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

获取实时消息分发参数；Get real-time message distribution parameters;

可选地，上述处理器还被设置为存储设置为执行以下步骤的计算机程序：Optionally, the processor is further configured to store a computer program configured to perform the following steps:

获取待分发的实时消息；确定部署了转发节点的业务集群；所述转发节点配置有实时消息分发参数；通过所述转发节点按照所述实时消息分发参数对所述实时消息进行分发。Acquire a real-time message to be distributed; determine a service cluster in which a forwarding node is deployed; the forwarding node is configured with a real-time message distribution parameter; and distribute the real-time message through the forwarding node according to the real-time message distribution parameter.

可选地，本实施例中的具体示例可以参考上述实施例及可选实施方式中所描述的示例，本实施例在此不再赘述。Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

显然，本领域的技术人员应该明白，上述的本公开的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本公开不限制于任何特定的硬件和软件结合。Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present disclosure can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any specific combination of hardware and software.

以上所述仅为本公开的优选实施例而已，并不用于限制本公开，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。 The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

一种分布式分发***部署方法，包括：A distributed distribution system deployment method, comprising:

获取实时消息分发参数；Get real-time message distribution parameters;

给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发。Deploy a target node to a target host in a business cluster, configure the real-time message distribution parameters on the target node, and use the target node configured with the real-time message distribution parameters as a forwarding node so that the target host in the business cluster performs real-time message distribution through the forwarding node.
根据权利要求1所述的方法，其中，若部署模式为集群部署，获取实时消息分发参数，包括：The method according to claim 1, wherein, if the deployment mode is cluster deployment, obtaining the real-time message distribution parameters comprises:

从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型；所述目标配置模型设置为给所述业务集群待部署的目标节点配置实时消息分发参数；Determine a target configuration model corresponding to the service cluster to be deployed in the cluster from a preset configuration model; the target configuration model is set to configure real-time message distribution parameters for the target node to be deployed in the service cluster;

相应地，给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发，包括：Accordingly, deploying a target node to a target host in a service cluster, configuring the real-time message distribution parameter on the target node, and using the target node configured with the real-time message distribution parameter as a forwarding node, so that the target host in the service cluster performs real-time message distribution through the forwarding node, including:

给所述业务集群中的各个目标主机分别部署所述目标节点，根据所述目标配置模型，给各个所述目标节点分别配置所述实时消息分发参数，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使所述业务集群通过所述转发节点进行实时消息分发。The target node is deployed to each target host in the business cluster respectively, and the real-time message distribution parameters are configured for each target node according to the target configuration model, and the target node configured with the real-time message distribution parameters is used as a forwarding node, so that the business cluster performs real-time message distribution through the forwarding node.
根据权利要求2所述的方法，其中，从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型之前，所述方法还包括：The method according to claim 2, wherein, before determining the target configuration model corresponding to the service cluster to be cluster deployed from the preset configuration model, the method further comprises:

确定待集群部署的至少一个所述业务集群。At least one of the service clusters to be cluster deployed is determined.
根据权利要求2所述的方法，其中，从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型之前，所述方法还包括：获取所述配置模型；The method according to claim 2, wherein, before determining the target configuration model corresponding to the service cluster to be cluster deployed from the preset configuration model, the method further comprises: acquiring the configuration model;

其中，所述配置模型的生成过程包括：The generation process of the configuration model includes:

获取所有所述业务集群的组网关系；Obtaining the networking relationship of all the service clusters;

根据所述组网关系，确定每一个所述业务集群对应的目标配置模型，将所有所述目标配置模型的集合作为所述配置模型。According to the networking relationship, a target configuration model corresponding to each of the service clusters is determined, and a set of all the target configuration models is used as the configuration model.
根据权利要求2所述的方法，其中，给所述业务集群中的各个目标主机分别部署所述目标节点，并根据所述目标配置模型，给各个所述目标节点分别配置所述实时消息分发参数之后，所述方法还包括：获取部署调整方案；所述部署调整方案包括扩容或缩容； According to the method of claim 2, after respectively deploying the target node to each target host in the service cluster and respectively configuring the real-time message distribution parameters for each target node according to the target configuration model, the method further comprises: obtaining a deployment adjustment plan; the deployment adjustment plan comprises expansion or reduction;

若部署调整方案为扩容，则确定待扩容的目标业务集群对应的待添加的第一目标主机；If the deployment adjustment plan is capacity expansion, determine the first target host to be added corresponding to the target service cluster to be expanded;

获取所述待扩容的目标业务集群的所述目标配置模型；Acquire the target configuration model of the target service cluster to be expanded;

给所述第一目标主机部署第一目标节点，根据所述待扩容的目标业务集群的所述目标配置模型对所述第一目标节点配置实时消息分发参数，将配置了所述实时消息分发参数的所述第一目标节点作为所述第一目标主机的转发节点，以将所述第一目标主机添加到所述待扩容的目标业务集群；Deploy a first target node to the first target host, configure a real-time message distribution parameter for the first target node according to the target configuration model of the target service cluster to be expanded, and use the first target node configured with the real-time message distribution parameter as a forwarding node of the first target host to add the first target host to the target service cluster to be expanded;

若部署调整方案为缩容，则确定待缩容的目标业务集群对应的待移除的第二目标主机；If the deployment adjustment plan is to reduce capacity, determine the second target host to be removed corresponding to the target service cluster to be reduced;

将所述第二目标主机从所述待缩容的目标业务集群中移除。The second target host is removed from the target service cluster to be shrunk.
根据权利要求2所述的方法，其中，根据所述目标配置模型，给各个所述目标节点分别配置所述实时消息分发参数，包括：The method according to claim 2, wherein, according to the target configuration model, configuring the real-time message distribution parameters for each of the target nodes respectively comprises:

根据所述目标配置模型，给各个所述目标节点批量配置所述实时消息分发参数。The real-time message distribution parameters are configured in batches for each of the target nodes according to the target configuration model.
根据权利要求1所述的方法，其中，若部署模式为非集群部署，获取实时消息分发参数，包括：The method according to claim 1, wherein, if the deployment mode is non-cluster deployment, obtaining the real-time message distribution parameters comprises:

确定进行所述非集群部署的第三目标主机，以及获取所述第三目标主机待配置的第一实时消息分发参数；Determine a third target host for the non-cluster deployment, and obtain a first real-time message distribution parameter to be configured for the third target host;

相应地，给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发，包括：Accordingly, deploying a target node to a target host in a service cluster, configuring the real-time message distribution parameter on the target node, and using the target node configured with the real-time message distribution parameter as a forwarding node, so that the target host in the service cluster performs real-time message distribution through the forwarding node, including:

给所述第三目标主机部署第二目标节点，将所述第一实时消息分发参数配置于所述第二目标节点，并将配置了所述第一实时消息分发参数的所述第二目标节点作为所述第三目标主机的转发节点，以使所述第三目标主机通过所述转发节点进行实时消息分发。A second target node is deployed to the third target host, the first real-time message distribution parameter is configured on the second target node, and the second target node configured with the first real-time message distribution parameter is used as a forwarding node of the third target host, so that the third target host performs real-time message distribution through the forwarding node.
一种分布式实时消息分发方法，包括：A distributed real-time message distribution method, comprising:

获取待分发的实时消息；Get real-time messages to be distributed;

确定部署了转发节点的业务集群；所述转发节点配置有实时消息分发参数；Determining a service cluster in which a forwarding node is deployed; the forwarding node is configured with real-time message distribution parameters;

通过所述转发节点按照所述实时消息分发参数对所述实时消息进行分发。The real-time message is distributed by the forwarding node according to the real-time message distribution parameter.
一种分布式实时消息分发***，包括：转发节点、业务集群和拓扑管理***；A distributed real-time message distribution system, including: forwarding nodes, business clusters and topology Management systems;

所述拓扑管理***，设置为获取实时消息分发参数；给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发。The topology management system is configured to obtain real-time message distribution parameters; deploy target nodes to target hosts in a business cluster, configure the real-time message distribution parameters on the target nodes, and use the target nodes configured with the real-time message distribution parameters as forwarding nodes, so that the target hosts in the business cluster perform real-time message distribution through the forwarding nodes.
根据权利要求9所述的***，其中，所述***还包括网络管理***；所述业务集群包括至少一个生产者和至少一个消费者；The system according to claim 9, wherein the system further comprises a network management system; the service cluster comprises at least one producer and at least one consumer;

所述生产者，设置为向所述转发节点发送消息；The producer is configured to send a message to the forwarding node;

所述消费者，设置为从所述转发节点接收消息；The consumer is configured to receive messages from the forwarding node;

所述网络管理***，设置为对所述转发节点部署配置规则；The network management system is configured to deploy configuration rules for the forwarding nodes;

所述转发节点，设置为根据所述配置规则向所述消费者推送消息。The forwarding node is configured to push messages to the consumer according to the configuration rule.
根据权利要求10所述的***，其中，若部署模式为集群部署，所述转发节点包括级联转发模式；The system according to claim 10, wherein, if the deployment mode is cluster deployment, the forwarding node includes a cascade forwarding mode;

所述拓扑管理***，具体设置为从预设的配置模型中，确定待集群部署的业务集群对应的目标配置模型；所述目标配置模型设置为给所述业务集群待部署的目标节点配置实时消息分发参数；给所述业务集群中的各个目标主机分别部署所述目标节点，根据所述目标配置模型，给各个所述目标节点分别配置所述实时消息分发参数，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使所述业务集群通过所述转发节点进行实时消息分发；The topology management system is specifically configured to determine the target configuration model corresponding to the business cluster to be cluster-deployed from a preset configuration model; the target configuration model is configured to configure real-time message distribution parameters for the target nodes to be deployed in the business cluster; the target nodes are respectively deployed to each target host in the business cluster, and the real-time message distribution parameters are respectively configured for each target node according to the target configuration model, and the target nodes configured with the real-time message distribution parameters are used as forwarding nodes, so that the business cluster performs real-time message distribution through the forwarding nodes;

所述转发节点，设置为根据所述配置规则向下一级转发节点或者所述消费者推送消息。The forwarding node is configured to push messages to the next level forwarding node or the consumer according to the configuration rule.
根据权利要求11所述的***，其中，所述转发节点包括：接收队列、路由转发引擎和发送队列；The system according to claim 11, wherein the forwarding node comprises: a receiving queue, a routing forwarding engine, and a sending queue;

所述接收队列设置为接收所述生产者推送的消息；The receiving queue is configured to receive messages pushed by the producer;

所述路由转发引擎设置为将所述接收队列中的消息转发至所述发送队列；The routing forwarding engine is configured to forward the messages in the receiving queue to the sending queue;

所述发送队列设置为将所述消息推送至所述消费者或者下一级所述转发节点。The sending queue is configured to push the message to the consumer or the forwarding node at the next level.
根据权利要求12所述的***，其中，所述转发节点还包括：接收文件缓存和发送文件缓存；The system according to claim 12, wherein the forwarding node further comprises: a receiving file cache and a sending file cache;

所述接收文件缓存设置为在所述接收队列接收的消息达到第一预设阈值时，缓存所述生产者或者上一级所述转发节点推送的消息；所述第一预设阈值小于或等于所述接收队列允许接收消息的最大值；The receiving file cache is configured to cache messages pushed by the producer or the forwarding node at the previous level when the messages received by the receiving queue reach a first preset threshold; The preset threshold is less than or equal to the maximum value of the message that the receiving queue is allowed to receive;

所述发送文件缓存设置为在所述发送队列推送的消息达到第二预设阈值时，缓存所述路由转发引擎转发的所述消息；所述第二预设阈值小于或等于所述发送队列允许发送消息的最大值。The sending file cache is configured to cache the message forwarded by the routing forwarding engine when the message pushed by the sending queue reaches a second preset threshold; the second preset threshold is less than or equal to the maximum value of the message allowed to be sent by the sending queue.
根据权利要求10所述的***，其中，The system according to claim 10, wherein:

所述拓扑管理***，还设置为对部署了所述转发节点的各个所述业务集群的组网情况进行展示，以及对各个所述业务集群的运行状态进行监控。The topology management system is further configured to display the networking status of each of the service clusters where the forwarding nodes are deployed, and to monitor the operating status of each of the service clusters.
一种分布式分发***部署装置，包括：A distributed distribution system deployment device, comprising:

第一获取模块，设置为获取实时消息分发参数；A first acquisition module, configured to acquire real-time message distribution parameters;

配置模块，设置为给业务集群中的目标主机部署目标节点，将所述实时消息分发参数配置于所述目标节点，并将配置了所述实时消息分发参数的所述目标节点作为转发节点，以使业务集群中的所述目标主机通过所述转发节点进行实时消息分发。The configuration module is configured to deploy a target node to a target host in a business cluster, configure the real-time message distribution parameters on the target node, and use the target node configured with the real-time message distribution parameters as a forwarding node, so that the target host in the business cluster performs real-time message distribution through the forwarding node.
一种分布式实时消息分发装置，包括：A distributed real-time message distribution device, comprising:

第二获取模块，设置为获取待分发的实时消息；A second acquisition module is configured to acquire real-time messages to be distributed;

确定模块，设置为确定部署了转发节点的业务集群；所述转发节点配置有实时消息分发参数；A determination module, configured to determine a service cluster in which a forwarding node is deployed; the forwarding node is configured with real-time message distribution parameters;

分发模块，设置为通过所述转发节点按照所述实时消息分发参数对所述实时消息进行分发。The distribution module is configured to distribute the real-time message through the forwarding node according to the real-time message distribution parameter.
一种存储介质，其中，所述存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行权利要求1至8任一项中所述的方法。A storage medium, wherein a computer program is stored in the storage medium, wherein the computer program is configured to execute the method described in any one of claims 1 to 8 when run.
一种电子装置，包括存储器和处理器，其中，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行权利要求1至8任一项中所述的方法。 An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method described in any one of claims 1 to 8.