CN114598593A - Message processing method, system, computing device and computer storage medium - Google Patents
Message processing method, system, computing device and computer storage medium Download PDFInfo
- Publication number
- CN114598593A CN114598593A CN202210141634.3A CN202210141634A CN114598593A CN 114598593 A CN114598593 A CN 114598593A CN 202210141634 A CN202210141634 A CN 202210141634A CN 114598593 A CN114598593 A CN 114598593A
- Authority
- CN
- China
- Prior art keywords
- agent server
- message
- server
- standby
- main
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Hardware Redundancy (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a message processing method and a message processing system. The message processing method is applied to a distributed message system and comprises the following steps: the method comprises the steps that a first standby agent server scans a target message which meets the release requirement in a first main agent server under the condition that the first main agent server fails; sending the target message to a second main agent server; the target message is stored in a message queue of the second main agent server side for the consumer to consume the target message. The technical scheme provided by the embodiment of the invention realizes the technical effect of ensuring the normal consumption of the target message under the condition of not switching the main proxy server and the standby proxy server.
Description
Technical Field
The embodiment of the invention relates to the field of cloud computing, in particular to a message processing method, a message processing system, a message processing device and a computer storage medium.
Background
The rockmq system is a distributed and open-source distributed message system, and is widely used in the field of message processing.
In the related art, a service end of a rocktmq system usually adopts a regional deployment mode, a Name server (route discovery service end) and a Brokersever (proxy service end) are deployed in each region, and a master-slave cold standby scheme is adopted to improve disaster tolerance. The master-slave cold standby scheme is to deploy a master proxy server and a slave proxy server, wherein in a partition deployment mode, the master proxy server and the slave proxy server corresponding to the master proxy server are deployed in different areas, namely, the master and the slave proxy servers are deployed in a mixed manner in the same area, and the master proxy server and the slave proxy server are both provided. The main proxy server can be read and written, the standby proxy server serves as a copy of the main proxy server, has the full data on the main proxy server, can only receive the data synchronized by the main proxy server, and is readable and unwritable.
Because the RocktMQ supports some special types of target messages, such as timing messages, transaction messages, delay messages and the like, the target messages are firstly stored in special Topic (theme), and when the release condition is met, the main service end releases the target messages to the message queue, so that the consumers can consume the target messages. In a deployment mode of an open-source master-slave cold standby scheme, if a proxy server in one area has a problem, for example, a master proxy server in a first area is disconnected, since a standby proxy server cannot maintain a target message, in order to ensure normal consumption of the target message, the master proxy server in the first area needs to be pulled up again. In the process of the active/standby switching, the consumption of the target message may be delayed or lost.
Disclosure of Invention
The embodiment of the invention provides a message processing method and a message processing system.
In a first aspect, an embodiment of the present invention provides a message processing method applied in a distributed message system, including:
the method comprises the steps that a first standby agent server scans a target message which meets the release requirement in a first main agent server under the condition that the first main agent server fails;
sending the target message to a second main agent server; the target message is used for being stored in a message queue of the second main agent server side, so that the consumer can consume the target message.
In a second aspect, an embodiment of the present invention provides a message processing method, which is applied to a distributed message system, and the method includes:
under the condition that the first main proxy server recovers work, the first main proxy server acquires the address of the first standby proxy server from the route discovery server;
the first main agent server establishes communication connection with the first standby agent server based on the address;
the first master agent server acquires metadata sent by the first standby agent server through the communication connection, so that the first master agent server determines whether to update the metadata according to the version number of the metadata, wherein the version number of the metadata is determined according to the consumption number of target messages under the condition that the first master agent server fails, and the target messages comprise messages which are pulled from the first master agent server by the first standby agent server and sent to a second master agent server.
In a third aspect, an embodiment of the present invention provides a message processing method, applied to a second master proxy server of a distributed message, including:
acquiring a target message sent by a first standby agent server; the target message is obtained by scanning from a first main agent server side when the first standby agent server side fails;
and storing the target message into a message queue for the consumer to consume the target message from the message queue.
In a fourth aspect, an embodiment of the present invention provides a first standby agent server, which includes a scanning module and a sending module.
The scanning module is used for scanning a target message which meets the release requirement in a first main agent server under the condition that the first main agent server fails;
the sending module is used for sending the target message to a second main agent server; the target message is used for being stored in a message queue of the second main agent server side, so that the consumer can consume the target message.
In a fifth aspect, an embodiment of the present invention provides a second master proxy server, including:
the target message acquisition module is used for acquiring a target message sent by a first standby agent server, wherein the target message is obtained by scanning the first standby agent server from a first main agent server under the condition that the first main agent server fails;
and the storage module is used for storing the target message into a message queue so as to enable a consumer to consume the target message from the message queue.
In a sixth aspect, an embodiment of the present invention provides a first master proxy server, including:
the address acquisition module is used for acquiring the address of the first standby agent server from the route discovery server under the condition that the first main agent server recovers work;
the connection establishing module is used for establishing communication connection with the first standby agent server side based on the address;
and the metadata acquisition module is used for acquiring metadata sent by the first standby agent server through the communication connection so that the first main agent server determines whether to update the metadata according to the version number of the metadata, wherein the version number of the metadata is determined according to the consumption number of target messages under the condition that the first main agent server fails, and the target messages comprise messages which are pulled from the first main agent server by the first standby agent server and sent to a second main agent server.
In a seventh aspect, an embodiment of the present invention provides a computing device, including a processing component and a storage component;
the processing component is used for operating the first standby agent server and/or the second main agent server;
the storage component stores one or more computer instructions; the one or more computer instructions are called and executed by the processing component to implement the message processing method provided by the embodiment of the invention.
In an eighth aspect, an embodiment of the present invention provides a computer storage medium, which stores a computer program, and when the computer program is executed by a computer, the computer program implements a message processing method according to the embodiment of the present invention.
In the embodiment of the invention, under the condition that the first main agent server fails, the first standby agent server corresponding to the first main agent server is utilized to scan the target message meeting the release requirement in the first main agent server, and the target message is sent to the second main agent server deployed in the same physical machine as the first standby agent server, so that the technical scheme that a consumer consumes the target message from the consumption queue of the second main agent server is provided, and the technical effect of ensuring the normal consumption of the target message under the condition of not switching the main agent server and the standby agent server is realized.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating the structure of a message queue system provided by the present invention;
fig. 2 is a flowchart of an embodiment of a message processing method according to the present invention;
FIG. 3 schematically illustrates a timeline diagram of the receipt of a targeted message and the failure of a first master agent server;
fig. 4 is a flowchart of an embodiment of a message processing method according to another embodiment of the present invention;
FIG. 5 is a diagram schematically illustrating a message processing method in the case where the first master proxy server resumes operation;
fig. 6 is a flowchart of an embodiment of a message processing method according to another embodiment of the present invention;
FIG. 7 is an architecture diagram of a message processing system according to an embodiment of the present invention;
fig. 8 is a block diagram of a first standby agent server according to an embodiment of the present invention;
fig. 9 is a block diagram of a second master agent server according to an embodiment of the present invention;
fig. 10 is a block diagram of a first master agent server according to an embodiment of the present invention;
fig. 11 is a block diagram of a computing device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.
In some flows described in the present specification and claims and above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, with the order of the operations being given as 101, 102, etc. merely to distinguish between various operations, and the order of the operations itself does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit the types of "first" and "second".
The technical scheme provided by the embodiment of the invention can be applied to a distributed message system and is used for enabling the target message stored by a main agent server to be normally consumed under the condition that the main agent server of the distributed message system fails.
The RocketMQ system is a distributed and open-source distributed message system, has the characteristics of low delay, high concurrency, high availability, high reliability and the like, and is widely applied to the field of message processing. The rockmq system, like other types of distributed message systems, supports publishing and subscribing of messages, and also supports implementation of some target messages, such as transaction messages, timing messages, delay messages, and the like.
A distributed messaging system may include a producer, a consumer, and a server. For convenience of understanding, as shown in fig. 1, a schematic structural diagram of a distributed message system is shown, and the distributed message system may include a Producer (Producer)101, a Consumer (Consumer)102, a route discovery server (Name server)103, and a Broker server (Broker server)104, where the Producer, the Consumer, the route discovery server, and the Broker server may all be deployed in a cluster form and are formed by multiple nodes, and the Producer and the Consumer mean both clients with respect to the route discovery server and the Broker server. In one practical application, for example, the rockmq system is implemented using the structure shown in fig. 1.
In connection with the system architecture diagram shown in fig. 1, the following first explains the technical terms that may be involved in this document:
message (message): and a carrier for carrying and transmitting information in the message queue.
Topic (Topic): one type of message, the published message is of a certain type, called Topic.
Broker server (Broker server): the nodes providing the publish/subscribe function in the distributed message system are in charge of storing messages, forwarding messages and the like for the role of message transfer.
Message queue: messages are stored under each Topic by one to more message queues.
Route discovery server (Name server): the node is used for storing routing information in the distributed message system, and the routing information can comprise a browser server list and a topic list in the browser server. The list of affiliated brokerservers of publish/subscribe topic can be obtained through the Name server. After the Broker server is started, a registration request is sent to each Name server, and metadata such as an address of the Broker server and a Topic list are reported.
Producer (Producer): also known as message publishers, are responsible for producing and sending messages. In a distributed message system, a producer can randomly establish long connection with a route discovery server, acquire a Broker server in which Topic of a message to be issued is located, then establish long connection with the Broker server, and periodically send a heartbeat packet to the Broker server.
Consumer (Consumer): also known as message subscribers, are responsible for receiving and consuming messages. In the distributed message system, the consumer can also randomly establish a long connection with a route discovery server, obtain the Broker server where the subscribed Topic is located from the long connection, and then establish the long connection with the Broker server.
Timing messages: the Producer sends a message to the Broker server, but does not expect this message to be delivered immediately, but rather defers to delivery to the Consumer for consumption at some time after the current point in time, i.e., a timed message. The timing message issued by the producer is temporarily stored in the message storage component, and is written into the message queue after reaching a certain time.
And (3) delaying messages: the Producer sends a message to the Broker server, but rather than expecting the message to be delivered immediately, the message is delayed for a certain time before being delivered to the Consumer for consumption, i.e., a delayed message. The timing message issued by the producer will be temporarily stored in the message storage component, and then written into the message queue after a certain time delay.
Transaction message: multiple messages are consumed either simultaneously or not, and the message with the characteristic is a transaction message. The producer can issue a plurality of semi-transaction messages carrying the same transaction identifier, the Broker server temporarily stores the semi-transaction messages into the storage engine, the producer can send a confirmation request to the Broker server after the transaction execution is finished, and the Broker server writes the plurality of semi-transaction messages temporarily stored in the same transaction into the message queue as transaction information.
Message storage component (commit log): all messages received by the rockmq are saved.
Metadata, data in the rockmq for recording consumption and sending states, such as consumption sites, timing message replay sites, and the like.
The Raft consensus algorithm is a distributed protocol with strong consistency, decentralization and high availability, and can be used for switching the election masters.
RTO (Recovery Time Objective) is a unit of Time, i.e., the Time after a disaster occurs, from when an information system or service function stops to when it must be recovered.
The RPO (Recovery Point Object) refers to an elapsed time Point to which data can be recovered after a disaster or an emergency occurs, and is a data loss amount tolerable by a service system.
In the related art, a server of a distributed message system usually adopts a partition deployment mode, a Name server and a Brokersever are deployed in each area, and a master-slave cold standby scheme is adopted to improve disaster tolerance. The master-slave cold standby scheme is to deploy a master proxy server and a slave proxy server, wherein in a partition deployment mode, the master proxy server and the slave proxy server corresponding to the master proxy server are deployed in different areas, namely, the master and the slave proxy servers are deployed in a mixed manner in the same area, and the master proxy server and the slave proxy server are both provided. The main proxy server can be read and written, the standby proxy server serves as a copy of the main proxy server, has the full data on the main proxy server, can only receive the data synchronized by the main proxy server, and is readable and unwritable.
Because the distributed message supports some special types of target messages, such as timing messages, transaction messages, delay messages and the like, the target messages are firstly stored in special Topic, and when the releasing condition is met, the main service end releases the target messages to the message queue, so that consumers can consume the target messages. In a deployment mode of an open-source master-slave cold standby scheme, if a proxy server in one area has a problem, for example, a master proxy server in a first area is disconnected, since a standby proxy server cannot maintain a target message, in order to ensure normal consumption of the target message, the standby proxy server deployed in a second area needs to be manually pulled into the master proxy server, or the master proxy server in the first area needs to be pulled up again. In the process of the primary-standby switching, consumption delay or loss of the target message may be caused.
In the related art, a Raft hot standby scheme is adopted to improve disaster recovery capability. The raw hot standby scheme can be used for overcoming the defect that the automatic fault transfer capability does not exist in the proxy server group in the master-slave cold standby scheme, namely after the master proxy server fails, the backup proxy server needs to be pulled into the master proxy server manually.
In the Raft hot standby scheme, after the main agent server fails, a new main agent server can be automatically selected from the system to provide services through a Raft algorithm.
The method has the advantages that the method can complete fault transfer (fai loader) without depending on external components, but due to the limitation of the Raft algorithm, the Raft hot standby scheme has to guarantee the deployment modes of three copies and more in the proxy server group, the resource utilization rate is low, and due to the limitation of the Raft algorithm based on most assignments, long election time exists during fault transfer execution.
In addition, zookeeper-based hot standby solutions also exist in the related art. Zookeeper-based hot standby schemes rely on the external components zookeeper and Controller to ensure high availability, providing switching capability in the event of a failure. The zookeeper function is mainly a persistent storage state machine, stores the state of the proxy server in a temporary node mode, and provides a mechanism for notifying state change. And the Controller component observes the state change of the proxy server through the zookeeper, executes the state machine change and pushes a new state machine to the zookeeper. And the proxy server reports the state of the proxy server to the zookeeper, and monitors the change of a storage state machine in the zookeeper to complete fault transfer.
The disadvantage of zookeeper based hot standby is that zookeeper and Controller need to rely on external components, which puts extra operational burden, and there is a long election time in performing failover since zookeeper consistent ZAB protocol is based on the requirements of most parties.
In order to solve at least part of technical problems existing in the related art, in the embodiment of the present invention, in order to improve the disaster tolerance of the distributed message system and ensure that the target message stored by the master agent server can be normally consumed under the condition that the master agent server of the distributed message system fails, the inventors have proposed a technical solution of the present invention through a series of studies, in the embodiment of the present invention, a first standby agent server scans the target message that meets the delivery requirement in a first master agent server under the condition that the first master agent server fails; sending the target message to a second main agent server deployed in the same physical machine as the first standby agent server; the target message is stored in a message queue of the second main agent server side for the consumer to consume the target message.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 2 is a flowchart of an embodiment of a message processing method provided in an embodiment of the present invention, where the method may be applied to a distributed message system, and the method may include the following steps:
201, a first standby agent server scans a target message which meets the release requirement in a first main agent server under the condition that the first main agent server fails.
According to the embodiment of the invention, the first standby agent server can be a copy of the first main agent server, wherein the first main agent server can be read and written, and the first standby agent server can be used as a copy of the main agent server, has the full data on the first main agent server, can only receive the data synchronized by the first main agent server, and can be read and not be written.
According to the embodiment of the invention, the first standby agent server and the first main agent server can be pre-established with a long connection communication link, and data synchronization is carried out in real time or periodically by using the long connection communication link.
According to embodiments of the invention, the distributed messages may comprise a RockketMQ system.
According to an embodiment of the present invention, the target message may be a non-real-time message supported by the rockmq system, that is, the target message is not expected to be consumed by the consumer immediately after being delivered to the rockmq system by the producer, but is consumed by the consumer after the target message meets the delivery requirement.
According to embodiments of the present invention, the delivery requirements may include, for example, arrival of delivery time or receipt of a delivery request, etc.
According to an embodiment of the present invention, the target message may be a message that is sent to the rockmq system before the first master proxy server fails and that does not satisfy the release requirement before the first master proxy server fails.
FIG. 3 schematically shows a timeline diagram of the receipt of a targeted message and the failure of a first master agent server.
As shown in fig. 3, 301 is a time line, and indicates the time passage in the direction of the arrow. A may refer to a time point when the target message is sent to the rocktmq system, B may refer to a time point when the first master agent server fails, and C may refer to a time point when the target message satisfies the delivery requirement. That is, the failure of the first master agent server occurs between the time when the rockmq system receives the target message and the time when the target message meets the delivery requirement, the target message is not delivered in time, and the first master agent server fails.
According to an embodiment of the present invention, in the rockmq system, a target message of non-real-time may be referred to as a secondary message, and the secondary message is generally only authorized to be maintained by the master proxy server.
Since the target message is received when the first master agent server works normally and is not delivered before the first master agent server fails, the failure of the first master agent server may cause that the secondary message stored in the first master agent server cannot be delivered normally.
In the related art, after the first master agent server fails, the first master agent server generally needs to be pulled up to ensure normal consumption of the secondary message stored by the first master agent server. Or, the copy of the first master proxy server, that is, the first standby proxy server, may be pulled to the master proxy server, so that the first standby proxy server switched to the master proxy server performs delivery of the secondary message. However, the time for pulling up the first master proxy server or pulling up the first standby proxy server to the master proxy server is usually long, which may cause delay or loss of the secondary message. For example, in fig. 3, the delivery time of the target message is time point C, and the operation of pulling up the first master agent server or pulling up the first slave agent server as the master agent server can only be completed at time point D after time point C, so that the target message that should be delivered at time point C needs to be delayed until time point D before being delivered, which seriously affects user experience.
In the embodiment of the present invention, after the first master agent server fails, the first backup agent server may be used to scan the target message stored in the first master agent server by acting on the first master agent server, and pull out the target message meeting the release requirement, so that the consumer consumes the target message meeting the release requirement, without waiting for pulling up the first master agent server or pulling up the first backup agent server into the master agent server to scan the target message.
202, sending the target message to a second main agent server; the target message is stored in a message queue of the second main agent server side for the consumer to consume the target message.
According to the embodiment of the invention, due to the readable and non-writable property of the first standby agent server, after the first standby agent server pulls out the target message meeting the release requirement from the first main agent server, the target message can be sent to another main agent server, such as a second main agent server.
According to the embodiment of the invention, due to the readable and writable property of the second main agent server, after the second main agent server receives the target message, the target message can be stored in the message queue of the second main agent server for the consumer to consume the target message.
According to the embodiment of the invention, the area can refer to physical isolation of the main proxy server and the standby proxy server, and the fault of the proxy server in one area can not affect the corresponding proxy server in the other area. Different areas may refer to different machine rooms.
According to the embodiment of the invention, after the target message is stored in the consumption queue of the second main agent server, the consumer can consume the target message only by pulling the target message from the message queue of the second main agent server.
Due to the high availability mechanism of the consumer client, namely when the consumer client consumes the messages, each main proxy server in the RockettMQ system is traversed until the expected target messages are pulled, the process that the first standby proxy server acts on the first main server to pull the target messages and puts the target messages to the second main proxy server is insensitive to the consumer client, and the user experience is improved.
In the embodiment of the invention, under the condition that the first main agent server fails, the first standby agent server corresponding to the first main agent server is utilized to scan the target message meeting the release requirement in the first main agent server, and the target message is sent to the second main agent server deployed in the same physical machine as the first standby agent server, so that the technical scheme that a consumer consumes the target message from the consumption queue of the second main agent server is provided, and the technical effect of ensuring the normal consumption of the target message under the condition of not switching the main agent server and the standby agent server is realized. Compared with a Raft hot standby scheme and a zookeeper-based hot standby scheme in the related art, the message processing method provided by the embodiment of the invention has the advantages that the main standby switching is not required, the RTO time is shorter, and no external component is required, so that no additional operation and maintenance burden exists.
According to the embodiment of the invention, the first standby agent server can send the pulled target message to any normally working main agent server in the same area as the first standby agent server. Under the condition that more than two normally working main proxy servers exist in the same area, the first standby proxy server can randomly select one of the main proxy servers to send the target message.
According to the embodiment of the invention, if the first proxy server sends the pulled target message to the second master proxy server which is not in the same physical machine as the first proxy server, more internal traffic flows will be generated in the same area, which causes waste of network resources.
According to the embodiment of the invention, the second main agent server and the first standby agent server are deployed in the same physical machine.
According to the embodiment of the invention, the target message is sent to the second main proxy server which is deployed in the same physical machine with the first standby proxy server, so that the technical effects of increasing the utilization rate of network resources and saving the network resources can be realized.
According to the embodiment of the present invention, step 201 may be specifically implemented as:
starting a scanning thread corresponding to the first main agent server under the condition that the first main agent server is monitored to be in fault;
and scanning the message storage component of the first main agent server by using the scanning thread so as to acquire the target message from the message storage component.
According to embodiments of the present invention, until the placement requirements are met, the targeted message is typically stored in a special Topic of the first master broker server message storage component, the special Topic not being visible to the consumer.
According to the embodiment of the invention, after monitoring that the first master agent server fails, the first slave agent server can scan the special Topic of the message storage component of the first master agent server in real time or periodically, and pull out the target message meeting the release requirement after each scanning.
According to the embodiment of the invention, the disaster recovery mechanism of the RocketMQ system can ensure normal consumption of the target message, and the common message can be normally issued and consumed.
According to the embodiment of the invention, each agent server of the RockettMQ system registers with the route discovery server before going online, and after registration, each agent server sends heartbeat to the route discovery server in work, and the route discovery server maintains a routing table representing the working state of all the agent servers in the system. If a certain proxy server side has heartbeat disconnection, the routing discovery server side will remove the proxy server side from the routing table.
For example, the rocktmq system has a machine room a and a machine room B, where the machine room a is deployed with a first main proxy server, a second standby proxy server and a first route discovery server, and the machine room B is deployed with a first standby proxy server, a second main proxy server and a second route discovery server.
Before the first main proxy server and the second standby proxy server are on line, the first main proxy server and the second standby proxy server can register in the first route discovery server and the second route discovery server respectively, and send heartbeats to the first route discovery server and the second route discovery server respectively in work. Correspondingly, the first standby agent server and the second main agent server can register in the first route discovery server and the second route discovery server respectively before going online, and send heartbeats to the first route discovery server and the second route discovery server respectively during working.
If the machine room a fails, that is, the first main proxy server, the second standby proxy server and the first route discovery server are down, the second route discovery server discovers that the first main proxy server and the second standby proxy server are in heartbeat disconnection, so that the first main proxy server and the second standby proxy server can be removed from the routing table, and at the moment, only the routing information of the first standby proxy server and the second main proxy server which can still normally work at present is recorded in the routing table.
When a producer needs to issue a message, the producer will first access the route discovery server to pull the route information of the proxy server in the system. The process of the producer accessing the route discovery server can be random access, namely, one of the first route discovery server and the second route discovery server is randomly selected to access, if the route information can be pulled, the pulled route information is used for issuing the message, and if the route information cannot be pulled, the other route discovery server in the system is selected to access until the route information is pulled.
After the first main proxy server and the second standby proxy server are failed, the second routing discovery server removes the first main proxy server and the second standby proxy server from the routing table, so that the routing information of the first main proxy server and the second standby proxy server in the routing information pulled by the producer is invisible, the producer can issue the message by using the second main proxy server visible in the routing information, and the normal message consumption is not interrupted and the message is not lost.
For the producer, the second main proxy server exists in the system, so that the message can still be normally issued. For the consumer, the message can also be pulled normally from the second master broker server.
According to the embodiment of the invention, the first standby agent server monitors the fault condition of the first main agent server by the following steps:
acquiring a routing table from a route discovery server, wherein the routing table is used for recording heartbeat information of a first main agent server and deleting the heartbeat information of the first main agent server from the routing table after the heartbeat information from the first main agent server is not received within preset time;
and under the condition that the heartbeat information of the first main agent server does not exist in the routing table, determining that the first main agent server fails.
According to the embodiment of the invention, the first standby agent server can periodically acquire the routing table from the route discovery server, and confirm whether the route discovery server rejects the first main agent server from the routing table, if the route discovery server rejects the first main agent server from the routing table, it indicates that the first main agent server has heartbeat disconnection, that is, the first main agent server has a fault, so that the first standby agent server can perform the operation of scanning the target message which meets the release requirement in the first main agent server. If the heartbeat information of the first main proxy server exists in the routing table and represents that the first main proxy server is in a normal working state, the first standby proxy server only needs to receive and store the data synchronized by the first main proxy server.
According to the embodiment of the invention, the working state of the first main agent server can be determined by acquiring the routing table from the route discovery server and inquiring the routing table without additionally establishing a communication link with the first main agent server, so that the complexity of monitoring the working state of the first main agent server is simplified.
According to the embodiment of the invention, the routing table is further used for recording heartbeat information of the second main agent server;
according to the embodiment of the present invention, sending the target message to the second master agent server deployed in the same physical machine as the first standby agent server may specifically be implemented as follows:
determining whether heartbeat information of a second main agent server exists in a routing table;
and under the condition that the heartbeat information of the second main agent server exists in the routing table, sending the target message to the second main agent server.
According to the embodiment of the invention, the working state of the second main agent server can be determined by determining whether the heartbeat information of the second main agent server exists in the routing table, if the heartbeat information of the second main agent server exists in the routing table, the second main agent server is in a normal working state and does not have a fault, and at the moment, the target message can be sent to the second main agent server, so that a consumer can pull the target message from the message queue of the second main agent server.
If the heartbeat information of the second main proxy server does not exist in the routing table, it indicates that the second main proxy server is disconnected by heartbeat, that is, the second main proxy server fails, and at this time, the first standby proxy server may determine another main proxy server, for example, a third main proxy server, from the routing table and send the target message to the third main proxy server. The third master agent server may be a master agent server deployed in the same machine room as the first standby agent server.
According to the embodiment of the present invention, scanning the target message that meets the delivery requirement in the first master agent server may specifically be implemented as:
traversing messages to be processed stored in a message storage component of a first main agent server;
and determining the messages to be processed meeting the release requirements as target messages.
According to an embodiment of the present invention, the first master agent server may utilize the scan thread to scan the information to be processed in the special Topic in the message storage component.
According to the embodiment of the invention, the first standby agent server side can temporarily not process the to-be-processed information which does not meet the release requirement until the to-be-processed information meets the release requirement.
According to the embodiment of the invention, for example, when the first standby agent server scans the special Topic of the message storage component of the first main agent server by using the scanning thread at the first time, the message a and the message B do not meet the release requirement, and the message C meets the release requirement, the message C can be pulled and sent to the second main agent server, so that the message C is consumed successfully.
After the message C is successfully consumed, the message a and the message B are also stored in the special Topic of the message storage component of the first master agent server. At the second moment, the first standby agent server may query the routing table again, determine whether the first main agent server resumes the online state, and if the first main agent server does not resume the online state, the first standby agent server may restart the scanning thread to scan the special Topic of the message storage component of the first main agent server at the second moment, and pull the message meeting the release requirement at the second moment.
According to an embodiment of the present invention, in case the pending message comprises a timing message, the placement requirement comprises:
the timing message reaches the preset delivery time;
in the case that the pending message comprises a transaction message, the placement requirement comprises:
acknowledgement information for the transaction message is obtained.
Fig. 4 is a flowchart of an embodiment of a message processing method according to another embodiment of the present invention, where the method may be applied to a distributed message system, and the method may include the following steps:
401, when the first master agent server recovers, the first master agent server obtains the address of the first standby agent server from the route discovery server;
402, the first main proxy server establishes communication connection with the first standby proxy server based on the address;
403, the first master agent server obtains metadata sent by the first standby agent server through the communication connection, so that the first master agent server determines whether to update the metadata according to a version number of the metadata, where the version number of the metadata is determined according to a consumption number of a target message under a condition that the first master agent server fails, and the target message includes a message that the first standby agent server pulls from the first master agent server and sends to the second master agent server.
According to the embodiment of the present invention, the specific flow of the first standby agent server pulling from the first main agent server and sending to the second main agent server is the same as or similar to the operation of the message processing method shown in fig. 2, and the embodiment of the present invention is not described herein again.
Fig. 5 is a schematic diagram schematically showing a message processing method in the case where the first master agent server resumes work.
As shown in fig. 5, after the failure of the first master agent server 501 is repaired, the first master agent server 501 may first access the route discovery server 502 to obtain the address of the first backup agent server 502, but at this time, the first backup agent server is not registered in the route discovery server 502, that is, at this time, the first master agent server 501 is temporarily not formally online and is invisible to the outside.
After obtaining the address of the first standby agent server 502, the first master agent server 501 may send the address of the first master agent server 501 to the first standby agent server 502.
After receiving the address of the first master agent server 501, the first slave agent server 502 may establish a communication connection with the first master agent server 501, complete a handshake, and send the metadata of the first slave agent server 502 to the first master agent server 501 based on the established communication connection.
According to embodiments of the invention, the metadata may include consumption sites, timing message progress, and the like.
According to the embodiment of the present invention, after the first master proxy server 501 fails, the first standby proxy server 502 proxies the first master proxy server 501 to store and deliver the target message, so that the master-standby relationship of the proxy server of the rocktmq system does not change. Due to the readable and non-writable nature of the first standby agent server 502, after the first master agent server 501 is recovered, the commimitlog of each of the first standby agent server 502 and the first master agent server 501 has data consistency.
However, after the first master proxy server 501 goes offline, the first standby proxy server 502 takes the role of standby reading and proxies the target message, and thus, part of the metadata in the first standby proxy server 502 may be more advanced than the metadata of the first master proxy server 501. In the related art, if the first master agent server 501 goes online again, the metadata is updated based on the metadata of the first master agent server 501, that is, the metadata is synchronized with the first standby agent server 502, which may cause the metadata of the group of agent servers to go back, and a large amount of messages are repeatedly consumed.
In the embodiment of the present invention, after the first master agent server 501 comes online, the first master agent server 502 reversely obtains the metadata of the first standby agent server 502, and determines whether to update the metadata based on the received version number of the metadata of the first standby agent server 502, so as to solve the technical problem that a large amount of messages are repeatedly consumed due to metadata rollback in the related art.
In an embodiment of the invention, the concept of adding version numbers to the metadata innovatively, and in order to prevent version number updates from being too frequent, the concept of a step size is also proposed, for example, for a consumption site, the version number of the consumption site can be updated to the next version every 500 times the consumption site is updated. It should be noted that the update step size of the version number can be flexibly set by those skilled in the art according to the actual application requirement, and may be, for example, 300, 400, 700, 1000, and the like.
In a specific example, before the first master proxy server fails, the version number of the consumption site is 2.1.1, and correspondingly, the version number of the consumption site of the first slave proxy server is also 2.1.1. In the process from the first main agent server being offline due to a fault to the first main agent server being online again after the fault is repaired, due to the target message agent operation of the first standby agent server, the version number of the consumption site of the standby agent server is updated to 2.1.2, that is, in the time from the fault of the first main agent server to the recovery of the first main agent server, the first standby agent server agent consumes 500 entry mark messages. In this case, if the metadata of the first standby agent server is synchronized by using the consumption bit with the version number of 2.1.1 after the first master agent server goes back online, the 500 entry mark messages that have been consumed may be repeatedly consumed.
In the embodiment of the present invention, after the first master agent server goes back online, the first master agent server may reversely obtain the metadata of the first standby agent server, and with reference to the above example, after the first master agent server obtains the metadata of the first standby agent server, it is determined that the version number of the metadata of the first standby agent server has been updated, and then the first master agent server may also correspondingly update its own consumption site to 2.1.2, thereby avoiding the repeated consumption of the message.
According to the embodiment of the present invention, after the metadata of the first standby agent server is obtained in the reverse direction, if the version number of the metadata of the first standby agent server is the same as the version number of the first master agent server, it indicates that the first standby agent server does not perform consumption of the target message by acting on the first master agent server during the failure of the first master agent server, and at this time, the metadata of the first standby agent server may be covered by the metadata of the first master agent server.
Fig. 6 is a flowchart of an embodiment of a message processing method according to another embodiment of the present invention, where the method may be applied to a second master agent server of a distributed message system, and the method may include the following steps:
601, acquiring a target message sent by a first standby agent server; the target message is obtained by scanning the first standby agent server from the first main agent server under the condition that the first main agent server fails;
and 602, storing the target message in a message queue for the consumer to consume the target message from the message queue.
Fig. 7 is an architecture diagram of a message processing system according to an embodiment of the present invention, where the message processing system 700 includes a first master agent server 701, a second master agent server 702, and a first standby agent server 703 corresponding to the first master agent server 701; the first standby agent server 703 and the second main agent server 702 are located in different areas from the first main agent server 701 respectively;
the first standby agent server 703 is configured to:
under the condition that the first main agent server 701 fails, scanning a target message which meets the release requirement in the first main agent server 701;
sending the target message to a second master agent server 702 deployed in the same physical machine as the first slave agent server 703; the target message is stored in the message queue of the second master agent server 702 for the consumer to consume the target message.
According to the embodiment of the present invention, the message processing system 700 further includes a second standby agent server 704 corresponding to the second master agent server 702. It should be noted that the second standby agent server 704 is not limited to be deployed in the area a, and may also be deployed in an area C different from the area B.
According to embodiments of the present invention, zone A and zone B in message processing system 700 each include a NameServer.
According to some embodiments of the invention, each proxy server may be deployed in a physical machine individually.
According to the preferred embodiment of the present invention, each physical machine may also be deployed in a hybrid manner between a master physical machine and a slave physical machine, that is, at least one master agent server and at least one slave agent server may be deployed in one physical machine.
According to the embodiment of the invention, the first standby agent server and the second main agent server are deployed in the same physical machine in the same area.
According to the embodiment of the invention, the main and standby hybrid deployment is carried out in each physical machine, so that the internal flow circulation of the standby agent server during target message sending can be saved, and the utilization rate of network resources is improved.
Fig. 8 is a block diagram of a first standby agent server according to an embodiment of the present invention, where the first standby agent server 800 includes a scanning module 801 and a sending module 802.
A scanning module 801, configured to scan a target message that meets a delivery requirement in a first master agent server under a condition that the first master agent server fails;
a sending module 802, configured to send the target message to a second master agent server; the target message is stored in a message queue of the second main agent server side for the consumer to consume the target message.
The scan module 801 includes a start unit and a scan unit according to an embodiment of the present invention.
The starting unit is used for starting a scanning thread corresponding to the first main agent server under the condition that the fault of the first main agent server is monitored;
and the scanning unit is used for scanning the message storage component of the first main agent server by using the scanning thread so as to acquire the target message from the message storage component.
According to the embodiment of the present invention, the first standby agent server 800 further includes an obtaining module and a failure determining module.
The acquisition module is used for acquiring a routing table from the route discovery server, wherein the routing table is used for recording heartbeat information of the first main agent server and deleting the heartbeat information of the first main agent server from the routing table after the heartbeat information from the first main agent server is not received within preset time;
and the fault determining module is used for determining that the first main proxy server fails under the condition that the heartbeat information of the first main proxy server does not exist in the routing table.
According to the embodiment of the invention, the routing table is further used for recording heartbeat information of the second main agent server side.
The sending module 802 includes a heartbeat acknowledgment unit, a sending unit, according to an embodiment of the present invention.
The heartbeat confirming unit is used for determining whether heartbeat information of the second main proxy server exists in the routing table;
and the sending unit is used for sending the target message to the second main proxy server under the condition that the heartbeat information of the second main proxy server exists in the routing table.
According to an embodiment of the present invention, in case the pending message comprises a timing message, the placement requirement comprises:
the timing message reaches the preset delivery time;
in the case that the pending message comprises a transaction message, the placement requirement comprises:
acknowledgement information for the transaction message is obtained.
Fig. 9 is a block diagram of a second master agent server according to an embodiment of the present invention, where the second master agent server 900 includes:
a target message obtaining module 901, configured to obtain a target message sent by a first standby agent server, where the target message is obtained by scanning from a first main agent server when the first standby agent server fails;
a storage module 902, configured to store the target message in a message queue, so that the consumer consumes the target message from the message queue.
Fig. 10 is a block diagram of a first master agent server according to an embodiment of the present invention, where the first master agent server 1000 includes:
an address obtaining module 1001, configured to obtain, from the route discovery server, an address of the first standby agent server when the first master agent server resumes working;
a connection establishing module 1002, configured to establish a communication connection with the first standby agent server based on the address;
a metadata obtaining module 1003, configured to obtain metadata sent by the first standby agent server through the communication connection, so that the first master agent server determines whether to update the metadata according to a version number of the metadata, where the version number of the metadata is determined according to a consumption number of a target message when the first master agent server fails, and the target message includes a message that the first standby agent server pulls from the first master agent server and sends the message to a second master agent server.
The first standby proxy server in fig. 8 may execute the message processing method in the embodiment shown in fig. 2, and details of the implementation principle and the technical effect are not repeated. The specific manner in which each module and unit of the 8 devices in the above embodiments perform operations has been described in detail in the embodiments related to the method, and will not be described in detail herein.
In one possible design, the first standby agent server or the second master agent server of the embodiment shown in fig. 8 may be implemented as a computing device, as shown in fig. 11, which may include a storage component 1101 and a processing component 1102;
the storage component 1101 stores one or more computer instructions for execution by the processing component 1102 for implementing the message processing method shown in fig. 2.
Of course, a computing device may also necessarily include other components, such as input/output interfaces, communication components, and so forth. The input/output interface provides an interface between the processing component and a peripheral interface module, which may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, and the like.
The computing device may be a physical device or an elastic computing host provided by a cloud computing platform, and the computing device may be a cloud server, and the processing component, the storage component, and the like may be a basic server resource rented or purchased from the cloud computing platform.
When the computing device is a physical device, the computing device may be implemented as a distributed cluster consisting of a plurality of servers or terminal devices, or may be implemented as a single server or a single terminal device.
In practical applications, the computing device may be specifically deployed to a node in the distributed message system, and implemented as a proxy server in the distributed message system.
An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a computer, the message processing method according to the embodiment shown in fig. 2 may be implemented.
An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a computer, the message processing method in the embodiment shown in fig. 2 may be implemented.
The processing components in the respective embodiments above may include one or more processors executing computer instructions to perform all or part of the steps of the above methods. Of course, the processing elements may also be implemented as one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components configured to perform the above-described methods.
The storage component is configured to store various types of data to support operations in the device. The memory components may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A message processing method, applied to a distributed message system, the method comprising:
the method comprises the steps that a first standby agent server scans a target message which meets the release requirement in a first main agent server under the condition that the first main agent server fails;
sending the target message to a second main agent server; the target message is used for being stored in a message queue of the second main agent server side, so that the consumer can consume the target message.
2. The method of claim 1, wherein the scanning, by the first standby agent server, for the target message that meets the delivery requirement in the first master agent server in case of a failure of the first master agent server comprises:
under the condition that the first main agent server is monitored to be in fault, starting a scanning thread corresponding to the first main agent server;
and scanning a message storage component of the first main agent server by using the scanning thread so as to acquire the target message from the message storage component.
3. The method according to claim 1 or 2, wherein the first standby agent server monitors the failure condition of the first main agent server by:
acquiring a routing table from a routing discovery server, wherein the routing table is used for recording heartbeat information of the first main agent server and deleting the heartbeat information of the first main agent server from the routing table after the heartbeat information from the first main agent server is not received within preset time;
and determining that the first main proxy server fails under the condition that the heartbeat information of the first main proxy server does not exist in the routing table.
4. The method according to claim 3, wherein the routing table is further configured to record heartbeat information of the second master agent server;
the sending the target message to a second master agent server deployed in the same physical machine as the first slave agent server comprises:
determining whether heartbeat information of the second main agent server exists in the routing table;
and under the condition that the heartbeat information of the second main agent server exists in the routing table, sending the target message to the second main agent server.
5. The method of claim 1, wherein the second master broker server and the first standby broker server are deployed in the same physical machine.
6. The method of claim 5,
in a case that the pending message comprises a timing message, the placement requirement comprises:
the timing message reaches the preset delivery time;
in a case that the message to be processed includes a transaction message, the placement requirement includes:
and acquiring confirmation information aiming at the transaction message.
7. A message processing method, applied to a distributed message system, the method comprising:
under the condition that the first main proxy server recovers work, the first main proxy server acquires the address of the first standby proxy server from the route discovery server;
the first main agent server establishes communication connection with the first standby agent server based on the address;
the first master agent server acquires metadata sent by the first standby agent server through the communication connection so that the first master agent server can determine whether to update the metadata according to the version number of the metadata, wherein the version number of the metadata is determined according to the consumption number of target messages under the condition that the first master agent server fails, and the target messages comprise messages which are pulled from the first master agent server by the first standby agent server and sent to a second master agent server.
8. The message processing method is characterized in that the method is applied to a second main agent server of a distributed message system; the method comprises the following steps:
acquiring a target message sent by a first standby agent server; the target message is obtained by scanning a first main agent server side when a first standby agent server side fails;
and storing the target message into a message queue for the consumer to consume the target message from the message queue.
9. A computing device comprising a processing component and a storage component;
the processing component is used for operating the first standby agent server and/or the second main agent server;
the storage component stores one or more computer instructions; the one or more computer instructions are for execution by the processing component to invoke to implement the message processing method of any one of claims 1-6 and/or the message processing method of claim 7 and/or the message processing method of claim 8.
10. A computer storage medium, characterized in that a computer program is stored which, when executed by a computer, implements the message processing method of any one of claims 1-6 and/or the message processing method of claim 7 and/or the message processing method of claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210141634.3A CN114598593B (en) | 2022-02-16 | 2022-02-16 | Message processing method, system, computing device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210141634.3A CN114598593B (en) | 2022-02-16 | 2022-02-16 | Message processing method, system, computing device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114598593A true CN114598593A (en) | 2022-06-07 |
| CN114598593B CN114598593B (en) | 2023-08-29 |
Family
ID=81805448
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210141634.3A Active CN114598593B (en) | 2022-02-16 | 2022-02-16 | Message processing method, system, computing device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114598593B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114827278A (en) * | 2022-06-24 | 2022-07-29 | 小米汽车科技有限公司 | Task scheduling method, system, device and equipment based on message queue |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106953901A (en) * | 2017-03-10 | 2017-07-14 | 重庆邮电大学 | A kind of trunked communication system and its method for improving message transmission performance |
| CN107197015A (en) * | 2017-05-23 | 2017-09-22 | 阿里巴巴集团控股有限公司 | A kind of message treatment method and device based on Message Queuing system |
| CN108306941A (en) * | 2018-01-05 | 2018-07-20 | 上海你我贷互联网金融信息服务有限公司 | A kind of distributed information system |
| CN110601903A (en) * | 2019-09-25 | 2019-12-20 | 广州荔支网络技术有限公司 | Data processing method and device based on message queue middleware |
| US20200244726A1 (en) * | 2019-01-29 | 2020-07-30 | Verizon Patent And Licensing Inc. | Messaging system failover |
| CN112597249A (en) * | 2020-12-26 | 2021-04-02 | 湖南快乐阳光互动娱乐传媒有限公司 | Synchronous distribution and storage method and system for service data |
| US20210157691A1 (en) * | 2019-11-26 | 2021-05-27 | International Business Machines Corporation | Inter-processor communications fault handling in high performance computing networks |
| CN113641511A (en) * | 2021-07-09 | 2021-11-12 | 阿里云计算有限公司 | Message communication method and device |
| CN113849320A (en) * | 2021-08-20 | 2021-12-28 | 阿里巴巴新加坡控股有限公司 | Development method, control method and computing device |
-
2022
- 2022-02-16 CN CN202210141634.3A patent/CN114598593B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106953901A (en) * | 2017-03-10 | 2017-07-14 | 重庆邮电大学 | A kind of trunked communication system and its method for improving message transmission performance |
| CN107197015A (en) * | 2017-05-23 | 2017-09-22 | 阿里巴巴集团控股有限公司 | A kind of message treatment method and device based on Message Queuing system |
| CN108306941A (en) * | 2018-01-05 | 2018-07-20 | 上海你我贷互联网金融信息服务有限公司 | A kind of distributed information system |
| US20200244726A1 (en) * | 2019-01-29 | 2020-07-30 | Verizon Patent And Licensing Inc. | Messaging system failover |
| CN110601903A (en) * | 2019-09-25 | 2019-12-20 | 广州荔支网络技术有限公司 | Data processing method and device based on message queue middleware |
| US20210157691A1 (en) * | 2019-11-26 | 2021-05-27 | International Business Machines Corporation | Inter-processor communications fault handling in high performance computing networks |
| CN112597249A (en) * | 2020-12-26 | 2021-04-02 | 湖南快乐阳光互动娱乐传媒有限公司 | Synchronous distribution and storage method and system for service data |
| CN113641511A (en) * | 2021-07-09 | 2021-11-12 | 阿里云计算有限公司 | Message communication method and device |
| CN113849320A (en) * | 2021-08-20 | 2021-12-28 | 阿里巴巴新加坡控股有限公司 | Development method, control method and computing device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114827278A (en) * | 2022-06-24 | 2022-07-29 | 小米汽车科技有限公司 | Task scheduling method, system, device and equipment based on message queue |
| CN114827278B (en) * | 2022-06-24 | 2022-09-02 | 小米汽车科技有限公司 | Task scheduling method, system, device and equipment based on message queue |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114598593B (en) | 2023-08-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6438707B1 (en) | Fault tolerant computer system | |
| US9325757B2 (en) | Methods and systems for fault-tolerant distributed stream processing | |
| EP2902922B1 (en) | Distributed file system and data backup method for distributed file system | |
| US20030005350A1 (en) | Failover management system | |
| CN110601903B (en) | Data processing method and device based on message queue middleware | |
| US20020075870A1 (en) | Method and apparatus for discovering computer systems in a distributed multi-system cluster | |
| CN102394914A (en) | Cluster brain-split processing method and device | |
| US20210136145A1 (en) | Method for Changing Member in Distributed System and Distributed System | |
| CN110275680B (en) | Double-control double-active storage system | |
| CN110677282B (en) | Hot backup method of distributed system and distributed system | |
| EP2224341A1 (en) | Node system, server switching method, server device, and data transfer method | |
| CN102571850A (en) | Transaction committing system, method and equipment | |
| CN108173971A (en) | A kind of MooseFS high availability methods and system based on active-standby switch | |
| CN102055950B (en) | MCU backup method and equipment in video conference | |
| CN114124650A (en) | Master-slave deployment method of SPTN (shortest Path bridging) network controller | |
| CN114598593B (en) | Message processing method, system, computing device and computer storage medium | |
| CN108512753B (en) | Method and device for transmitting messages in cluster file system | |
| CN114363350A (en) | Service management system and method | |
| CN114598711B (en) | Data migration method, device, equipment and medium | |
| CN113992681B (en) | Method for guaranteeing strong consistency of data in distributed system | |
| CN114422335A (en) | Communication method, communication device, server and storage medium | |
| JP2009075710A (en) | Redundant system | |
| CN110716827A (en) | Hot backup method suitable for distributed system and distributed system | |
| CN113821334A (en) | Method, device and system for configuring edge side equipment | |
| Jehl et al. | Asynchronous reconfiguration for Paxos state machines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |