CN110968586B - Distributed transaction processing method and device - Google Patents
Distributed transaction processing method and device Download PDFInfo
- Publication number
- CN110968586B CN110968586B CN201811159001.5A CN201811159001A CN110968586B CN 110968586 B CN110968586 B CN 110968586B CN 201811159001 A CN201811159001 A CN 201811159001A CN 110968586 B CN110968586 B CN 110968586B
- Authority
- CN
- China
- Prior art keywords
- event
- publisher
- subscriber
- bus
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Multi Processors (AREA)
Abstract
The invention discloses a distributed transaction processing method and a device, wherein an event bus is used as a communication medium between a publisher and a subscriber, and the event bus is published by the publication direction event, and meanwhile, service data associated with the event is sent to a message queue; then, the event bus distributes the event to the corresponding subscriber, and decoupling of the transaction and the message queue of the service data is realized. Moreover, the event bus records event states after the event is operated by the publisher and subscriber, so that the consistency of event processing is ensured; the event bus determines the event status of the event and sends the event status to the publisher so that the publisher performs the corresponding operation. As can be seen from the above process, the event bus mechanism is adopted to effectively decouple the transaction and the message queue of the service data, i.e. effectively decouple the micro service and the message queue; moreover, the event state is recorded and updated by the event bus, and therefore, the micro service becomes lighter, reducing development and maintenance costs of the micro service.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a distributed transaction processing method and apparatus.
Background
A transaction is a logical unit in the execution of a database management system and consists of a finite sequence of database operations. When a transaction is committed to a DBMS (database management system), the DBMS needs to ensure that all operations in the transaction complete successfully and the results are permanently saved in the database, and if some operations in the transaction do not complete successfully, all operations in the transaction need to be rolled back to the state before the transaction is executed; at the same time, the transaction has no effect on the execution of the database or other transaction, all as if it were running independently.
A distributed transaction is a transaction that runs in a distributed environment, for which each operational step of the transaction may run on a different machine's service. The key to distributed transactions is that there must be a way to know all actions a transaction takes anywhere and the decision to commit or rollback a transaction must produce a uniform result, i.e., all commit or all rollback. The distributed system has CAP law, and the content of the law means that in one distributed system, consistency, availability, partition tolerance (partition fault tolerance) cannot be combined.
According to CAP law, at most two of them can be implemented in a distributed system, and thus, how to solve CAP in a distributed system is a problem that needs to be solved at present.
Disclosure of Invention
In view of the above problems, the present application provides a distributed transaction processing method and apparatus, so as to solve the technical problems of troublesome practical application and high maintenance cost in the later period caused by the coupling of the existing asynchronous assurance mode message table to the service system.
In a first aspect, the present application provides a distributed transaction processing method, including:
the event bus receives an event sent by a publisher and records the event state of the event;
the event bus acquires service data associated with the event from a message queue, and the service data is sent to the message queue by the publisher when the event is published;
the event bus sends the event and the business data associated with the event to a target subscriber subscribing the event;
the event bus updates an event state corresponding to the event according to the response message of the event returned by the target subscriber;
the event bus determines an event state corresponding to the event and sends the event state to a publisher corresponding to the event so that the publisher executes an operation corresponding to the event state.
Optionally, recording an event state of the event includes:
the event bus records event identifications of the events and corresponding event states in a message table, wherein the event identifications are unique identifications for identifying the events.
Optionally, recording an event state of the event includes:
the event bus records an event identifier, an event state and a subscriber identifier associated with the event corresponding to the event in a message table;
wherein the event identity is a unique identity identifying an event and the subscriber identity is a unique identity identifying a party to the subscribed event.
Optionally, recording an event state of the event includes:
the event bus records an event identifier and an event state corresponding to the event, and a publisher identifier and a subscriber identifier associated with the event in a message table;
the event identifier is a unique identifier for identifying an event, the publisher identifier is a unique identifier for identifying a party for publishing the event, and the subscriber identifier is a unique identifier for subscribing the event party.
Optionally, the message table is used for recording event states of all events in the system where the event bus is located, and the identification of the publisher and the identification of the subscriber associated with the event.
Optionally, the method further comprises:
the event bus scans the event states of all the recorded events, and cleans up the event information of the ended events in the record according to a preset strategy, wherein the ended events comprise successfully processed events and events with failure times exceeding a preset number.
In a second aspect, the present application further provides a distributed transaction processing apparatus, applied to an event bus, the apparatus comprising:
the recording module is used for receiving the event sent by the publisher and recording the event state of the event;
the acquisition module is used for acquiring service data associated with the event from a message queue, and the service data is sent to the message queue by the publisher when the event is published;
the sending module is used for sending the event and the business data associated with the event to a target subscriber subscribing the event;
the state updating module is used for updating the event state corresponding to the event according to the response message of the event returned by the target subscriber;
the state determining module is used for determining an event state corresponding to the event and sending the event state to a publisher corresponding to the event so that the publisher executes an operation corresponding to the event state.
Optionally, the recording module is specifically configured to record, in a message table, an event identifier of the event and a corresponding event state, where the event identifier is a unique identifier for identifying the event.
In a third aspect, the present application provides a storage medium having stored thereon a program which when executed by a processor implements the distributed transaction method provided by any one of the possible implementations of the first aspect.
In a fourth aspect, the present application further provides a processor, where the processor is configured to execute a program, where the program executes the distributed transaction processing method according to any one of the possible implementation manners of the first aspect.
The distributed transaction processing method provided by the embodiment adopts an event bus as a communication medium between a publisher and a subscriber, and the publisher publishes an event to the event bus and simultaneously sends service data associated with the event to a message queue; then, the event bus distributes the event to the corresponding subscriber, and decoupling of the transaction and the message queue of the service data is realized. Moreover, the event bus records event states after the event is operated by the publisher and subscriber, so that the consistency of event processing is ensured; the event bus determines the event status of the event and sends the event status to the publisher so that the publisher performs the corresponding operation. For example, if the service processing fails, the publisher performs operations such as event rollback, and if the service processing is successful, the publisher submits event processing to the upper layer application to complete. As can be seen from the above process, after the event bus mechanism is adopted, the message queues of the transaction and the service data are effectively decoupled, i.e. the micro service and the message queues are effectively decoupled; moreover, the event state is recorded and updated by the event bus, and therefore, the micro service becomes lighter, reducing development and maintenance costs of the micro service.
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 shows a schematic diagram of a publish-subscribe scheme;
FIG. 2 illustrates a flow chart of a distributed transaction processing method according to an embodiment of the application;
FIG. 3 illustrates a flow chart of another distributed transaction processing method in accordance with an embodiment of the present application;
fig. 4 shows a block diagram of a distributed transaction processing arrangement according to an embodiment of the application.
Detailed Description
The inventors have found that: in the current processing process of distributed transactions, the problem of CAP is usually solved by adopting an asynchronous ensuring mode, and the asynchronous ensuring mode needs a message producer to establish a message table for recording the state of message transmission. The message table and the service data are submitted in a transaction, i.e. the message table and the service data are in a database, i.e. the message table is coupled to the service system.
More and more are micro-service architecture systems, some of which may involve problems of distributed transactions, for example, at least including a micro-service 1 and a micro-service 2 in one system, wherein the communication between the two micro-services adopts asynchronous communication, the micro-service 2 provides data to the micro-service 1, and the micro-service 1 stores the data; how to ensure that the data provided by the micro service 2 is accurately written and that the situation of repetition or loss does not occur; the presence of distributed transactions, if the above-described asynchronous assurance approach (or referred to as a local message table) is employed, results in significant difficulty in the development of micro-services, as well as high maintenance costs. In order to solve the problem, the application provides a distributed transaction processing method, which adopts an event bus as a communication medium between a publisher and a subscriber, and the event bus operates a message table to ensure the consistency of distributed transactions and effectively decouple the coupling between the message table and a service system.
Referring to FIG. 1, a schematic diagram of a publish-subscribe model is shown, including subscribers, publishers and event buses, as follows:
the event bus is an implementation of a publish-subscribe model, which is a centralized event handling mechanism that allows different components to communicate with each other without interdependence, achieving a decoupling purpose.
Event (Event): which may also be referred to as a message, is herein collectively referred to as an event representation. The event may be regarded as an object, and may be a character string returned by the network request, or may be a certain switch state.
The Publisher (publishing) publishes an event via its interface by publishing the object of the event.
Subscriber (Subscriber): objects subscribing to a certain event type. When an event publisher publishes such an event, the event bus executes an event processing function of the event subscriber, which is an event response function, and the event processing function sends the event to the subscriber subscribing to the event.
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Referring to fig. 2, a flowchart of a distributed transaction processing method according to an embodiment of the present application is shown, where the method applies to a server;
The embodiment adopts an event bus mechanism as a communication medium between the publisher and the subscriber, monitors the states of the publisher and the subscriber by the event bus, and records and updates the event states of all events so as to ensure the consistency of the distributed transactions.
In this embodiment, it will be emphasized that the event status after the publisher and subscriber operate on the same event is recorded in a message table.
As shown in fig. 2, the distributed transaction processing method includes the steps of:
s110, the publisher publishes the event and sends service data associated with the event to a message queue.
The event comprises an event identification, a title and content of the event; the event identifier is a unique identifier of the event, and the event can be uniquely determined through the event identifier; the title of the event represents queue information of the event, and the subscriber subscribes to the event by subscribing to the title of the event, wherein the event identifier can comprise the event title; the event content characterizes the specific transactional operation of the event.
Injecting an interface for releasing the event into a controller of a releasing party, designating the title and the content of the released event, and releasing the event through the interface; and meanwhile, the business data associated with the issued event is sent to a message queue, so that the decoupling of the micro-service and the business data is realized.
Under an application scenario, an event to be issued by an issuer is not successfully issued, and the issuer retries according to a preset retry strategy. For example, when the first transmission fails, retry is continued 3 times; if the retry still fails, the retry may be repeated every predetermined time period until the number of retries reaches the predetermined number. The preset retry strategy may be set according to actual requirements. In order to adapt to the application scenario, after the publisher publishes the event through the interface for publishing the event, the information of the event is persisted so as to be subjected to retry publishing later.
S120, after receiving the event issued by the issuer, the event bus records the event state of the event.
The event bus needs to record the corresponding relation between the event identification and the current event state corresponding to the event, the corresponding relation between the publisher identification and the published event, and the corresponding relation between the subscriber identification and the subscribed event, and the three corresponding relations can be associated through the event identification.
In one embodiment of the application, the event bus may record the above information in the form of a message table. In other embodiments of the present application, the event bus may record the above information in other forms, and the present application is not limited in this regard.
In an application scenario, an event is subscribed by only one subscriber, and at least an event identifier and a corresponding event state are recorded in a message table; in addition, the publisher identifier and the subscriber identifier may be recorded in a message table, or may be recorded in a location other than the message table, for example, a storage location and a storage form specified by an event handling mechanism of the event bus itself, which is not limited in the present application.
In the case where an event is subscribed by only one subscriber, the embodiment is described by taking the case where the event status after the publisher and the subscriber operate on the same event is contained in one message table as an example. For example, a message table is created for each pair of the publisher and subscriber, and for the message table, only the event states of the events operated by the publisher and subscriber corresponding to the message table are stored in the message table; alternatively, at least two pairs of publishers and subscribers in the system share a message table.
In another application scenario, an event is subscribed by a plurality of subscribers, and at least three items of event identification, event state and subscriber identification are recorded in a message table; the publisher identifier may be recorded in a message table, or may be recorded elsewhere than in the message table, for example, in a storage location and storage form specified by an event handling mechanism of the event bus itself, which is not limited in this regard by the present application.
In the case that an event is subscribed by a plurality of subscribers, the embodiment takes only one message table as an example for a publisher and a plurality of subscribers corresponding to the publisher; for example, a publisher and corresponding subscribers have a message table; or at least two publishers and corresponding subscribers in the system share one message table. The subscriber corresponding to the publisher refers to a subscriber subscribing to the event published by the publisher.
In the scenario that event states after the publisher and subscriber operate on the same event are recorded in the same message table, the event states to be recorded in the message table may include: queuing, successful reception, failed processing, and successful processing;
wherein the enqueue list has stored the event into the event queue by the event bus; the successful receipt indicates that the event was successfully received by the subscriber; the reception failure means that the subscriber fails to receive; the processing failure represents the service processing failure of the event by the subscriber; processing success indicates that the subscriber is successful in processing the event.
When the event bus receives the event issued by the issuer, the event is stored in the event queue, and then the current event state of the event is recorded in the message table as an enqueue.
S130, the event bus acquires service data associated with the event from the message queue according to the event identification of the event issued by the issuer.
When an event is released, a releasing party sends service data associated with the event to a message queue, wherein the message queue comprises a service data queue corresponding to the stored event, and the releasing party sends the service data associated with the released event to the service data queue.
When the event bus processes the event, the service data associated with the event is read from the message queue.
And S140, the event bus sends the event and the service data corresponding to the event to a subscriber subscribing the event.
The event bus records the corresponding relation between the subscriber and the subscribed event (for example, the corresponding relation between the subscriber identifier and the event identifier), when the event bus processes a certain event, the event identifier of the event is obtained, the subscriber subscribing the event is determined according to the corresponding relation between the subscriber and the subscribed event, and the event bus distributes the event to the subscriber and simultaneously sends the service data associated with the event to the subscriber.
And S150, if the event bus does not receive the confirmation receiving message sent by the subscriber within the first preset duration after sending the event to the subscriber, updating the current event state of the event.
After the event bus sends the event to the subscriber, if the acknowledgement receiving message returned by the subscriber is not received within the first preset duration, the subscriber is determined to fail to receive. Meanwhile, the current event state in which the event is recorded is reception failure.
And S160, if the event bus sends the event to the subscriber and receives a confirmation receiving message returned by the subscriber within a first preset time period, the current event state of the event is updated.
After receiving the event and the associated service data sent by the event bus, the subscriber sends a confirmation receiving message to the event bus, wherein the confirmation receiving message is used for indicating that the subscriber has successfully received the event sent by the event bus.
After receiving the confirmation receiving message sent by the subscriber, the event bus updates the current event state corresponding to the event to be successful in receiving, for example, records the event identification and the successful in receiving of the event in the message table.
S170, after the subscriber successfully receives the subscribed event and service data, corresponding service processing is executed, and a response message is returned to the event bus.
After receiving the subscribed event, the subscriber carries out corresponding service processing on the event and returns corresponding response information to the event bus; if the service processing is successful, a successful response message is returned; and if the service processing fails, returning a failure response message.
Preferably, in the process of carrying out service processing on the received event by the subscriber, if abnormality occurs, retry is carried out according to a preset retry strategy, and the preset retry strategy can be set according to actual requirements. For example, the retry strategy herein may be the same as the retry strategy at the time of event issue.
S180, after receiving the response message returned by the subscriber and having successfully processed, the event bus updates the current event state of the event to be successfully processed.
For example, if the subscriber is successful in processing the event, the current event state for the event may be updated in the message table to be successful in processing.
And S190, after the event bus receives the response message of processing failure returned by the subscriber, updating the current event state of the event into processing failure.
If the subscriber fails to process the event, the event bus may update the event status of the event in the message table to a processing failure.
S1100, the event bus inquires the event state of the event and sends the event state to the publisher.
The event bus searches the event state of the event corresponding to the event identifier according to the event identifier;
in one embodiment of the application, the event state is recorded in the form of a message table, and the event bus only maintains one message table, namely the event states of events issued by a plurality of event issuers are recorded in the message table, and the event state corresponding to the event is searched from the message table according to the event identification. And the event bus sends the event state of the event to the publisher according to the event state obtained by the query so as to enable the publisher to execute corresponding operations.
S1110, the publisher executes operations corresponding to the event states.
For example, if the subscriber fails to process, the publisher rolls back the event; if the subscriber processes successfully, the publisher submits the event to the upper layer application.
Alternatively, for a scenario in which one event is subscribed to by only one subscriber, in other embodiments of the present application, the event operated by the publisher may be recorded in the same message table (e.g., a first message table), and the event processed by the subscriber may be recorded in another message table (e.g., a second message table);
for example, each publisher in the system corresponds to a first message table and each subscriber corresponds to a second message table. Or at least two publishers in the system correspond to a first message table, and at least two subscribers correspond to a second message table.
For a scenario in which one event is subscribed to by multiple subscribers, in other embodiments of the present application, a first message table is created for the publisher while a second message table is created for the subscribers.
For example, each publisher corresponds to a first message table, or at least two publishers in the system share a first message table; each subscriber corresponds to a second message table, or at least two subscribers in the system share a second message table.
In the application scenario of creating the first message table and the second message table, the event state still includes: enqueue, receive success, receive failure, process success. The update event state in the application scenario is the same as the update procedure described in the above embodiment, and will not be described here again.
Preferably, in order to reduce redundant data in the message table, the event bus may further clear event information corresponding to an ended event in the message table according to a preset policy, where the ended event includes an event that is successfully processed and an event that has failed more than a preset number of times.
The distributed transaction processing method provided by the embodiment adopts an event bus as a communication medium between a publisher and a subscriber, and the publisher publishes an event to the event bus and simultaneously sends service data associated with the event to a message queue; then, the event bus distributes the event to the corresponding subscriber, and decoupling of the transaction and the message queue of the service data is realized. Moreover, the event bus records event states after the event is operated by the publisher and subscriber, so that the consistency of event processing is ensured; the event bus determines the event status of the event and sends the event status to the publisher so that the publisher performs the corresponding operation. For example, if the service processing fails, the publisher performs operations such as event rollback, and if the service processing is successful, the publisher submits event processing to the upper layer application to complete. As can be seen from the above process, after the event bus mechanism is adopted, the message queues of the transaction and the service data are effectively decoupled, i.e. the micro service and the message queues are effectively decoupled; moreover, the event state is recorded and updated by the event bus, and therefore, the micro service becomes lighter, reducing development and maintenance costs of the micro service.
On the other hand, in order to more clearly and intuitively know the whole processing process of the event from the message tables, a plurality of message tables can be set, and event states after the publisher and the subscriber operate the event can be respectively recorded. For example, for publishers or subscribers, the event bus needs to create and maintain two message tables, respectively, which may be, for example, published and Received tables. Namely, a publisher Published table (which may be referred to as a first message table) and a publisher Received table (which may be referred to as a third message table) are created and maintained for publishers, while a subscriber Published table (which may be referred to as a fourth message table) and a subscriber Received table (which may be referred to as a second message table) are created and maintained for subscribers, all four tables being maintained and updated by an event bus.
The publisher Published table is used for recording the event state of the event Published by the publisher, and the publisher Received table is used for recording the event state of the event which is Received by the publisher and returned by the subscriber and is processed by the subscriber; the subscriber public table is used for recording the event state of the event Received by the subscriber after the subscriber processes the event, and the subscriber Received table is used for recording the event state of the event Received by the subscriber.
Under an application scenario that an event is subscribed by only one subscriber, at least an event identifier and a corresponding event state are included in the four message tables; in addition, the publisher and subscriber identifications associated with the events may be recorded in a message table or may be recorded in other locations besides the message table, such as, for example, the storage locations and storage forms specified by the event bus, as the application is not limited in this regard.
Under the application scenario that an event is subscribed by a plurality of subscribers, because the event bus is required to distinguish the event state of each subscriber after the event is operated, the subscriber identifier associated with the event state also needs to be embodied in the message table corresponding to the publisher; namely, the first message table and the third message table at least comprise event identifications, event states and associated subscriber identifications; in addition, the publisher identifier associated with the event may be recorded in a message table, or may be recorded in other locations besides a message table, such as a storage location and a storage form specified by the event bus, which is not limited by the present application.
According to the message table sharing situation, the following situations are included: each publisher in the system corresponds to a first message table and a third message table, and each subscriber corresponds to a second message table and a fourth message table; or at least two publishers correspond to a first message table and a third message table in the system, and at least two subscribers correspond to a second message table and a fourth message table.
For the second message table and the fourth message table, if at least two subscribers share one second message table and share one fourth message table, in this case, the subscriber identity associated with the event state needs to be represented in the message table of the subscribers; namely, the second message table and the fourth message table at least comprise: event identification, event status, and associated subscriber identification. Further, the publisher identification associated with the event may be recorded in the message table, or may be recorded in other locations besides the message table, such as a storage location and a storage form specified by the event bus;
for the second and fourth message tables, if a subscriber corresponds to a second and fourth message table, the subscriber identity associated with the event status need not be stored in the message table. Thus, the second and fourth message tables include at least event identifications and event states; in addition, subscriber identities associated with event states may be stored in corresponding message tables, or may be stored elsewhere than in the message tables, such as where an event bus specifies and in what form.
Referring to fig. 3, a flowchart of another distributed transaction processing method according to an embodiment of the present application is shown, where the embodiment is subscribed by a subscriber for an event, and each publisher corresponds to an application scenario of a first message table and a third message table, and each subscriber corresponds to a second message table and a fourth message table, and details related content of event status update in each message table.
As shown in fig. 3, the method may include the steps of:
s210, the publisher publishes the event and sends service data associated with the event to a message queue.
If the event release fails, the publisher reissues the event according to a preset retry strategy. In order to implement event retransmission, the publisher needs to persistence the information of the event while publishing the event, that is, the published event is stored locally for a period of time, so that the event is reissued when the event is failed to be published.
S220, after receiving the event issued by the issuer, the event bus records the event state of the event in the issuer Published table.
In this embodiment, the publisher Published table has a status name field for indicating the status of the current event.
For example, in the scenario of creating and maintaining four message tables, the state of an event may include: wait (the state represents waiting for processing), success (the state represents successful processing), failure (the state represents processing failure).
After the event bus detects the event issued by the issuer, the event is stored in the event queue. At the same time, the event identification of the event and the status of wait (i.e., wait for event bus processing) are recorded in the publisher Published table.
And S230, the event bus acquires service data associated with the event from the message queue according to the event identification of the event.
And after the event bus reads the event from the event queue, acquiring service data associated with the event from the message queue according to the event identification of the event.
S240, the event bus sends the event and the business data corresponding to the event to the subscriber.
After the event bus reads the event from the event queue, obtaining a subscriber subscribing the event according to the corresponding relation between the subscriber and the subscribed event; the event is then sent to subscribers that subscribe to the event. And simultaneously, sending the business data corresponding to the event to a subscriber subscribing the event.
S250, if the event bus does not receive the confirmation receiving message returned by the subscriber within the first preset duration after the event bus sends the event to the subscriber, the event state of the event in the publisher Published table is updated.
If the event bus does not receive the confirmation receiving message returned by the subscriber within the first preset time after the event bus distributes the event to the subscriber, the event bus determines that the subscriber fails to receive, and the event identification and the failed state are recorded in the publisher Published table.
After S250 is performed, S2100 is performed, where, for example, the event bus notifies the publisher of the event publication failure according to the failure state in the publisher Published table, and accordingly, the publisher rolls back the event, thereby ensuring the consistency of the event in the distributed transaction processing system.
And S260, if the event bus receives a confirmation receiving message returned by the subscriber within a first preset time after the event bus sends the event to the subscriber, updating the event state of the event in the subscriber Received list and the publisher Published list.
After receiving the acknowledgement receiving message returned by the subscriber, the event bus records the event identification and waiting state of the event in the subscriber Received table, and updates the event state of the event to waiting (i.e. waiting processing) in the publisher Published table.
S270, after the subscriber receives the subscribed event and the associated business data successfully, corresponding business processing is executed, and a response message is returned to the event bus.
If the subscriber succeeds in processing the event business, a response message of successful event processing is sent to an event bus; if the subscriber fails to process the business of the event, a response message of the event processing failure is sent to the event bus.
When the subscriber processes the event, the subscriber can retry according to a preset retry strategy. The retry strategy may be the same as the publisher retry strategy.
S280, the event bus updates event states of the Published tables and the Received tables corresponding to the subscriber and the publisher according to the response message returned by the subscriber.
If the subscriber successfully processes the event, recording the event state of the event as success in a subscriber Published table; and recording the event status of the event as successful in the publisher Received table. Meanwhile, the event state of the event in the subscriber Received table is updated to be successful, and the event state of the event in the publisher published table is updated to be successful.
If the subscriber fails to process, the event status of the event in the four message tables (i.e., the publisher Published table, the publisher Received table, and the subscriber Published table) is updated to fail.
S290, the event bus searches the event state of the event in each message table and sends the event state to the publisher.
The event bus scans the message tables according to a preset strategy (for example, scanning once at preset time intervals), determines the final event state of the event according to the event states in the four message tables, sends the event state to the publisher, and the publisher performs corresponding operations.
And S2100, the publisher executes corresponding operations according to the received event states.
For example, if the subscriber processing is successful (i.e., the event publication is successful), the publisher may submit the results of the event processing completion to the upper-level application; if the subscriber fails to process (i.e., the event publication fails), the publisher rolls back the event.
In addition, in a scenario where one event is subscribed by a plurality of subscribers, the update process of the event status in each message table is similar to the update process described above, and will not be described in detail herein.
Preferably, in order to reduce redundant data in the message table, the event bus may further clear event information corresponding to an ended event in the message table according to a preset policy, where the ended event includes an event that is successfully processed and an event that has failed more than a preset number of times.
In the publishing method provided by the embodiment, the event bus needs to maintain the message tables of the publisher and the subscriber, for example, the event bus updates the corresponding message tables according to the received response messages of the two parties. Then, the event state corresponding to the event in each message table is scanned, and the determined event state is sent to the publisher. The publisher performs a corresponding operation according to the event state of the event. The method adopts the event bus mechanism, effectively decouples the coupling between the micro service and the message queue, and the message table is operated by the event bus and is not coupled with the message queue, so that the micro service becomes lighter, and the development and maintenance cost of the micro service is reduced.
The application also provides a distributed transaction processing device embodiment corresponding to the distributed transaction processing method embodiment.
Referring to fig. 4, a block diagram of a distributed transaction processing device according to an embodiment of the present application is shown, where the device is applied to an event bus, and as shown in fig. 4, the device may include: a logging module 110, an acquisition module 120, a sending module 130, a status updating module 140, and a status determining module 150.
The recording module 110 is configured to receive an event sent by the publisher and record an event state of the event.
In one embodiment of the present application, the recording module 110 is configured to record the event identifier and the corresponding event status of the event in the message table.
In another embodiment of the present application, the recording module 110 is configured to record an event identifier corresponding to an event, an event status, and a subscriber identifier associated with the event in a message table.
In yet another embodiment of the present application, the recording module 110 is configured to record an event identifier and an event status corresponding to an event, and a publisher identifier and a subscriber identifier associated with the event in a message table.
In yet another embodiment of the present application, the recording module 110 is configured to record the event status of all events in the system, and the publisher identifier and subscriber identifier associated with each event in the message table.
The event identifier is a unique identifier for identifying an event, the publisher identifier is a unique identifier for identifying a party for publishing the event, and the subscriber identifier is a unique identifier for identifying a party for subscribing the event.
The acquiring module 120 is configured to acquire service data associated with the event from the message queue.
Wherein the service data is sent by the publisher to the message queue upon publication of the event.
And the sending module 130 is configured to send the event and the service data associated with the event to a target subscriber subscribed to the event.
And the state updating module 140 is configured to update an event state corresponding to the event according to a response message of the event returned by the target subscriber.
The state determining module 150 is configured to determine an event state corresponding to the event, and send the event state to a publisher corresponding to the event, so that the publisher performs an operation corresponding to the event state.
In another embodiment of the present application, the apparatus further includes a cleaning module 160 for scanning the event status of each recorded event and cleaning event information of the completed event in the record according to a preset policy.
The finished event comprises an event which is successfully processed and an event of which the failure times exceed the preset times.
The distributed transaction processing device provided by the embodiment adopts the event bus as a communication medium between the publisher and the subscriber, and distributes the event to the corresponding subscriber by the event bus, so that decoupling of the message queue of the transaction and the service data is realized, namely, the micro service and the message queue are effectively decoupled; and the event bus records the event states of the publishers and subscribers after operating the events, so that the consistency of event processing is ensured. Thus, the micro-services become lighter, reducing the development and maintenance costs of the micro-services.
The distributed transaction processing device comprises a processor and a memory, wherein the recording module, the acquisition module, the sending module, the state updating module, the state determining module, the cleaning module and the like are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor includes a kernel, and the kernel fetches the corresponding program unit from the memory. The kernel can be provided with one or more than one, and the asynchronous assurance of the event through the event bus is realized by adjusting the kernel parameters, so that the system development and maintenance cost is reduced.
The memory may include volatile memory, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flashRAM), among other forms in computer readable media, the memory including at least one memory chip.
The embodiment of the invention provides a storage medium, on which a program is stored, which when executed by a processor implements the distributed transaction processing method.
The embodiment of the invention provides a processor which is used for running a program, wherein the distributed transaction processing method is executed when the program runs.
The embodiment of the application provides equipment, wherein the equipment can be a server, a PC, a PAD, a mobile phone and the like. The device comprises a processor, a memory and a program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the following steps:
the event bus receives an event sent by a publisher and records the event state of the event;
the event bus acquires service data associated with the event from a message queue, and the service data is sent to the message queue by the publisher when the event is published;
the event bus sends the event and the business data associated with the event to a target subscriber subscribing the event;
the event bus updates an event state corresponding to the event according to the response message of the event returned by the target subscriber;
the event bus determines an event state corresponding to the event and sends the event state to a publisher corresponding to the event so that the publisher executes an operation corresponding to the event state.
In one possible implementation of the present application, recording an event state of the event includes:
the event bus records event identifications of the events and corresponding event states in a message table, wherein the event identifications are unique identifications for identifying the events.
In another possible implementation of the present application, recording an event state of the event includes:
the event bus records an event identifier, an event state and a subscriber identifier associated with the event corresponding to the event in a message table;
wherein the event identity is a unique identity identifying an event and the subscriber identity is a unique identity identifying a party to the subscribed event.
In yet another possible implementation of the present application, recording an event state of the event includes:
the event bus records an event identifier and an event state corresponding to the event, and a publisher identifier and a subscriber identifier associated with the event in a message table;
the event identifier is a unique identifier for identifying an event, the publisher identifier is a unique identifier for identifying a party for publishing the event, and the subscriber identifier is a unique identifier for subscribing the event party.
In yet another possible implementation manner of the present application, the message table is used to record the event status of all events in the system where the event bus is located, and the identifier of the publisher and the identifier of the subscriber associated with the event.
In another possible implementation manner of the present application, the method further includes: the event bus scans the event states of all the recorded events, and cleans up the event information of the ended events in the record according to a preset strategy, wherein the ended events comprise successfully processed events and events with failure times exceeding a preset number.
The application also provides a computer program product adapted to perform, when executed on a data processing device, a program initialized with the method steps of:
the event bus receives an event sent by a publisher and records the event state of the event;
the event bus acquires service data associated with the event from a message queue, and the service data is sent to the message queue by the publisher when the event is published;
the event bus sends the event and the business data associated with the event to a target subscriber subscribing the event;
the event bus updates an event state corresponding to the event according to the response message of the event returned by the target subscriber;
the event bus determines an event state corresponding to the event and sends the event state to a publisher corresponding to the event so that the publisher executes an operation corresponding to the event state.
In one possible implementation of the present application, recording an event state of the event includes:
the event bus records event identifications of the events and corresponding event states in a message table, wherein the event identifications are unique identifications for identifying the events.
In another possible implementation of the present application, recording an event state of the event includes:
the event bus records an event identifier, an event state and a subscriber identifier associated with the event corresponding to the event in a message table;
wherein the event identity is a unique identity identifying an event and the subscriber identity is a unique identity identifying a party to the subscribed event.
In yet another possible implementation of the present application, recording an event state of the event includes:
the event bus records an event identifier and an event state corresponding to the event, and a publisher identifier and a subscriber identifier associated with the event in a message table;
the event identifier is a unique identifier for identifying an event, the publisher identifier is a unique identifier for identifying a party for publishing the event, and the subscriber identifier is a unique identifier for subscribing the event party.
In yet another possible implementation manner of the present application, the message table is used to record the event status of all events in the system where the event bus is located, and the identifier of the publisher and the identifier of the subscriber associated with the event.
In another possible implementation manner of the present application, the method further includes: the event bus scans the event states of all the recorded events, and cleans up the event information of the ended events in the record according to a preset strategy, wherein the ended events comprise successfully processed events and events with failure times exceeding a preset number.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash memory (flashRAM). Memory is an example of a computer-readable medium.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. A method of distributed transaction processing, comprising:
the event bus receives an event sent by a publisher and records the event state of the event;
the event bus acquires service data associated with the event from a message queue, and the service data is sent to the message queue by the publisher when the event is published; the method comprises the steps that an interface for releasing an event is injected into a controller of a publisher, and the publisher releases the event through the interface;
The event bus sends the event and the business data associated with the event to a target subscriber subscribing the event;
the event bus updates an event state corresponding to the event according to the response message of the event returned by the target subscriber;
the event bus determines an event state corresponding to the event and sends the event state to a publisher corresponding to the event so that the publisher executes an operation corresponding to the event state.
2. The method of claim 1, wherein recording the event status of the event comprises:
the event bus records event identifications of the events and corresponding event states in a message table, wherein the event identifications are unique identifications for identifying the events.
3. The method of claim 1, wherein recording the event status of the event comprises:
the event bus records an event identifier, an event state and a subscriber identifier associated with the event corresponding to the event in a message table;
wherein the event identity is a unique identity identifying an event and the subscriber identity is a unique identity identifying a party to the subscribed event.
4. The method of claim 1, wherein recording the event status of the event comprises:
The event bus records an event identifier and an event state corresponding to the event, and a publisher identifier and a subscriber identifier associated with the event in a message table;
the event identifier is a unique identifier for identifying an event, the publisher identifier is a unique identifier for identifying a party for publishing the event, and the subscriber identifier is a unique identifier for subscribing the event party.
5. The method according to any of claims 2-4, wherein the message table is used for recording event status of all events in the system where the event bus is located, and the identification of the publisher and the identification of the subscriber to which an event is associated.
6. The method according to claim 1, wherein the method further comprises:
the event bus scans the event states of all the recorded events, and cleans up the event information of the ended events in the record according to a preset strategy, wherein the ended events comprise successfully processed events and events with failure times exceeding a preset number.
7. A distributed transaction processing apparatus for use in an event bus, the apparatus comprising:
the recording module is used for receiving the event sent by the publisher and recording the event state of the event;
The acquisition module is used for acquiring service data associated with the event from a message queue, and the service data is sent to the message queue by the publisher when the event is published; the method comprises the steps that an interface for releasing an event is injected into a controller of a publisher, and the publisher releases the event through the interface;
the sending module is used for sending the event and the business data associated with the event to a target subscriber subscribing the event;
the state updating module is used for updating the event state corresponding to the event according to the response message of the event returned by the target subscriber;
the state determining module is used for determining an event state corresponding to the event and sending the event state to a publisher corresponding to the event so that the publisher executes an operation corresponding to the event state.
8. The apparatus according to claim 7, wherein the recording module is configured to record an event identifier of the event and a corresponding event status in a message table, the event identifier being a unique identifier for identifying the event.
9. A storage medium having a program stored thereon, wherein the program, when executed by a processor, implements the distributed transaction method of any of claims 1 to 6.
10. A processor for running a program, characterized in that the program runs on performing the distributed transaction method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811159001.5A CN110968586B (en) | 2018-09-30 | 2018-09-30 | Distributed transaction processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811159001.5A CN110968586B (en) | 2018-09-30 | 2018-09-30 | Distributed transaction processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110968586A CN110968586A (en) | 2020-04-07 |
| CN110968586B true CN110968586B (en) | 2023-08-25 |
Family
ID=70029293
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811159001.5A Active CN110968586B (en) | 2018-09-30 | 2018-09-30 | Distributed transaction processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110968586B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113742034A (en) * | 2020-05-29 | 2021-12-03 | 北京沃东天骏信息技术有限公司 | Event processing method and device, computer readable storage medium and electronic equipment |
| CN111679892A (en) * | 2020-06-04 | 2020-09-18 | 中国建设银行股份有限公司 | Distributed transaction processing method, device, equipment and medium |
| CN111679922A (en) * | 2020-06-10 | 2020-09-18 | 中国银行股份有限公司 | Event-driven message publishing method, device and system |
| CN113961639A (en) * | 2020-06-22 | 2022-01-21 | 金篆信科有限责任公司 | Distributed transaction processing method, terminal and computer readable storage medium |
| CN112437130A (en) * | 2020-11-11 | 2021-03-02 | 上海电器科学研究所（集团）有限公司 | City underground pipe gallery business function system based on little service |
| CN112435004A (en) * | 2020-11-20 | 2021-03-02 | 金蝶软件（中国）有限公司 | Method and device for realizing service flow processing |
| CN112486478B (en) * | 2020-11-26 | 2023-08-11 | 上海万物新生环保科技集团有限公司 | Event processing method and device based on field driving |
| CN112527525B (en) * | 2020-12-11 | 2024-04-02 | 广州伊智信息科技有限公司 | Distributed event bus processing method, terminal and medium based on message queue |
| CN112581289A (en) * | 2020-12-14 | 2021-03-30 | 京东数字科技控股股份有限公司 | Data processing method, device, equipment and computer readable storage medium |
| CN113067831B (en) * | 2021-03-26 | 2022-11-04 | 中国工商银行股份有限公司 | Communication method, communication device and communication system |
| CN114116024B (en) * | 2021-09-29 | 2024-01-30 | 北京智芯微电子科技有限公司 | Peripheral drive processing method, device and system for embedded operating system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893395A (en) * | 2015-01-26 | 2016-08-24 | 阿里巴巴集团控股有限公司 | Message re-checking method and system of distributed transaction |
| CN107592351A (en) * | 2017-09-06 | 2018-01-16 | 北京锐安科技有限公司 | A kind of multi-user's distribution subscription method and system based on Redis |
| CN107657532A (en) * | 2016-07-25 | 2018-02-02 | 武汉票据交易中心有限公司 | The processing method and system of a kind of operation flow |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7571195B2 (en) * | 2004-05-28 | 2009-08-04 | Ebay Inc. | Publication of informational messages to software applications in a computing environment |
-
2018
- 2018-09-30 CN CN201811159001.5A patent/CN110968586B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893395A (en) * | 2015-01-26 | 2016-08-24 | 阿里巴巴集团控股有限公司 | Message re-checking method and system of distributed transaction |
| CN107657532A (en) * | 2016-07-25 | 2018-02-02 | 武汉票据交易中心有限公司 | The processing method and system of a kind of operation flow |
| CN107592351A (en) * | 2017-09-06 | 2018-01-16 | 北京锐安科技有限公司 | A kind of multi-user's distribution subscription method and system based on Redis |
Non-Patent Citations (1)
| Title |
|---|
| 实时数据库报警与事件关键技术实现;陆会明等;《电力自动化设备》;20090410(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110968586A (en) | 2020-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110968586B (en) | Distributed transaction processing method and device | |
| US11614867B2 (en) | Distributed storage system-based data processing method and storage device | |
| JP3851272B2 (en) | Stateful program entity workload management | |
| US10439937B2 (en) | Service addressing in distributed environment | |
| CN111143093B (en) | Asynchronous message distributed processing method, device, equipment and storage medium | |
| CN106997557B (en) | Order information acquisition method and device | |
| CN111784329B (en) | Service data processing method and device, storage medium and electronic device | |
| CN107040576B (en) | Information pushing method and device and communication system | |
| WO2008101756A1 (en) | Method and system for concurrent message processing | |
| CN106612330A (en) | System and method supporting distributed multi-file importing | |
| CN110381077A (en) | For the treating method and apparatus of digital certificate | |
| CN112822091A (en) | Message processing method and device | |
| CN107025257A (en) | A kind of transaction methods and device | |
| CN108241616B (en) | Message pushing method and device | |
| CN112995262B (en) | Distributed transaction submission method, system and computing equipment | |
| CN112949797B (en) | Information carrier management method, device, storage medium and server | |
| CN112860746B (en) | Cache reduction-based method, equipment and system | |
| CN114338584A (en) | Message withdrawing method and message transmission system | |
| US20180063242A1 (en) | Method and apparatus for operating infrastructure layer in cloud computing architecture | |
| US20220012673A1 (en) | Maintaining status information for customer-support agents across multiple channels | |
| CN108345431B (en) | Data reading method and device | |
| CN112596882A (en) | Method, device and system for scheduling delayed tasks | |
| CN115103322B (en) | Subscription merging method, device, electronic equipment, storage medium and product | |
| US20180121184A1 (en) | System and method for service matching of instant message software | |
| CN112583770B (en) | Data transmission method, device, storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |