CN113282426A - Message processing system and method and computing device - Google Patents
Message processing system and method and computing device Download PDFInfo
- Publication number
- CN113282426A CN113282426A CN202110458765.XA CN202110458765A CN113282426A CN 113282426 A CN113282426 A CN 113282426A CN 202110458765 A CN202110458765 A CN 202110458765A CN 113282426 A CN113282426 A CN 113282426A
- Authority
- CN
- China
- Prior art keywords
- message
- message body
- configuration
- queue
- data storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 35
- 230000002159 abnormal effect Effects 0.000 claims abstract description 60
- 238000013500 data storage Methods 0.000 claims abstract description 58
- 238000003672 processing method Methods 0.000 claims abstract description 20
- 238000004806 packaging method and process Methods 0.000 claims abstract description 5
- 238000011084 recovery Methods 0.000 claims description 44
- 230000008569 process Effects 0.000 claims description 10
- 238000004891 communication Methods 0.000 abstract description 34
- 230000005856 abnormality Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005538 encapsulation Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000007723 transport mechanism Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1415—Saving, restoring, recovering or retrying at system level
- G06F11/1443—Transmit or communication errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Computer And Data Communications (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a message processing system, comprising: a message queue; a data storage device; the producer application is respectively connected with the message queue and the data storage device and comprises a producer processing device, the producer application is suitable for generating an initial message body, packaging the initial message body based on the configuration information through the producer processing device, generating a configuration message body, delivering the configuration message body to the message queue, and storing the configuration message body to the data storage device when the delivery of the configuration message body is abnormal; the consumer application is respectively connected with the message queue and the data storage device and comprises a consumer processing device which is suitable for acquiring the configuration message body from the message queue and unsealing the configuration message body so as to acquire the corresponding initial message body. The invention also discloses a corresponding message processing method and a corresponding computing device. According to the technical scheme of the invention, the problems of message delivery and acquisition when the communication is abnormal based on the message queue can be solved.
Description
Technical Field
The present invention relates to the field of computer and internet technologies, and in particular, to a message processing system, a message processing method, and a computing device.
Background
In the present day when internet micro-services are popular, message middleware is generally utilized to realize the decoupling and asynchronization of cross-service association logic.
In the prior art, the execution logic of the scheme implemented based on the message middleware is generally: the producer application executes self logic to generate a message body and delivers the message body to a message queue; the consumer application monitors the message queue and executes the consumer's own logic after obtaining the message from the message queue.
According to the existing technical solutions, the following abnormal situations often occur: communication between the producer application and the message queue is abnormal, so that the message is undelivered; abnormal communication between the consumer application and the message queue causes the consumer application to unsuccessfully acquire the message and a large amount of backlog of the message in the message queue; consumer application internal exceptions cause the execution logic of the message to produce unexpected results, requiring re-execution.
Therefore, a message processing system and method are needed to solve the above problems.
Disclosure of Invention
To this end, the present invention provides a message processing system, method and computing device to solve or at least alleviate the above-presented problems.
According to an aspect of the present invention, there is provided a message processing system including: a message queue; a data storage device; the producer application is respectively connected with the message queue and the data storage device and comprises a producer processing device, the producer application is suitable for generating an initial message body, packaging the initial message body based on configuration information through the producer processing device, generating a configuration message body, delivering the configuration message body to the message queue, and storing the configuration message body to the data storage device when the delivery of the configuration message body is abnormal; and the consumer application is respectively connected with the message queue and the data storage device and comprises a consumer processing device, and the consumer processing device is suitable for acquiring the configuration message body from the message queue and unsealing the configuration message body so as to acquire the corresponding initial message body and process the initial message body.
Optionally, in the message processing system according to the present invention, the configuration information includes producer application information, target queue information, and exception status information.
Optionally, in the message processing system according to the invention, the consumer processing device is further adapted to: when the initial message body is processed abnormally, the abnormal state information of the message body is updated, and the message body and the corresponding abnormal state information are stored in the data storage device.
Optionally, in the message processing system according to the invention, the consumer processing device is further adapted to: updating the delivery state in the abnormal state information to be completed, and accumulating the compensation times by 1; and if the compensation times reach the threshold value after accumulating 1, setting the message as a dead message.
Optionally, in the message processing system according to the present invention, further comprising: the system comprises a data storage device, a message queue and a downgrade service application, wherein the downgrade service application is respectively connected with the message queue and the data storage device and comprises a downgrade queue, and the downgrade service application is suitable for acquiring a configuration message body which exceeds a preset time length from the message queue and delivering the configuration message body to the downgrade queue; and is adapted to update the configuration information of the configuration message body and store the updated configuration message body in the data storage device.
Optionally, in the message processing system according to the present invention, further comprising: and the message recovery device is respectively connected with the message queue and the data storage device, is suitable for acquiring the message body to be redelivered from the data storage device at regular time, and is suitable for delivering the configured message body to the corresponding target message queue based on the target queue information of the message body.
Optionally, in the message processing system according to the present invention, the message recovery means is further adapted to: and when the delivery state in the abnormal state information corresponding to the message body is determined to be to-be-delivered, to-be-consumed or to-be-completed and the compensation times are less than the threshold value, determining the message body to be delivered again.
Optionally, in the message processing system according to the present invention, further comprising: the message recovery platform is connected with the message recovery device and is suitable for acquiring an abnormal message list from the message recovery device, wherein the abnormal message list comprises message identifications of one or more message bodies to be redelivered; and is adapted to request the message recovery device to re-deliver the message body corresponding to one or more message identifications in the exception message list.
Optionally, in the message processing system according to the present invention, the producer processing means is further adapted to: generating a message identification corresponding to the initial message body; and performing encapsulation processing on the initial message body based on the message identification and the configuration information to generate a configuration message body.
According to an aspect of the present invention, there is provided a message processing method, executed in a message processing system, comprising the steps of: generating an initial message body; packaging the initial message body based on configuration information to generate a configuration message body, and delivering the configuration message body to the message queue; when the delivery of the configuration message body is abnormal, the configuration message body is stored in a data storage device; and acquiring a configuration message body from the message queue, and performing unpacking processing on the configuration message body to acquire a corresponding initial message body and process the initial message body.
Optionally, in the message processing method according to the present invention, the configuration information includes producer application information, target queue information, and abnormal state information, and the method further includes the steps of: when the initial message body is processed abnormally, the abnormal state information in the configuration information of the message body is updated, and the message body and the corresponding abnormal state information are stored in the data storage device.
Optionally, in the message processing method according to the present invention, further comprising the steps of: acquiring a configuration message body exceeding a preset time length from a message queue, and delivering the configuration message body to the downgrade queue; and updating the configuration information of the configuration message body, and storing the updated configuration message body in the data storage device.
Optionally, in the message processing method according to the present invention, further comprising the steps of: acquiring a message body to be delivered again from the data storage device at regular time; and delivering the configured message body to a corresponding target message queue based on the target queue information of the message body.
According to an aspect of the invention, there is provided a computing device comprising: at least one processor; a memory storing program instructions configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the message processing method as described above.
According to an aspect of the present invention, there is provided a readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform a message processing method as described above.
According to the technical scheme of the invention, the message processing system is provided, the configuration information is further packaged for the message body to form the configuration message body which can be persisted, and the problems of message delivery and acquisition when the communication is abnormal based on the message queue can be solved. Specifically, when the producer application is in communication abnormality with the message queue to cause message body delivery failure, the encapsulated configuration message body can be written into the data storage device so as to be delivered again through the message recovery device; and when the consumer application is in communication abnormality with the message queue, and the message body in the message queue is not acquired due to timeout, the message body which is not acquired due to timeout is delivered to the degraded queue through the degraded service application and is stored in the data storage device, so that the problem of message backlog in the message queue can be avoided, and the message recovery device can be used for re-delivering the abnormal message body in the data storage device.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which are indicative of various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to be within the scope of the claimed subject matter. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description read in conjunction with the accompanying drawings. Throughout this disclosure, like reference numerals generally refer to like parts or elements.
FIG. 1 shows a schematic diagram of a message processing system 100 according to one embodiment of the invention;
FIG. 2 shows a schematic diagram of a computing device 200, according to one embodiment of the invention; and
fig. 3 shows a flow diagram of a message processing method 300 according to one embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
FIG. 1 shows a schematic diagram of a message processing system 100 according to one embodiment of the invention.
As shown in fig. 1, the message processing system 100 includes a message queue 150, one or more producer applications 110 coupled to the message queue 150, one or more consumer applications 120 coupled to the message queue 150, and a data storage device 140. Wherein one or more of the producer application 110, the consumer application 120, respectively, are also communicatively coupled to the data storage 140.
It should be noted that the producer application 110 and the consumer application 120, respectively, in the present invention may be implemented as application programs of different functions residing in the computing device, and the decoupling is implemented based on message queue communication. Here, the present invention does not limit the specific type of application.
In particular, the producer application 110 is an application that can post messages into the message queue 150, and the consumer application 120 is an application that can retrieve messages from the message queue 150. In particular, the producer application 110 may encapsulate the message to be delivered as a message body for delivery into the corresponding message queue. One or more message bodies delivered by the producer application 110 may be included in the message queue and may be sent to consumer applications 120 subscribed to the message queue according to the delivery order of the message bodies. It is to be appreciated that the producer application 110 is communicatively coupled to the consumer application 120 based on the message queue 150 and that asynchronous communication is enabled based on the message queue 150.
According to one embodiment of the invention, the system 100 includes a message queue server, which may include one or more message queues 150, each message queue 150 may be connected to a corresponding one or more producer applications 110, one or more consumer applications 120, to receive message bodies posted by the corresponding producer application 110 and to send the message bodies in the message queue to the corresponding consumer application 120. Here, each producer application 110 may post a message body to a particular message queue, and each message queue may be subscribed to and retrieve a message body in the message queue by one or more consumer applications 120.
According to an embodiment of the present invention, the producer application 110 includes a producer processing means 115, and the producer application 110 may encapsulate the message to be delivered to generate an initial message body and further encapsulate the initial message body by calling the producer processing means 115. Specifically, the producer processing means 115 performs further encapsulation processing on the initial message body based on the configuration information by acquiring the configuration information corresponding to the initial message body to generate the configuration message body. It should be understood that the configuration message body is a message body formed by encapsulating corresponding configuration information in the initial message body.
In one embodiment, the configuration information may include producer application information (e.g., including a producer application identifier) corresponding to the message body, target queue information for delivery (e.g., including a target queue identifier), communication parameters agreed upon by the producer application and the consumer application, current abnormal state information of the message body, and the like. Here, the abnormal state information includes, for example, the current delivery state and the compensation state of the message body. Specifically, the delivery status of the message body includes statuses of to-deliver, to-consume, to-complete, completed, and the like. The compensation state of the message body includes whether to compensate and the current compensation times. It should be appreciated that the initial configuration message body generated by the producer processing device 115 is encapsulated for the initial message body based on the configuration information, wherein the compensation status in the configuration information is no and the number of compensations is 0.
In one embodiment, when the compensation times of the message body exceed the threshold value, the message body is set as a dead letter, and the message body is not subjected to redelivery processing.
According to an embodiment of the present invention, the producer processing device 115 may post the configuration message body into the corresponding target message queue 150 based on the target queue information in the configuration information after generating the configuration message body based on the initial message body and the corresponding configuration information.
In one embodiment, the producer processing device 115 also generates a message identification UUID corresponding to the initial message body in advance before performing the encapsulation process on the initial message body. Further, the initial message body is encapsulated based on the message identifier and the configuration information, specifically, the message identifier and the configuration information are added to the head of the initial message body, so as to generate the configuration message body.
In addition, when the producer processing device 115 delivers the configuration message body exception and fails to deliver the configuration message body to the target message queue, the producer processing device 115 updates the configuration information of the configuration message body, for example, updates the delivery status in the exception status information in the configuration information to be delivered, and stores the configuration message body after updating the configuration information and the exception status information thereof in the data storage device 140 in association as the exception status message body, so as to deliver the message body in the exception status again in the following.
According to an embodiment of the present invention, the consumer application 120 includes a consumer processing apparatus 125, and the consumer processing apparatus 125 is connected to the message queue 150, and may obtain the configuration message body from the message queue 150, and perform decapsulation processing on the configuration message body, so as to split the configuration message body into the corresponding initial message body and the configuration information, and obtain the initial message body corresponding to the configuration message body, so as to perform corresponding processing on the initial message body.
In one embodiment, if an exception occurs in processing the initial message body after the consumer processing device 125 unpacks the configuration message body, the configuration information for the configuration message body is updated, such as updating the consumer application information and exception status information, and the updated configuration message body is stored to the data storage device 140.
In particular, the consumer processing device 125 includes a message receiving device coupled to the message queue 150 and an exception handling device coupled to the data storage device 140. The consumer processing device 125 obtains the configuration message body from the message queue 150 through the message receiving device, and performs decapsulation processing on the configuration message body so as to obtain an initial message body corresponding to the configuration message body and corresponding configuration information. The consumer application 120 may then process the initial message body based on the application's own processing logic.
If the consumer application 120 processes the initial message body abnormally, the abnormal processing device in the consumer processing device 125 updates the configuration information of the message body, for example, adds the consumer application information to the configuration information corresponding to the message body, and updates the delivery status in the abnormal status information to be completed. Subsequently, the exception handling apparatus stores the message body and its exception status information in association to the data storage apparatus 140 as an exception status message body.
Here, the exception handling apparatus in the consumer handling apparatus 125 updates the delivery status in the exception status information to "yes" and adds 1 to the number of offsets (number of retries) in addition to updating the delivery status in the exception status information to be completed when updating the exception status information in the message body configuration information. In addition, if the number of compensations reaches a threshold after 1 is accumulated, the message is treated as a dead message.
In one embodiment, the system 100 also includes a degraded services application 130, and the degraded services application 130 may be a services program residing in the computing device. The destage service application 130 is coupled to each message queue 150 and to the data storage device 140. The destage services application 130 includes a destage queue 135, the destage queue 135 being connected to each message queue 150.
The destage service application 130 may monitor the status of consumption of the message bodies in each message queue 150 and if it is monitored that there is a configuration message body in the message queue 150 that has not been consumed for more than a predetermined length of time, retrieve the configuration message body from the message queue 150 and post the configuration message body to the destage queue 135. The destage services application 130 then updates the configuration information for the configuration message body, e.g., updates the delivery status of the exception status information for the message body to be consumed, and stores the updated configuration message body to the data storage 140. In this way, when communication between the consumer application 120 and the message queue 150 is abnormal, and the message body in the message queue 150 is not acquired due to timeout, the message body which is not acquired due to timeout is delivered to the degraded queue by the degraded service application 130 and stored in the data storage device, so that the problem of message backlog in the message queue 150 can be avoided.
Here, the destaging service application 130 updates the delivery status in the abnormal status information to yes and adds 1 to the number of compensation times (number of retries) in addition to updating the delivery status in the abnormal status information to be consumed when updating the configuration information of the configuration message body. In addition, if the number of compensations reaches a threshold after 1 is accumulated, the message is treated as a dead message.
In one embodiment, the system 100 further includes a message recovery device 160, the message recovery device 160 being coupled to each of the message queues 150 and to the data storage device 140. The message recovery device 160 may periodically retrieve the message body to be redelivered from the data storage device 140 and is adapted to deliver the message body to the corresponding target message queue based on the target queue information of the message body.
For example, in one embodiment, the message recovery device 160 includes a timer, and the message body in the abnormal state stored in the data storage device 140 is periodically queried by the timer to obtain the message body in the abnormal state, and whether the message body is a message body to be redelivered is determined by the abnormal state information corresponding to the message body. Here, the message body to be redelivered includes a message body whose delivery status in the abnormal status information is to be delivered, to be consumed, or to be completed, and the number of times of compensation is less than the threshold value. That is, when the message recovery apparatus 160 determines that the delivery status in the abnormal status information corresponding to the message body is to be delivered, consumed, or completed, and the number of times of compensation is less than the threshold value, it determines that the message body is a message body to be delivered again. And re-delivering the message body to the corresponding target message queue according to the target queue information in the configuration information by acquiring the configuration information corresponding to the message body, thereby realizing the re-delivery of the abnormal message body.
In one embodiment, the system 100 further includes a message recovery platform 170, and the message recovery platform 170 is connected to the message recovery device 160 and can obtain the exception message list from the message recovery device 160 based on the message query interface. The exception message list comprises message identifications of one or more message bodies to be redelivered and corresponding exception state information. Also, message recovery platform 170 may send one or more message identifications in the exception message list to message recovery device 160 based on the message delivery interface to request message recovery device 160 to redeliver the corresponding message body for the one or more message identifications in the exception message list.
It should be appreciated that message recovery platform 170 is a client platform provided to a user (e.g., a technician), and message recovery platform 170 may be implemented, for example, as a web page or an application page, and the technician may request to retrieve an exception message list from message recovery device 160 at message recovery platform 170, to view exception status information for one or more message bodies based on the exception message list, and to manually instruct message recovery device 160 to redeliver one or more of the message bodies therein.
According to the message processing system 100 of the present invention, the configuration information is further encapsulated for the message body to form a configuration message body which can be persisted, so that the problem of message delivery and acquisition when the communication is abnormal based on the message queue can be solved. Specifically, when the producer application is in communication abnormality with the message queue to cause message body delivery failure, the encapsulated configuration message body can be written into the data storage device so as to be delivered again through the message recovery device; and when the consumer application is in communication abnormality with the message queue, and the message body in the message queue is not acquired due to timeout, the message body which is not acquired due to timeout is delivered to the degraded queue through the degraded service application and is stored in the data storage device, so that the problem of message backlog in the message queue can be avoided, and the message recovery device can be used for re-delivering the abnormal message body in the data storage device. In addition, according to the message processing system of the invention, the recovery of the message based on an automatic or manual mode can be realized.
In one embodiment, the producer application 110, the consumer application 120, the message queue 150, the demotion service application 130, the data store 140, the message recovery device 160, and the message recovery platform 170, respectively, in the message processing system 100 may reside in respective computing devices.
In one embodiment, the message processing system 100 may reside in a computing device, and the message processing system 100 is adapted to perform the message processing method 300 of the present invention, such that the message processing method 300 of the present invention may be performed in the computing device.
FIG. 2 shows a block diagram of a computing device 200, according to one embodiment of the invention.
As shown in FIG. 2, in a basic configuration 202, a computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between the processor 204 and the system memory 206.
Depending on the desired configuration, the processor 204 may be any type of processing, including but not limited to: a microprocessor (μ P), a microcontroller (μ C), a Digital Signal Processor (DSP), or any combination thereof. The processor 204 may include one or more levels of cache, such as a level one cache 210 and a level two cache 212, a processor core 214, and registers 216. Example processor cores 214 may include Arithmetic Logic Units (ALUs), Floating Point Units (FPUs), digital signal processing cores (DSP cores), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations the memory controller 218 may be an internal part of the processor 204.
Depending on the desired configuration, system memory 206 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 106 may include an operating system 220, one or more applications 222, and program data 224. The application 222 is actually a plurality of program instructions that direct the processor 204 to perform corresponding operations. In some embodiments, application 222 may be arranged to cause processor 204 to operate with program data 224 on an operating system.
Computing device 200 may also include a storage interface bus 234. The storage interface bus 234 enables communication from the storage devices 232 (e.g., removable storage 236 and non-removable storage 238) to the basic configuration 202 via the bus/interface controller 230. At least a portion of the operating system 220, applications 222, and data 224 may be stored on removable storage 236 and/or non-removable storage 238, and loaded into system memory 206 via storage interface bus 234 and executed by the one or more processors 204 when the computing device 200 is powered on or the applications 222 are to be executed.
Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via the bus/interface controller 230. The example output device 242 includes a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. Example peripheral interfaces 244 can include a serial interface controller 254 and a parallel interface controller 256, which can be configured to facilitate communications with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 258. An example communication device 246 may include a network controller 260, which may be arranged to facilitate communications with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
A network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media, such as carrier waves or other transport mechanisms, in a modulated data signal. A "modulated data signal" may be a signal that has one or more of its data set or its changes made in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or private-wired network, and various wireless media such as acoustic, Radio Frequency (RF), microwave, Infrared (IR), or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
In a computing device 200 according to the present invention, the application 222 includes a plurality of program instructions that perform the message processing method 300, which may instruct the processor 204 to perform the message processing method 300 of the present invention such that the computing device 200 encapsulates and processes the message body by performing the message processing method 300 of the present invention.
Fig. 3 shows a flow diagram of a message processing method 300 according to one embodiment of the invention. The method 300 is suitable for execution in the message processing system 100 described above.
As shown in fig. 3, the method 300 begins at step S310.
In step S310, an initial message body is generated.
Subsequently, in step S320, the initial message body is encapsulated based on the configuration information, a configuration message body is generated, and the configuration message body is delivered to the message queue. Here, by acquiring configuration information corresponding to the initial message body, the initial message body is further subjected to encapsulation processing based on the configuration information to generate a configuration message body. It should be understood that the configuration message body is a message body formed by encapsulating corresponding configuration information in the initial message body.
In one embodiment, the configuration information may include producer application information (e.g., including a producer application identifier) corresponding to the message body, target queue information for delivery (e.g., including a target queue identifier), communication parameters agreed upon by the producer application and the consumer application, current abnormal state information of the message body, and the like. Here, the abnormal state information includes, for example, the current delivery state and the compensation state of the message body. Specifically, the delivery status of the message body includes statuses of to-deliver, to-consume, to-complete, completed, and the like. The compensation state of the message body includes whether to compensate and the current compensation times. It should be appreciated that the initial configuration message body generated by the producer processing device 115 is encapsulated for the initial message body based on the configuration information, wherein the compensation status in the configuration information is no and the number of compensations is 0.
In step S330, when the delivery of the configuration message body is abnormal, the configuration message body is stored in the data storage device. Specifically, when the configuration message body is abnormally delivered and is not successfully delivered to the target message queue, the configuration message body is updated, for example, the delivery state in the abnormal state information in the configuration information is updated to be delivered, and the configuration message body after the configuration information is updated and the abnormal state information thereof are stored in the data storage device 140 in an associated manner as the abnormal state message body, so that the message body in the abnormal state is delivered again in the following.
It should be noted that steps S310 to S330 are performed by the producer application 110 in the system 100, and the specific execution logic thereof is as described above for the producer application 110.
In step S340, a configuration message body is obtained from the message queue, and the configuration message body is unpacked, so as to split the configuration message body into a corresponding initial message body and configuration information, and obtain the initial message body corresponding to the configuration message body, so as to perform corresponding processing on the initial message body. Here, step S340 is performed by the consumer application 120 in the system 100.
According to one embodiment, when the initial message body is processed abnormally, the consumer application 120 updates the configuration information of the message body, for example, adds the consumer application information to the configuration information corresponding to the message body, and updates the delivery status in the abnormal status information to be completed, and additionally, updates the compensation status to "yes", and increments the number of compensation (number of retries) by 1. Subsequently, the message body and its abnormal state information are stored in the data storage 140 in association as an abnormal state message body. Here, if the number of compensations reaches the threshold value after the number of compensations is added up to 1, the message is treated as a dead credit.
It should be noted that step S340 is performed by the consumer application 120 in the system 100. The specific execution logic in step S340 is as described above with respect to the consumer application 120.
According to one embodiment, in the method 300 of the present invention, the consumption status of the message body in each message queue 150 is also monitored by the destage service application 130, and if it is monitored that there is a configuration message body in the message queue 150 that has not been consumed for more than a predetermined period of time, the configuration message body is retrieved from the message queue 150 and posted to the destage queue 135. The destage services application 130 then updates the configuration information for the configuration message body, e.g., updates the delivery status of the exception status information for the message body to be consumed, and stores the updated configuration message body to the data storage 140.
Here, the destaging service application 130 updates the delivery status in the abnormal status information to yes and adds 1 to the number of compensation times (number of retries) in addition to updating the delivery status in the abnormal status information to be consumed when updating the configuration information of the configuration message body. In addition, if the number of compensations reaches a threshold after 1 is accumulated, the message is treated as a dead message.
According to one embodiment, in the method 300 of the present invention, the message body to be redelivered is also periodically retrieved from the data storage device by the message recovery device 160, and the message body is delivered to the corresponding target message queue based on the target queue information of the message body.
According to one embodiment, before a message body to be redelivered is acquired, whether the message body is the message body to be redelivered is determined through abnormal state information corresponding to the message body. The message body to be redelivered comprises the message body of which the delivery state in the abnormal state information is to be delivered, consumed or completed and the compensation times are less than the threshold value. That is, when the message recovery apparatus 160 determines that the delivery status in the abnormal status information corresponding to the message body is to be delivered, consumed, or completed, and the number of times of compensation is less than the threshold value, it determines that the message body is a message body to be delivered again. And re-delivering the message body to the corresponding target message queue according to the target queue information in the configuration information by acquiring the configuration information corresponding to the message body.
According to one embodiment, a technician may retrieve the list of exception messages from message recovery device 160 based on a message query interface at message recovery platform 170. The exception message list comprises message identifications of one or more message bodies to be redelivered and corresponding exception state information. And, may send one or more message identifications in the exception message list to message recovery device 160 based on the message delivery interface to request message recovery device 160 to redeliver the corresponding message body of the one or more message identifications in the exception message list. In this way, message retrieval device 160 may be instructed to redeliver one or more of the message bodies therein by manual operation.
It can be seen that according to the message processing method 300 of the present invention, a persistent configuration message body is formed by further encapsulating configuration information for the message body, and the problem of message delivery and acquisition when communication is abnormal based on a message queue can be solved. Specifically, when the producer application is in communication abnormality with the message queue to cause message body delivery failure, the encapsulated configuration message body can be written into the data storage device so as to be delivered again through the message recovery device; and when the consumer application is in communication abnormality with the message queue, and the message body in the message queue is not acquired due to timeout, the message body which is not acquired due to timeout is delivered to the degraded queue through the degraded service application and is stored in the data storage device, so that the problem of message backlog in the message queue can be avoided, and the message recovery device can be used for re-delivering the abnormal message body in the data storage device.
A8, the system of any one of A1-A7, further comprising: the message recovery platform is connected with the message recovery device and is suitable for acquiring an abnormal message list from the message recovery device, wherein the abnormal message list comprises message identifications of one or more message bodies to be redelivered; and is adapted to request the message recovery device to re-deliver the message body corresponding to one or more message identifications in the exception message list.
A9, the system as in any of A1-A8, wherein the producer processing device is further adapted to: generating a message identification corresponding to the initial message body; and performing encapsulation processing on the initial message body based on the message identification and the configuration information to generate a configuration message body.
B11, the method of B10, wherein the configuration information includes producer application information, target queue information, exception status information, the method further comprising the steps of: when the initial message body is processed abnormally, the abnormal state information in the configuration information of the message body is updated, and the message body and the corresponding abnormal state information are stored in the data storage device.
B12, the method according to B10 or B11, further comprising the steps of: acquiring a configuration message body exceeding a preset time length from a message queue, and delivering the configuration message body to the downgrade queue; and updating the configuration information of the configuration message body, and storing the updated configuration message body in the data storage device.
B13, the method according to any one of B10-B12, further comprising the steps of: acquiring a message body to be delivered again from the data storage device at regular time; and delivering the configured message body to a corresponding target message queue based on the target queue information of the message body.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U.S. disks, floppy disks, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the mobile terminal generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to perform the message processing method of the present invention according to instructions in the program code stored in the memory.
By way of example, and not limitation, readable media may comprise readable storage media and communication media. Readable storage media store information such as computer readable instructions, data structures, program modules or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with examples of this invention. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into multiple sub-modules.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
Furthermore, some of the described embodiments are described herein as a method or combination of method elements that can be performed by a processor of a computer system or by other means of performing the described functions. A processor having the necessary instructions for carrying out the method or method elements thus forms a means for carrying out the method or method elements. Further, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is used to implement the functions performed by the elements for the purpose of carrying out the invention.
As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The present invention has been disclosed in an illustrative rather than a restrictive sense with respect to the scope of the invention, as defined in the appended claims.
Claims (10)
1. A message processing system, comprising:
a message queue;
a data storage device;
the producer application is respectively connected with the message queue and the data storage device and comprises a producer processing device, the producer application is suitable for generating an initial message body, packaging the initial message body based on configuration information through the producer processing device, generating a configuration message body, delivering the configuration message body to the message queue, and storing the configuration message body to the data storage device when the delivery of the configuration message body is abnormal; and
the consumer application is respectively connected with the message queue and the data storage device and comprises a consumer processing device, and the consumer processing device is suitable for acquiring the configuration message body from the message queue and unsealing the configuration message body so as to acquire the corresponding initial message body and process the initial message body.
2. The system of claim 1, wherein the configuration information includes producer application information, target queue information, and exception status information.
3. The system of claim 2, wherein the consumer processing device is further adapted to:
when the initial message body is processed abnormally, the abnormal state information of the message body is updated, and the message body and the corresponding abnormal state information are stored in the data storage device.
4. The system of claim 3, wherein the consumer processing device is further adapted to:
updating the delivery state in the abnormal state information to be completed, and accumulating the compensation times by 1;
and if the compensation times reach the threshold value after accumulating 1, setting the message as a dead message.
5. The system of any of claims 1-4, further comprising:
the system comprises a data storage device, a message queue and a downgrade service application, wherein the downgrade service application is respectively connected with the message queue and the data storage device and comprises a downgrade queue, and the downgrade service application is suitable for acquiring a configuration message body which exceeds a preset time length from the message queue and delivering the configuration message body to the downgrade queue; and is adapted to update the configuration information of the configuration message body and store the updated configuration message body in the data storage device.
6. The system of any of claims 1-5, further comprising:
and the message recovery device is respectively connected with the message queue and the data storage device, is suitable for acquiring the message body to be redelivered from the data storage device at regular time, and is suitable for delivering the configured message body to the corresponding target message queue based on the target queue information of the message body.
7. The system of claim 6, wherein the message recovery device is further adapted to:
and when the delivery state in the abnormal state information corresponding to the message body is determined to be to-be-delivered, to-be-consumed or to-be-completed and the compensation times are less than the threshold value, determining the message body to be delivered again.
8. A message processing method, executed in a message processing system, comprising the steps of:
generating an initial message body;
packaging the initial message body based on configuration information to generate a configuration message body, and delivering the configuration message body to the message queue;
when the delivery of the configuration message body is abnormal, the configuration message body is stored in a data storage device; and
and acquiring a configuration message body from the message queue, and performing unpacking processing on the configuration message body to acquire a corresponding initial message body and process the initial message body.
9. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the message processing method of any of claims 1-7.
10. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the message processing method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110458765.XA CN113282426B (en) | 2021-04-27 | 2021-04-27 | Message processing system, method and computing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110458765.XA CN113282426B (en) | 2021-04-27 | 2021-04-27 | Message processing system, method and computing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113282426A true CN113282426A (en) | 2021-08-20 |
| CN113282426B CN113282426B (en) | 2024-05-31 |
Family
ID=77277462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110458765.XA Active CN113282426B (en) | 2021-04-27 | 2021-04-27 | Message processing system, method and computing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113282426B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115103017A (en) * | 2022-06-13 | 2022-09-23 | 珠海格力电器股份有限公司 | Message middleware processing system and method |
| WO2023040330A1 (en) * | 2021-09-14 | 2023-03-23 | 超聚变数字技术有限公司 | Data processing method, device, and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108196961A (en) * | 2017-12-28 | 2018-06-22 | 广东蜂助手网络技术股份有限公司 | A kind of asynchronous message processing method, terminal, system and storage medium |
| US20190324828A1 (en) * | 2018-04-18 | 2019-10-24 | Open Text GXS ULC | Producer-Side Prioritization of Message Processing |
| CN111782414A (en) * | 2020-05-12 | 2020-10-16 | 北京皮尔布莱尼软件有限公司 | Delay message processing method and system |
| CN112104519A (en) * | 2020-08-06 | 2020-12-18 | 北京健康之家科技有限公司 | Delayed message delivery method and device, storage medium and computer equipment |
| CN112437001A (en) * | 2020-11-16 | 2021-03-02 | 深圳壹账通智能科技有限公司 | Method and device for guaranteeing reliable delivery and consumption of messages |
| CN112667414A (en) * | 2020-12-23 | 2021-04-16 | 平安普惠企业管理有限公司 | Message queue-based message consumption method and device, computer equipment and medium |
-
2021
- 2021-04-27 CN CN202110458765.XA patent/CN113282426B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108196961A (en) * | 2017-12-28 | 2018-06-22 | 广东蜂助手网络技术股份有限公司 | A kind of asynchronous message processing method, terminal, system and storage medium |
| US20190324828A1 (en) * | 2018-04-18 | 2019-10-24 | Open Text GXS ULC | Producer-Side Prioritization of Message Processing |
| CN111782414A (en) * | 2020-05-12 | 2020-10-16 | 北京皮尔布莱尼软件有限公司 | Delay message processing method and system |
| CN112104519A (en) * | 2020-08-06 | 2020-12-18 | 北京健康之家科技有限公司 | Delayed message delivery method and device, storage medium and computer equipment |
| CN112437001A (en) * | 2020-11-16 | 2021-03-02 | 深圳壹账通智能科技有限公司 | Method and device for guaranteeing reliable delivery and consumption of messages |
| CN112667414A (en) * | 2020-12-23 | 2021-04-16 | 平安普惠企业管理有限公司 | Message queue-based message consumption method and device, computer equipment and medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023040330A1 (en) * | 2021-09-14 | 2023-03-23 | 超聚变数字技术有限公司 | Data processing method, device, and system |
| CN115103017A (en) * | 2022-06-13 | 2022-09-23 | 珠海格力电器股份有限公司 | Message middleware processing system and method |
| CN115103017B (en) * | 2022-06-13 | 2023-12-08 | 珠海格力电器股份有限公司 | Message middleware processing system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113282426B (en) | 2024-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111782414B (en) | Delay message processing method and system | |
| CN113282426A (en) | Message processing system and method and computing device | |
| CN108196961B (en) | Asynchronous message processing method, terminal, system and storage medium | |
| CN108055343B (en) | Data synchronization method and device for computer room | |
| US9843418B2 (en) | Fault tolerant retry subsystem and method | |
| CN108491301B (en) | Electronic device, abnormality early warning method based on redis and storage medium | |
| CN111835467B (en) | Message sending method, device, computer equipment and storage medium | |
| CN111090532B (en) | Application service calling method, device, electronic equipment and computer storage medium | |
| CN111371898B (en) | Message monitoring method, device, equipment and storage medium | |
| CN113079083B (en) | Message processing method, client, server and instant communication system | |
| CN108900627B (en) | Network request method, terminal device and storage medium | |
| CN104461404A (en) | Metadata storage method, device and system | |
| CN114064074A (en) | Upgrade task management and control method and device, electronic equipment and storage medium | |
| US20160127300A1 (en) | Dynamic granular messaging persistence | |
| CN107317742B (en) | Message processing method and system, computer readable storage medium | |
| CN114124881B (en) | Message pushing method based on priority and related device | |
| CN111526181A (en) | Data request processing method and system and computing device | |
| US20160294926A1 (en) | Using a single work item to send multiple messages | |
| CN113010379A (en) | Electronic equipment monitoring system | |
| CN112817992A (en) | Method, device, electronic equipment and readable storage medium for executing change task | |
| US20220253300A1 (en) | Software update device, server, software update system, and software update method | |
| US20100229024A1 (en) | Message producer with message type validation | |
| US20170192795A1 (en) | Universal Computing Element (UCE) and the Mode of Computing with the Use of UCE | |
| CN115964108A (en) | Cloud desktop destroying method and device, electronic equipment and storage medium | |
| CN111552581B (en) | Event delivery method, device, system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |