CN113778948A - Message persistent storage method and device - Google Patents
Message persistent storage method and device Download PDFInfo
- Publication number
- CN113778948A CN113778948A CN202111020268.8A CN202111020268A CN113778948A CN 113778948 A CN113778948 A CN 113778948A CN 202111020268 A CN202111020268 A CN 202111020268A CN 113778948 A CN113778948 A CN 113778948A
- Authority
- CN
- China
- Prior art keywords
- message
- stored
- balanced binary
- binary tree
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000002085 persistent effect Effects 0.000 title claims abstract description 45
- 239000012634 fragment Substances 0.000 claims description 35
- 230000015654 memory Effects 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 230000002688 persistence Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method and a device for persistent storage of messages, wherein the method comprises the following steps: determining a balanced binary tree corresponding to the message to be stored according to the message ID of the message to be stored and the number of the balanced binary trees, wherein the number of the balanced binary trees is determined by the maximum depth of a message queue and the hash depth; and storing the message to be stored into the message queue, and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored. The invention adopts the balanced binary tree as the index of the message, improves the efficiency of searching the message and can effectively utilize disk resources.
Description
Technical Field
The invention relates to a message queue, in particular to a message persistent storage method and a message persistent storage device.
Background
The message queue (message middleware) realizes persistence through recording files, and ensures that messages written into the queue are not lost after the message middleware is down and restarted. In practical application, because the message is written into the file, different processing mechanisms can cause great differences in processing speed, hard disk space utilization rate and memory utilization rate.
The current persistent storage mode of the message is to write the message into a disk in sequence and use a sparse storage mode index. And sequentially writing the messages into a file, and indexing by using a sparse storage mode. When the message is read, the position interval where the message is stored is found first, the offset of the message in the file is found through the dichotomy, and the corresponding message is obtained.
In the existing message persistent storage mode, files are written in sequence, so that the files can be deleted and new file storage can be established unless all messages in the files are consumed. If only a few messages in a file are not consumed for a long time, a large amount of disk space is wasted. The index in the sparse storage mode also needs to be written into a file, and the space of a disk is occupied as well.
Therefore, the existing message persistent storage mode has the situation of wasting disk space, and needs to be improved.
Disclosure of Invention
The present invention provides a method and an apparatus for persistent storage of messages to solve at least one technical problem in the above background art.
In order to achieve the above object, according to one aspect of the present invention, there is provided a message persistent storage method including:
determining a balanced binary tree corresponding to the message to be stored according to the message ID of the message to be stored and the number of the balanced binary trees, wherein the number of the balanced binary trees is determined by the maximum depth of a message queue and the hash depth;
and storing the message to be stored into the message queue, and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored.
Optionally, the message persistent storage method further includes:
determining a balanced binary tree corresponding to the query target message according to the message ID of the query target message and the number of the balanced binary trees;
searching information corresponding to the query target message from a balanced binary tree corresponding to the query target message;
and searching the query target message from the message queue according to the searched information.
Optionally, the message persistent storage method further includes:
when the downtime is recovered, determining the number of balanced binary trees according to the maximum depth and the hash depth of the message queue;
generating a number of balanced binary trees of the determined number of balanced binary trees;
acquiring information corresponding to each message stored in the message queue, and determining a balanced binary tree corresponding to each message according to the message ID of each message;
and storing the information corresponding to each message into the balanced binary tree corresponding to the message.
Optionally, the storing the information of the message to be stored in the balanced binary tree corresponding to the message to be stored specifically includes:
determining a storage target node in a balanced binary tree according to the message ID of the message to be stored;
and storing the information of the message to be stored into the storage target node.
Optionally, the storing the message to be stored into the message queue specifically includes:
determining the number of fragments according to the size of the message to be stored and the size of the file block in the message queue;
if the number of the fragments is equal to 1, storing the message to be stored into a file block of the message queue;
and if the number of the fragments is more than 1, storing the fragments corresponding to the message to be stored into the file blocks of the message queue respectively.
Optionally, the message persistent storage method further includes:
and if the number of the fragments is more than 1, storing the fragment storage position information corresponding to the message to be stored into a node linked list of a balanced binary tree corresponding to the message to be stored.
In order to achieve the above object, according to another aspect of the present invention, there is provided a message persistent storage apparatus, comprising:
the first balanced binary tree determining module is used for determining a balanced binary tree corresponding to the message to be stored according to the message ID of the message to be stored and the number of the balanced binary trees, wherein the number of the balanced binary trees is determined by the maximum depth and the hash depth of a message queue;
and the information storage module is used for storing the message to be stored into the message queue and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored.
Optionally, the message persistent storage device further includes:
a second balanced binary tree determining module, configured to determine, according to the message ID of the query target message and the number of the balanced binary trees, a balanced binary tree corresponding to the query target message;
the information searching module is used for searching the information corresponding to the query target message from the balanced binary tree corresponding to the query target message;
and the message searching module is used for searching the query target message from the message queue according to the searched information.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the message persistent storage method when executing the computer program.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above message persistent storage method.
The invention has the beneficial effects that:
the invention stores the information of the message into the balanced binary tree corresponding to the message after the message is stored into the message queue by establishing the balanced binary tree as the index of the message, and does not need to write a file separately for the index of the message.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts. In the drawings:
FIG. 1 is a first flowchart of a message persistence method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a message persistence method according to an embodiment of the present invention;
FIG. 3 is a third flowchart of a message persistent storage method according to an embodiment of the present invention;
FIG. 4 is a first block diagram of a message persistent storage device according to an embodiment of the present invention;
FIG. 5 is a second block diagram of a message persistent storage device according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present invention and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 is a first flowchart of a message persistent storage method according to an embodiment of the present invention, corresponding to a process of storing a message into a message queue, as shown in fig. 1, in an embodiment of the present invention, the message persistent storage method includes step S101 and step S102.
Step S101, determining a balanced binary tree corresponding to the message to be stored according to the message ID of the message to be stored and the number of the balanced binary trees, wherein the number of the balanced binary trees is determined by the maximum depth and the hash depth of the message queue.
In an embodiment of the present invention, the maximum DEPTH D of the message queue is obtained through the configuration file of the message queue, and the number n of the balanced binary tree is obtained by rounding up the D/HASH _ DEPTH, where HASH _ DEPTH is a preset HASH DEPTH. The configuration file of the message queue is configured with queue-related attributes, including the maximum depth of the queue. And during initialization, the maximum depth D of the queue is obtained by reading the configuration file.
In one embodiment of the present invention, when storing messages, each message needs to be individually assigned a non-repeating message ID. And the message ID takes the last 6 digits m, and m% n is used for acquiring the balanced binary tree into which the information of the message needs to be stored, namely determining the corresponding balanced binary tree of the message in the n balanced binary trees. Since the last 6 bits of the message ID are +1 in order, the message can be evenly distributed into each balanced binary tree. The last 6 digits are incremented from 0.
In one embodiment of the invention, the balanced binary tree of the invention is a hash balanced binary tree.
Step S102, storing the message to be stored into the message queue, and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored.
In an embodiment of the present invention, the information specifically includes: message ID, storage location information of the message, fragmentation condition of the message and the like. In one embodiment of the present invention, the storage location information of the message is used to indicate in which file block the message is specifically stored in the message queue.
In an embodiment of the present invention, the storing the information of the message to be stored in the balanced binary tree corresponding to the message to be stored in this step specifically includes:
determining a storage target node in a balanced binary tree according to the message ID of the message to be stored;
and storing the information of the message to be stored into the storage target node.
In one embodiment of the invention, the number of nodes c of each binary tree is the current DEPTH d/n of the queue, and is at most HASH _ DEPTH-1 and at least HASH _ DEPTH/2. The time complexity of the slot position is obtained to be O (1), and the time complexity of the traversal balanced binary tree is O (log)2 c). The target can be found by calculating the slot position once, and the time complexity is in place O (1). The traversal balanced binary tree is equivalent to a dichotomy, and thus has a temporal complexity of O (log)2 Number of nodes in binary tree). O (1) is the lowest time complexity, and the target can be found by one calculation.
Therefore, the hash balanced binary tree can be divided into N balanced binary tree indexes according to the maximum depth of the queue, the total message number is assumed to be N, and the original time complexity is O (log)2 N)，Complexity at time is O (log)2 N/n) Time complexity is reduced and efficiency is improved. The size of the used disk can be determined, and disk resources are effectively utilized under the condition of ensuring the speed.
In an embodiment of the present invention, the storing the message to be stored in the message queue in step S102 specifically includes:
determining the number of fragments according to the size of the message to be stored and the size of the file block in the message queue;
if the number of the fragments is equal to 1, storing the message to be stored into a file block of the message queue;
and if the number of the fragments is more than 1, storing the fragments corresponding to the message to be stored into the file blocks of the message queue respectively.
In an embodiment of the present invention, if the number of the fragments is greater than 1, the present invention further stores the fragment storage location information corresponding to the message to be stored in the node linked list of the balanced binary tree corresponding to the message to be stored.
Fig. 2 is a second flowchart of a message persistent storage method according to an embodiment of the present invention, corresponding to the process of querying a message from a message queue, as shown in fig. 2, in an embodiment of the present invention, the message persistent storage method further includes step S201 and step S203.
Step S201, according to the message ID of the query target message and the number of the balanced binary trees, determining the balanced binary tree corresponding to the query target message.
In one embodiment of the present invention, the present invention takes the last 6 digits m of the message ID, and calculates the balanced binary tree number using m% n, i.e. determines the corresponding one of n balanced binary trees of the query target message.
Step S202, finding out the information corresponding to the query target message from the balanced binary tree corresponding to the query target message.
Step S203, searching the query target message from the message queue according to the searched information.
In the embodiment of the invention, if the number of fragments of the query target message is greater than 1, the invention also obtains the fragment storage position information corresponding to the query target message from the node linked list of the balanced binary tree corresponding to the query target message. And then inquiring each fragment according to the fragment storage position information, and further combining each fragment to obtain an inquiry target message.
Fig. 3 is a third flowchart of the message persistent storage method according to the embodiment of the present invention, which corresponds to a process of recovering a balanced binary tree after downtime, and as shown in fig. 3, in an embodiment of the present invention, the message persistent storage method further includes step S301 and step S304.
And step S301, when the downtime is recovered, determining the number of the balanced binary trees according to the maximum depth and the hash depth of the message queue.
In step S302, the determined number of balanced binary trees is generated.
In the embodiment of the present invention, this step specifically initializes the root node of each balanced binary tree.
Step S303, obtaining the information corresponding to each message stored in the message queue, and determining the balanced binary tree corresponding to each message according to the message ID of each message.
And step S304, storing the information corresponding to each message into the balanced binary tree corresponding to the message.
In a specific embodiment of the present invention, when storing a message into a message queue, the maximum depth of the queue is d, the size of a file block is s, and a message with the size of l is written. And acquiring the number of the balanced binary trees as n through the maximum depth of the queue, using id of the message, and acquiring the corresponding balanced binary tree as a balanced binary tree with the number of m by using id% n. According to the message size l and the file block size s. Rounding up by l/s is used to determine the number of message fragments as t. The first slice is now written to the file, while the message specific information is written to the balanced binary tree. And then circulating t-1 times, writing the fragment corresponding to the message into the file each time, and simultaneously adding the position of the fragment into the node linked list of the binary tree until the writing is finished.
In one embodiment of the invention, when reading a message from a message queue, the balanced binary tree is confirmed by the message id and the corresponding node is found. And acquiring the number of the message pieces and the file and the position of each piece in the file. And reading each piece in sequence and splicing the pieces until the complete message is read.
Example (c): the maximum queue DEPTH is 100000, HASH DEPTH is 10000, file block size is 4096 bytes, and message size is 5000 bytes written.
The process of storing messages into the message queue specifically comprises:
1. according to the maximum queue depth, acquiring the number of binary trees which is 100000/10000+ 1-11;
2. the message id is 20210722141400234315123456000234, and the serial number of the balanced binary tree corresponding to the calculated message is 234% 11-3;
3. according to the message size and the file block size, calculating the number of the message fragments to be 5000/4096-2 (rounding up);
4. writing the first slice into a file, wherein the recording positions are as follows: the 1 st file, the 10 th block, writes the message concrete information into the balanced binary tree numbered 3;
5. writing the second piece into a file, wherein the recording positions are as follows: file No. 1, block No. 23 (where the file location is found by recording free locations), writes the file location to a linked list of nodes in the binary tree.
The process of reading the message from the message queue specifically includes:
1. confirming that the message id is 20210722141400234315123456000234, and calculating that the number of the corresponding balanced binary tree is 234% 11-3;
2. searching for information with the id of 20210722141400234315123456000234 from the balanced binary tree with the number of 3;
3. acquiring the file position, and reading the message content from the first file and the 10 th block;
4. reading the message content from the first file and the 23 rd block, and splicing the message content to the tail of the last file;
5. a complete message is obtained.
As can be seen from the above examples, the present inventionAnd writing the message into the fragments, and reading the recovery index when the downtime is recovered. Thus the index does not have to write a file separately. The hash balanced binary tree can be divided into N balanced binary tree indexes according to the maximum depth of the queue, assuming that the total message number is N and the original time complexity is O (log)2 N) At a temporal complexity of O (log)2 N/n) Time complexity is reduced and efficiency is improved. The size of the used disk can be determined, and disk resources are effectively utilized under the condition of ensuring the speed.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
Based on the same inventive concept, an embodiment of the present invention further provides a message persistent storage apparatus, which can be used to implement the message persistent storage method described in the foregoing embodiment, as described in the following embodiment. Because the principle of the message persistent storage device for solving the problem is similar to the message persistent storage method, the embodiment of the message persistent storage device can be referred to as the embodiment of the message persistent storage method, and repeated details are omitted. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 4 is a first structural block diagram of a message persistent storage apparatus according to an embodiment of the present invention, and as shown in fig. 4, the message persistent storage apparatus according to the embodiment of the present invention includes:
the first balanced binary tree determining module 1 is configured to determine a balanced binary tree corresponding to a message to be stored according to a message ID of the message to be stored and the number of the balanced binary trees, where the number of the balanced binary trees is determined by a maximum depth of a message queue and a hash depth;
and the information storage module 2 is used for storing the message to be stored into the message queue and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored.
Fig. 5 is a second structural block diagram of the message persistent storage apparatus according to the embodiment of the present invention, and as shown in fig. 5, the message persistent storage apparatus according to the embodiment of the present invention further includes:
a second balanced binary tree determining module 3, configured to determine, according to the message ID of the query target message and the number of the balanced binary trees, a balanced binary tree corresponding to the query target message;
the information searching module 4 is configured to search the information corresponding to the query target message from the balanced binary tree corresponding to the query target message;
and the message searching module 5 is used for searching the query target message from the message queue according to the searched information.
In an embodiment of the present invention, the message persistent storage apparatus of the present invention further includes:
the balanced binary tree number determining module is used for determining the number of balanced binary trees according to the maximum depth and the hash depth of the message queue when the downtime is recovered;
a balanced binary tree generating module, configured to generate a number of balanced binary trees of the determined balanced binary tree;
a third balanced binary tree determining module, configured to acquire information corresponding to each message stored in the message queue, and determine a balanced binary tree corresponding to each message according to a message ID of each message;
and the message writing module is used for storing the information corresponding to each message into the balanced binary tree corresponding to the message.
In an embodiment of the present invention, the information storage module 2 specifically includes:
a storage target node determining unit, configured to determine a storage target node in the balanced binary tree according to the message ID of the message to be stored;
and the first storage unit is used for storing the information of the message to be stored into the storage target node.
In an embodiment of the present invention, the information storage module 2 further includes:
the distribution determining unit is used for determining the number of fragments according to the size of the message to be stored and the size of the file block in the message queue;
the second storage unit is used for storing the message to be stored into the file block of the message queue if the number of the fragments is equal to 1;
and the third storage unit is used for storing the fragments corresponding to the message to be stored into the file blocks of the message queue respectively if the number of the fragments is greater than 1.
In an embodiment of the present invention, the message persistent storage apparatus of the present invention further includes:
and the fragment storage information writing module is used for storing the fragment storage position information corresponding to the message to be stored into the node linked list of the balanced binary tree corresponding to the message to be stored if the number of the fragments is greater than 1.
To achieve the above object, according to another aspect of the present application, there is also provided a computer apparatus. As shown in fig. 6, the computer device comprises a memory, a processor, a communication interface and a communication bus, wherein a computer program that can be run on the processor is stored in the memory, and the steps of the method of the above embodiment are realized when the processor executes the computer program.
The processor may be a Central Processing Unit (CPU). The Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and units, such as the corresponding program units in the above-described method embodiments of the present invention. The processor executes various functional applications of the processor and the processing of the work data by executing the non-transitory software programs, instructions and modules stored in the memory, that is, the method in the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory and when executed by the processor perform the method of the above embodiments.
The specific details of the computer device may be understood by referring to the corresponding related descriptions and effects in the above embodiments, and are not described herein again.
In order to achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above message persistent storage method. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for persistent storage of messages, comprising:
determining a balanced binary tree corresponding to the message to be stored according to the message ID of the message to be stored and the number of the balanced binary trees, wherein the number of the balanced binary trees is determined by the maximum depth of a message queue and the hash depth;
and storing the message to be stored into the message queue, and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored.
2. The message persistent storage method of claim 1, further comprising:
determining a balanced binary tree corresponding to the query target message according to the message ID of the query target message and the number of the balanced binary trees;
searching information corresponding to the query target message from a balanced binary tree corresponding to the query target message;
and searching the query target message from the message queue according to the searched information.
3. The message persistent storage method of claim 1, further comprising:
when the downtime is recovered, determining the number of balanced binary trees according to the maximum depth and the hash depth of the message queue;
generating a number of balanced binary trees of the determined number of balanced binary trees;
acquiring information corresponding to each message stored in the message queue, and determining a balanced binary tree corresponding to each message according to the message ID of each message;
and storing the information corresponding to each message into the balanced binary tree corresponding to the message.
4. The method according to claim 1, wherein the storing the information of the message to be stored in the balanced binary tree corresponding to the message to be stored specifically includes:
determining a storage target node in a balanced binary tree according to the message ID of the message to be stored;
and storing the information of the message to be stored into the storage target node.
5. The method according to claim 1, wherein the storing the message to be stored in the message queue specifically includes:
determining the number of fragments according to the size of the message to be stored and the size of the file block in the message queue;
if the number of the fragments is equal to 1, storing the message to be stored into a file block of the message queue;
and if the number of the fragments is more than 1, storing the fragments corresponding to the message to be stored into the file blocks of the message queue respectively.
6. The message persistent storage method of claim 5, further comprising:
and if the number of the fragments is more than 1, storing the fragment storage position information corresponding to the message to be stored into a node linked list of a balanced binary tree corresponding to the message to be stored.
7. A message persistent storage device, comprising:
the first balanced binary tree determining module is used for determining a balanced binary tree corresponding to the message to be stored according to the message ID of the message to be stored and the number of the balanced binary trees, wherein the number of the balanced binary trees is determined by the maximum depth and the hash depth of a message queue;
and the information storage module is used for storing the message to be stored into the message queue and storing the information of the message to be stored into a balanced binary tree corresponding to the message to be stored.
8. The message persistent storage device of claim 7, further comprising:
a second balanced binary tree determining module, configured to determine, according to the message ID of the query target message and the number of the balanced binary trees, a balanced binary tree corresponding to the query target message;
the information searching module is used for searching the information corresponding to the query target message from the balanced binary tree corresponding to the query target message;
and the message searching module is used for searching the query target message from the message queue according to the searched information.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when executed in a computer processor, implements the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111020268.8A CN113778948A (en) | 2021-09-01 | 2021-09-01 | Message persistent storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111020268.8A CN113778948A (en) | 2021-09-01 | 2021-09-01 | Message persistent storage method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113778948A true CN113778948A (en) | 2021-12-10 |
Family
ID=78840632
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111020268.8A Pending CN113778948A (en) | 2021-09-01 | 2021-09-01 | Message persistent storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113778948A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934384A (en) * | 2023-01-16 | 2023-04-07 | 云账户技术(天津)有限公司 | Method, system, electronic device and readable storage medium for implementing message queue |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020095422A1 (en) * | 2001-01-17 | 2002-07-18 | Burrows Kevin W. | Method for creating a balanced binary tree |
| CN102521334A (en) * | 2011-12-07 | 2012-06-27 | 广东工业大学 | Data storage and query method based on classification characteristics and balanced binary tree |
| CN102801784A (en) * | 2012-07-03 | 2012-11-28 | 华为技术有限公司 | Distributed type data storing method and equipment |
| US20130173665A1 (en) * | 2012-01-03 | 2013-07-04 | Oracle International Corporation | System and method for efficient representation of dynamic ranges of numeric values |
| CN103970739A (en) * | 2013-01-24 | 2014-08-06 | 中兴通讯股份有限公司 | Storage information processing method and device |
| CN107835175A (en) * | 2017-11-09 | 2018-03-23 | 深圳市云盾科技有限公司 | Using the network connection tracing method of balanced binary tree algorithm |
| CN108416000A (en) * | 2018-02-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Method for exhibiting data, device, equipment and storage medium |
| CN110737536A (en) * | 2019-09-19 | 2020-01-31 | 亚信创新技术(南京)有限公司 | shared memory based message storage method and message reading method |
| CN111026765A (en) * | 2019-12-16 | 2020-04-17 | 武汉轻工大学 | Dynamic processing method, equipment, storage medium and device for strictly balanced binary tree |
| CN111061680A (en) * | 2018-10-15 | 2020-04-24 | 北京京东尚科信息技术有限公司 | Data retrieval method and device |
| CN112799972A (en) * | 2021-01-27 | 2021-05-14 | 成都佰维存储科技有限公司 | Implementation method and device of SSD mapping table, readable storage medium and electronic equipment |
-
2021
- 2021-09-01 CN CN202111020268.8A patent/CN113778948A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020095422A1 (en) * | 2001-01-17 | 2002-07-18 | Burrows Kevin W. | Method for creating a balanced binary tree |
| CN102521334A (en) * | 2011-12-07 | 2012-06-27 | 广东工业大学 | Data storage and query method based on classification characteristics and balanced binary tree |
| US20130173665A1 (en) * | 2012-01-03 | 2013-07-04 | Oracle International Corporation | System and method for efficient representation of dynamic ranges of numeric values |
| CN102801784A (en) * | 2012-07-03 | 2012-11-28 | 华为技术有限公司 | Distributed type data storing method and equipment |
| CN103970739A (en) * | 2013-01-24 | 2014-08-06 | 中兴通讯股份有限公司 | Storage information processing method and device |
| CN107835175A (en) * | 2017-11-09 | 2018-03-23 | 深圳市云盾科技有限公司 | Using the network connection tracing method of balanced binary tree algorithm |
| CN108416000A (en) * | 2018-02-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Method for exhibiting data, device, equipment and storage medium |
| CN111061680A (en) * | 2018-10-15 | 2020-04-24 | 北京京东尚科信息技术有限公司 | Data retrieval method and device |
| CN110737536A (en) * | 2019-09-19 | 2020-01-31 | 亚信创新技术(南京)有限公司 | shared memory based message storage method and message reading method |
| CN111026765A (en) * | 2019-12-16 | 2020-04-17 | 武汉轻工大学 | Dynamic processing method, equipment, storage medium and device for strictly balanced binary tree |
| CN112799972A (en) * | 2021-01-27 | 2021-05-14 | 成都佰维存储科技有限公司 | Implementation method and device of SSD mapping table, readable storage medium and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| ZHENLIANG ZHANG: "Error Probability Bounds for Balanced Binary Relay Trees", 《IEEE TRANSACTIONS ON INFORMATION THEORY》, vol. 6, no. 58, pages 3548 * |
| 李毅飞 等: "一种基于平衡二叉树的CDP数据备份及重构方法", 《数据通信》, no. 2, pages 13 - 17 * |
| 王亮只 等: "平衡二叉树在穷举检测器生成算法中的应用", 《计算机安全》, no. 1, pages 16 - 18 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934384A (en) * | 2023-01-16 | 2023-04-07 | 云账户技术(天津)有限公司 | Method, system, electronic device and readable storage medium for implementing message queue |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3678346B1 (en) | Blockchain smart contract verification method and apparatus, and storage medium | |
| CN110018983B (en) | Metadata query method and device | |
| CN110149803B (en) | Data storage method, system and terminal equipment | |
| US11232073B2 (en) | Method and apparatus for file compaction in key-value store system | |
| CN110147204B (en) | Metadata disk-dropping method, device and system and computer-readable storage medium | |
| CN109582213B (en) | Data reconstruction method and device and data storage system | |
| US20170031948A1 (en) | File synchronization method, server, and terminal | |
| CN108228799B (en) | Object index information storage method and device | |
| CN112579602B (en) | Multi-version data storage method, device, computer equipment and storage medium | |
| WO2018006721A1 (en) | Method and apparatus for storing log file | |
| CN107798063B (en) | Snapshot processing method and snapshot processing device | |
| CN112785408A (en) | Account checking method and device based on Hash | |
| CN113010526A (en) | Storage method and device based on object storage service | |
| CN104035822A (en) | Low-cost efficient internal storage redundancy removing method and system | |
| CN113778948A (en) | Message persistent storage method and device | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN115409507A (en) | Block processing method, block processing device, computer equipment and storage medium | |
| US9189382B2 (en) | Noncontiguous representation of an array | |
| CN111309471A (en) | Data processing method and device and distributed system | |
| US11372832B1 (en) | Efficient hashing of data objects | |
| CN111737347B (en) | Method and device for sequentially segmenting data on Spark platform | |
| CN110990640B (en) | Data determination method, device, equipment and computer readable storage medium | |
| CN115964084A (en) | Data interaction method, electronic equipment and storage medium | |
| CN111177162A (en) | Data synchronization method and device | |
| CN116149573B (en) | Method, system, equipment and medium for processing queue by RAID card cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |