CN109460438B - Message data storage method, device, computer equipment and storage medium - Google Patents
Message data storage method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109460438B CN109460438B CN201811122763.8A CN201811122763A CN109460438B CN 109460438 B CN109460438 B CN 109460438B CN 201811122763 A CN201811122763 A CN 201811122763A CN 109460438 B CN109460438 B CN 109460438B
- Authority
- CN
- China
- Prior art keywords
- database
- message data
- sub
- time
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000013500 data storage Methods 0.000 title claims abstract description 25
- 238000004590 computer program Methods 0.000 claims description 20
- 230000001360 synchronised effect Effects 0.000 claims description 18
- 230000007774 longterm Effects 0.000 abstract description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000004140 cleaning Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Abstract
The application relates to a message data storage method, device, computer equipment and storage medium based on big data. The method comprises the following steps: acquiring message data generated by a terminal; storing the message data in a first database; synchronizing the message data stored in the first database to the second database at intervals of a preset first time; acquiring time information corresponding to each message data stored in a second database every other preset second time; when the time information of the message data exceeds the preset time, the message data with the time information exceeding the preset time in the message data is backed up to the file database. The method can completely acquire the historical data, whether the historical data is long-term historical message data or short-term historical message data, thereby ensuring long-term storage of the message data and facilitating inquiry of the message data at different times.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for storing message data, a computer device, and a storage medium.
Background
With the development of computer technology, more and more instant messaging applications are available, and the instant messaging applications generally have a message storage function, i.e. history messages can be queried and recorded. In the conventional technology, when an instant messaging application stores message data, a client storage mode is adopted, and after the client is unloaded and deleted, historical message data of the application cannot be obtained. To address this problem, there are applications that support cloud message data storage. However, this storage mode can only acquire the stored data in a short period, but the historical data relatively long in time cannot be acquired, so that the complete historical message data cannot be acquired.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a message data storage method, apparatus, computer device, and storage medium capable of storing message data in full.
A message data storage method, the method comprising:
acquiring message data generated by a terminal;
storing the message data in a first database;
synchronizing the message data stored in the first database to a second database at intervals of a preset first time;
Acquiring time information corresponding to each message data stored in a second database every other preset second time;
when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data into a file database.
In one embodiment, the first database includes at least a first sub-database and a second sub-database; the storing the message data in a first database includes: when the first sub-database is successfully connected, storing the message data into the first sub-database; and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
In one embodiment, the storing the message data to the first sub-database includes: storing the message data to a master node of the first sub-database; synchronizing, by a primary node of the first sub-database, the stored message data to a secondary node of the first sub-database; synchronizing the message data stored in the first database to a second database at preset first time intervals, including: synchronizing the message data stored in the secondary node of the first sub-database to a second database at preset first time intervals.
In one embodiment, after the synchronizing the message data stored in the first database into the second database at every preset first time, the method further includes: acquiring time information corresponding to the message data in the first database; and deleting the message data of which the time information exceeds the preset expiration time from the first database.
In one embodiment, after the message data with the time information exceeding the preset time in the message data is backed up to the file database, the method further includes: and deleting the message data of which the time information exceeds the preset time when the message data of which the time information exceeds the preset time is detected to be successfully backed up to the file database.
In one embodiment, the method further comprises: acquiring a message query request, wherein the message query request carries query time information; acquiring message data corresponding to the inquiry time information from a corresponding database according to the inquiry time information; when the query time information exceeds the query time, acquiring message data corresponding to the query time from the file database; and when the query time information does not exceed the query time, acquiring message data corresponding to the query time from the second database.
A message data storage device, the device comprising:
the data acquisition module is used for acquiring message data generated by the terminal;
the first storage module is used for storing the message data into a first database;
the second storage module is used for synchronizing the message data stored in the first database to a second database at intervals of preset first time;
the third storage module acquires time information corresponding to each message data stored in the second database at intervals of preset second time; when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data into a file database.
A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:
acquiring message data generated by a terminal;
storing the message data in a first database;
synchronizing the message data stored in the first database to a second database at intervals of a preset first time;
acquiring time information corresponding to each message data stored in a second database every other preset second time;
When detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data into a file database.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring message data generated by a terminal;
storing the message data in a first database;
synchronizing the message data stored in the first database to a second database at intervals of a preset first time;
acquiring time information corresponding to each message data stored in a second database every other preset second time;
when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data into a file database.
According to the message data storage method, the device, the computer equipment and the storage medium, the acquired terminal data are firstly stored in the first database, the data stored in the first database are synchronized to the second database periodically at intervals of preset first time, the message data stored in the second database are backed up to the file database periodically at intervals of preset second time, and when the message data with time information exceeding the preset time in the second database are stored in the file database, so that when the history data are required to be acquired, the history data can be completely acquired no matter the history message data in a long term or the history message data in a short term, when the message data which do not exceed the preset time in a short term are required to be acquired, the information data which do not exceed the preset time in a long term are acquired from the second database, and when the message data which do not exceed the preset time in a long term are required to be acquired from the file database, the long-term storage of the message data is ensured, and the message data with different times are convenient to be inquired.
Drawings
FIG. 1 is an application scenario diagram of a message data storage method in one embodiment;
FIG. 2 is a flow diagram of a message data storage method in one embodiment;
FIG. 3 is a flow chart illustrating the steps of storing message data in a first sub-database according to one embodiment;
FIG. 4 is a flowchart illustrating steps further included in a message data storage method according to one embodiment;
FIG. 5 is a flow chart of a method of message data storage in another embodiment;
FIG. 6 is a timing diagram of a message data storage method in one embodiment;
FIG. 7 is a block diagram of the structure of a message data store in one embodiment;
FIG. 8 is a block diagram of a message data store in another embodiment;
fig. 9 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The message data storage method provided by the application can be applied to an application environment shown in fig. 1. Wherein the terminal 110 communicates with the server 120 via a network. A preset application may be installed on the terminal 110, and the preset application may generate message data during use. The server 120 may include a first database, a second database, and a file database, and after the preset application installed on the terminal 110 generates message data, the message data may be stored in the first database, the message data stored in the first database is synchronized to the second database at preset first time intervals, and the message data stored in the second database is backed up to the file database at preset second time intervals. The terminal 110 may be, but not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, and the server 120 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, a message data storage method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
step 202, obtaining message data generated by a terminal.
The terminal can be provided with a preset instant messaging application, message data can be continuously generated in the using process of the instant messaging application, and the message data generated by the preset instant messaging application arranged on the terminal can be obtained. The message data may be text-only, expression message, picture, etc.
Step 204, storing the message data in a first database.
Step 206, synchronizing the message data stored in the first database to the second database at intervals of a preset first time.
Step 208, obtaining time information corresponding to each message data stored in the second database at intervals of preset second time.
Step 210, when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data to the file database.
The first database may be a Redis library, which is a high performance key-value database. After the message data generated by the terminal is acquired, the message data may be stored in the first database entirely. The preset first time and the preset second time are preset times, and the preset first time and the preset second time can be the same or different. The second database may be an Oracle database (a relational database management system), and may synchronize the message data stored in the first database to the second database at intervals of a preset first time, and back up the message data stored in the second database to the file at intervals of a preset second time, so as to ensure full-scale storage of the message data.
When the message data stored in the second database is backed up to the file database at intervals of preset second time, the message data can be backed up according to the time information of the message data. Each message data has corresponding time information, the time information refers to the time of generating the message data, and after the time information corresponding to each message data stored in the second database is acquired, the message data corresponding to the time information exceeding the preset time can be backed up and stored in the file database. Assuming that the preset time is set to 30 days, message data whose time information exceeds 30 days may be backed up to the file database, that is, message data generated before 30 days is backed up to the file database, while message data whose time information does not exceed 30 days is still stored in the second database. Thus, the message data exceeding the preset time can be obtained from the file database, and the message data not exceeding the preset time can be obtained from the second database, so that the complete storage of all the message data is ensured.
In one embodiment, after the message data with the time information exceeding the preset time in the message data is backed up to the file database, the method further comprises: when the message data with the time information exceeding the preset time is detected to be successfully backed up to the file database, deleting the message data with the time information exceeding the preset time.
After the message data exceeding the preset time in the second database is backed up and stored in the file database, the message data in the second database can be sorted. After the message data exceeding the preset time is successfully backed up to the file database, the part of the message data exceeding the preset time in the second database can be deleted, so that the redundancy of the data in the second database is avoided.
According to the message data storage method, the acquired terminal data are firstly stored in the first database, the data stored in the first database are synchronized to the second database at intervals of preset first time, the message data stored in the second database are backed up to the file database at intervals of preset second time, and when the message data with time information exceeding the preset time in the second database are stored in the file database, so that when the history data are required to be acquired, the history data in a long term or a short term can be completely acquired, when the short-term message data which do not exceed the preset time are required to be acquired, the information data which do not exceed the preset time can be acquired from the second database, and when the long-term message data exceeding the preset time are required to be acquired, the information data in the file database are acquired, so that the long-term storage of the message data is ensured, and the information data in different times are convenient to inquire.
In one embodiment, the first database includes at least a first sub-database and a second sub-database. Storing the message data in a first database, comprising: when the first sub-database is successfully connected, storing the message data into the first sub-database; and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
The first database may include a plurality of sub-databases, that is, the first database includes at least a first sub-database and a second sub-database. Each sub-database may store message data. When message data generated by a preset application installed on a terminal is stored in a first database, detecting which sub-database in the first database is successfully connected, and if the connection with the first sub-database in the first database is detected to be successful, storing the message data generated on the terminal in the first sub-database. If the connection with the second sub-database in the first database is detected to be successful, the message data generated on the terminal can be stored in the second sub-database. When the message data is stored in one of the sub-databases, detecting whether the connection with other sub-databases in the first sub-database is successful or not at preset time intervals, and if the connection is successful, storing the message data in the other sub-databases which are successfully connected in turn, so that the message data is stored in the first database. That is, when a certain sub-database fails to be connected, that is, when a breakdown exists, the message data is not stored in the sub-database, so that the failure storage of the message data is avoided, and even if the breakdown exists in the certain sub-database, the message data can be normally stored in other sub-databases, so that the safe and effective storage of the message data is ensured.
In one embodiment, as shown in FIG. 3, storing message data to a first sub-database includes:
step 302, storing the message data to a master node of a first sub-database.
Step 304, synchronizing the stored message data to the secondary node of the first sub-database by the primary node of the first sub-database.
Synchronizing the message data stored in the first database to the second database at preset first time intervals, including: and synchronizing the message data of the secondary node stored in the first sub-database to the second database at preset first time intervals.
Each sub-database in the first database may include a plurality of nodes, including at least one secondary node and a primary node. When the message data generated on the terminal is stored in a first sub-database in the first database, the message data is firstly stored in a main node in the first sub-database, and the main node of the first sub-database synchronizes the stored message data to a sub-node in the first sub-database. Thus, message data on the terminal is stored on both the primary node and the secondary node of the first sub-database. The master node is responsible for writing in the message data, namely storing the message data on the terminal, and the slave node is responsible for reading the message data, namely synchronizing the message data through the slave node when transferring and storing the message data, namely reading the data through the slave node. The read-write separation mode can also avoid the data bottleneck caused by a large number of read-write operations of the same node.
In one embodiment, after synchronizing the message data stored in the first database into the second database at every preset first time, further comprising: acquiring time information corresponding to message data in a first database; message data whose time information exceeds a preset expiration time is periodically deleted from the first database.
In order to avoid that a large amount of useless message data is stored in the first database after the message data in the first database is synchronized into the second database, and reduce the storage efficiency, the message data with the time exceeding the preset expiration time in the first database can be deleted periodically after the message data stored in the first database is synchronized into the second database. Each message data has corresponding time information, and the time generated by the message data can be obtained according to the time information. Then the message data in the first database may be cleared of message data whose time information exceeds a preset expiration time. The preset expiration time can be set by itself. For example, the preset expiration time is set to 1 hour, then message data generated 1 hour before in the first database is deleted periodically. The deletion of the message data generated 1 hour ago in the first database may be performed every ten minutes, or the deletion of the message data from the first database may be performed upon detecting that the message data is generated 1 hour ago.
Because the message data is stored in the first database at first, after the message data in the first database is synchronized to the second database, the expired data in the first database can be cleaned, and the condition that the data redundancy in the first database is caused because more useless expiration time is stored in the first database is avoided, so that the memory in the first database can be released in a regular cleaning mode.
In one embodiment, as shown in fig. 4, the method further includes:
step 402, a message query request is obtained, where the message query request carries query time information.
Step 404, obtaining message data corresponding to the inquiry time information from the corresponding database according to the inquiry time information.
And step 406, when the query time information exceeds the query time, acquiring message data corresponding to the query time from the file database.
And step 408, when the query time information does not exceed the query time, acquiring message data corresponding to the non-exceeded query time from the second database.
The message query request can be initiated by the user through the terminal, and the message query request carries query time information, namely, when the user triggers the message query request for message data, the user can determine the time information of the message data to be queried. For example, if the user needs to query the message data 10 days ago, the time limit of the message data to be queried can be checked on the corresponding data query interface. The message data before searching for 10 days can be limited by adding time limitation in the search command, namely the message data corresponding to the query time information can be obtained from the corresponding database according to the query time information. When the message data is generated on the terminal, the message data generated on the terminal is firstly stored in the first database, the message data in the first database is synchronized to the second database at intervals of preset first time, and the message data stored in the second database is backed up to the file database at intervals of preset second time. And when the message data stored in the second database is backed up to the file database at intervals of a preset second time, acquiring time information corresponding to each message data stored in the second database, and backing up the message data with the detected time information exceeding the preset time to the file database. Therefore, when the query time information exceeds the query time length, the message data corresponding to the query time length is acquired from the file database, and when the query time information does not exceed the query time length, the message data corresponding to the non-exceeded query time length is acquired from the second database.
Assuming that the preset time is 30 days, the message data of which the time information is detected to exceed 30 days is backed up to the file database, that is, the message data of which the time information is not detected to exceed 30 days is stored in the second database, and the message data of which the time information is detected to exceed 30 days is stored in the file database. When the message data is queried, if the query time information carried in the message query request is 30 days ago, the corresponding message data is acquired from the file database, and when the query time information carried in the message query request is within 30 days, the corresponding message data is acquired from the second database.
In one embodiment, as shown in fig. 5, a message data storage method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
step 502, message data generated by a terminal is acquired.
And step 504, when the first sub-database is successfully connected, storing the message data to the main node of the first sub-database.
Step 506, synchronizing the stored message data to the secondary node of the first sub-database by the primary node of the first sub-database.
Step 508, synchronizing the message data stored in the secondary node of the first sub-database to the second database at intervals of a preset first time.
The message data generated on the terminal may originate from a preset application installed on the terminal, which may generate corresponding message data during use. After the message data generated by the terminal is acquired, the message data may be stored in the first database entirely. When the message data is stored in the first database, the message data is stored in a sub-database which is successfully connected with the first database, and if the first sub-database is successfully connected, the message data is stored in the first sub-database; and if the connection of the second sub-database is successful, storing the message data into the second sub-database. In the storage process, message data are sequentially stored in each sub-database, namely, in the storage process, if the message data are stored in the first sub-database which is successfully connected at first, after a period of storage, the message data are stored in other sub-databases which are successfully connected, so that the bottleneck of reading and writing caused by a large amount of data storage in the same sub-database is avoided, and the data storage efficiency is reduced.
When the message data is stored in the first sub-database, the message data is stored in the main node in the first sub-database, and after the message data is stored in the main node in the first sub-database, the stored message data is synchronized to the auxiliary node in the first sub-database. The sub-databases adopt a mode of separated reading and writing, so that the main node is responsible for data writing, and the auxiliary node is responsible for data reading, so that when the message data stored in the first database is synchronized into the second database every preset first time, the message data stored in the auxiliary node of the first sub-database is actually synchronized into the second database.
And step 510, periodically deleting the message data in the first database, wherein the time exceeds the preset expiration time.
Step 512, obtain the time information corresponding to each message data stored in the second database.
And step 514, when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data to a file database.
In step 516, when it is detected that the message data whose time information exceeds the preset time is successfully backed up to the file database, the message data whose time information exceeds the preset time is deleted.
After storing the message data in the first database in the second database, the message data in the first database with a time exceeding a preset expiration time may be deleted periodically. Such as deleting message data generated 1 hour ago, the message data stored in the first database is message data generated in the last 1 hour, and historical message data 1 hour ago has been stored in the second database. Each message data has corresponding time information, and after the message data is stored in the second database, the second database can acquire the time information of each message data and backup the message data exceeding the preset time to the file database. Such as backing up 30 days ago message data to a file database.
As shown in the timing diagram of fig. 6, after the message data generated on the terminal is acquired, the message data may be stored in a first database, which may be a dis library. When the first database stores the message data, the message data is actually stored to the main node in the sub-database, and then the main node synchronizes the data to the auxiliary node. After the message data is stored in the first database, the message data can be transferred and stored in the second database through the first database, and the expiration data in the first database can be cleaned regularly. After the second database stores the message data in the first database synchronously, it can detect whether the message data in the second database has message data exceeding the preset duration, for example, detect whether the message data is 30 days ago. If so, the message data exceeding the preset time period can be stored in the file server, and the message data exceeding the preset time period is deleted from the second database. This avoids redundancy of data in the second database. When the message data is queried, the second database can query the message data with the time information not exceeding the preset time, and the file can query the message data with the time information exceeding the preset time.
The acquired terminal data is firstly stored in the first database, the data stored in the first database is synchronized to the second database at intervals of preset first time, and the message data stored in the second database is backed up to the file database at intervals of preset second time, so that when the historical data needs to be acquired, the historical data can be completely acquired whether the historical data is long-term historical message data or short-term historical message data. For example, when the quality inspection department needs to acquire sales data of sales personnel for statistics, historical message data of the sales personnel needs to be acquired, when the quality inspection department needs to acquire the message data of the sales personnel 30 days ago, the information data of the sales personnel can be queried and acquired from a file database, and when the quality inspection department needs to acquire the message data of the sales personnel within 30 days, the information data of the sales personnel can be queried and acquired from a second database.
It should be understood that, although the steps in the flowcharts of fig. 2-5 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2-5 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor does the order in which the sub-steps or stages are performed necessarily occur in sequence, but may be performed alternately or alternately with at least a portion of the other steps or sub-steps of other steps.
In one embodiment, as shown in FIG. 7, there is provided a message data storage device comprising: the device comprises a data acquisition module, a first storage module, a second storage module and a third storage module, wherein:
the data acquisition module 702 is configured to acquire message data generated by a terminal.
A first storage module 704, configured to store the message data into a first database.
The second storage module 706 is configured to synchronize the message data stored in the first database to the second database at intervals of a preset first time.
A third storage module 708, configured to acquire time information corresponding to each message data stored in the second database at intervals of a preset second time; when the time information of the message data exceeds the preset time, the message data with the time information exceeding the preset time in the message data is backed up to the file database.
In one embodiment, the first database includes at least a first sub-database and a second sub-database. The first storage module 704 is further configured to store the message data to the first sub-database when the first sub-database is successfully connected; and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
In one embodiment, the first storage module 704 is further configured to store message data to a master node of the first sub-database; the stored message data is synchronized by the primary node of the first sub-database to the secondary node of the first sub-database. The second storage module 706 is further configured to synchronize the message data stored in the secondary node of the first sub-database to the second database at every preset first time.
In one embodiment, the apparatus further includes a first cleaning module, configured to obtain time information corresponding to the message data in the first database; and deleting the message data of which the time information exceeds the preset expiration time from the first database.
In one embodiment, the apparatus further includes a second cleaning module, configured to delete the message data whose time information exceeds the preset time when it is detected that the message data whose time information exceeds the preset time is successfully backed up to the file database.
In one embodiment, as shown in fig. 8, the apparatus further includes a data query module 710, configured to obtain a message query request, where the message query request carries query time information; acquiring message data corresponding to the inquiry time information from a corresponding database according to the inquiry time information; when the query time information exceeds the query time, acquiring message data corresponding to the query time from a file database; and when the query time information does not exceed the query time, acquiring message data corresponding to the non-exceeded query time from the second database.
For specific limitations on the message data storage means, reference may be made to the limitations on the message data storage method hereinabove, and no further description is given here. The various modules in the message data store described above may be implemented in whole or in part in software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 9. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing message data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a message data storage method.
It will be appreciated by those skilled in the art that the structure shown in fig. 9 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory storing a computer program and a processor that when executing the computer program performs the steps of: acquiring message data generated by a terminal; storing the message data in a first database; synchronizing the message data stored in the first database to the second database at intervals of a preset first time; acquiring time information corresponding to each message data stored in a second database every other preset second time; when the time information of the message data exceeds the preset time, the message data with the time information exceeding the preset time in the message data is backed up to the file database.
In one embodiment, the first database includes at least a first sub-database and a second sub-database. Storing the message data in a first database, comprising: when the first sub-database is successfully connected, storing the message data into the first sub-database; and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
In one embodiment, storing message data to a first sub-database includes: storing the message data to a master node of a first sub-database; the stored message data is synchronized by the primary node of the first sub-database to the secondary node of the first sub-database. Synchronizing the message data stored in the first database to the second database at preset first time intervals, including: and synchronizing the message data of the secondary node stored in the first sub-database to the second database at preset first time intervals.
In one embodiment, after synchronizing the message data stored in the first database into the second database at every preset first time, the processor when executing the computer program further performs the steps of: acquiring time information corresponding to message data in a first database; and deleting message data with time information exceeding the preset expiration time from the first database.
In one embodiment, after backing up the message data whose time information exceeds the preset time in the message data into the file database, the processor when executing the computer program further implements the steps of: when the message data with the time information exceeding the preset time is detected to be successfully backed up to the file database, deleting the message data with the time information exceeding the preset time.
In one embodiment, the processor when executing the computer program further performs the steps of: acquiring a message query request, wherein the message query request carries query time information; acquiring message data corresponding to the inquiry time information from a corresponding database according to the inquiry time information; when the query time information exceeds the query time, acquiring message data corresponding to the query time from a file database; and when the query time information does not exceed the query time, acquiring message data corresponding to the non-exceeded query time from the second database.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of: acquiring message data generated by a terminal; storing the message data in a first database; synchronizing the message data stored in the first database to the second database at intervals of a preset first time; acquiring time information corresponding to each message data stored in a second database every other preset second time; when the time information of the message data exceeds the preset time, the message data with the time information exceeding the preset time in the message data is backed up to the file database.
In one embodiment, the first database includes at least a first sub-database and a second sub-database. Storing the message data in a first database, comprising: when the first sub-database is successfully connected, storing the message data into the first sub-database; and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
In one embodiment, storing message data to a first sub-database includes: storing the message data to a master node of a first sub-database; the stored message data is synchronized by the primary node of the first sub-database to the secondary node of the first sub-database. Synchronizing the message data stored in the first database to the second database at preset first time intervals, including: and synchronizing the message data of the secondary node stored in the first sub-database to the second database at preset first time intervals.
In one embodiment, after synchronizing the message data stored in the first database into the second database at every preset first time, the computer program when executed by the processor further performs the steps of: acquiring time information corresponding to message data in a first database; and deleting message data with time information exceeding the preset expiration time from the first database.
In one embodiment, after backing up the message data whose time information exceeds the preset time in the message data to the file database, the computer program when executed by the processor further performs the steps of: when the message data with the time information exceeding the preset time is detected to be successfully backed up to the file database, deleting the message data with the time information exceeding the preset time.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a message query request, wherein the message query request carries query time information; acquiring message data corresponding to the inquiry time information from a corresponding database according to the inquiry time information; when the query time information exceeds the query time, acquiring message data corresponding to the query time from a file database; and when the query time information does not exceed the query time, acquiring message data corresponding to the non-exceeded query time from the second database.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. A message data storage method, the method comprising:
acquiring message data generated by a terminal;
storing the message data into a first database, wherein the message data is stored into a main node of the first sub-database when the first sub-database in the first database is successfully connected, the main node is used for writing the message data, the message data is synchronized to a secondary node of the first sub-database through the main node of the first sub-database, and the secondary node is used for reading the message data;
Synchronizing the message data stored in the secondary node of the first sub-database to a second database at intervals of a preset first time;
acquiring time information corresponding to each message data stored in a second database every other preset second time;
when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data into a file database.
2. The method of claim 1, wherein the first database comprises at least a first sub-database and a second sub-database; the storing the message data in a first database includes:
when the first sub-database is successfully connected, storing the message data into the first sub-database;
and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
3. The method of claim 2, wherein the storing the message data to the first sub-database comprises:
storing the message data to a master node of the first sub-database;
synchronizing, by a primary node of the first sub-database, the stored message data to a secondary node of the first sub-database;
Synchronizing the message data stored in the first database to a second database at preset first time intervals, including: synchronizing the message data stored in the secondary node of the first sub-database to a second database at preset first time intervals.
4. The method of claim 1, further comprising, after said synchronizing said message data stored in said first database into a second database at every preset first time, the steps of:
acquiring time information corresponding to the message data in the first database;
and deleting the message data of which the time information exceeds the preset expiration time from the first database.
5. The method of claim 1, further comprising, after backing up the message data in which the time information in the message data exceeds a preset time in a file database: and deleting the message data of which the time information exceeds the preset time when the message data of which the time information exceeds the preset time is detected to be successfully backed up to the file database.
6. The method according to claim 1, wherein the method further comprises:
Acquiring a message query request, wherein the message query request carries query time information;
acquiring message data corresponding to the inquiry time information from a corresponding database according to the inquiry time information;
when the query time information exceeds the query time, acquiring message data corresponding to the query time from the file database;
and when the query time information does not exceed the query time, acquiring message data corresponding to the query time from the second database.
7. A message data storage device, the device comprising:
the data acquisition module is used for acquiring message data generated by the terminal;
the first storage module is used for storing the message data into a first database, and comprises a main node for storing the message data into the first sub-database when the first sub-database in the first database is successfully connected, wherein the main node is used for writing the message data, the message data is synchronized to a secondary node of the first sub-database through the main node of the first sub-database, and the secondary node is used for reading the message data;
the second storage module is used for synchronizing the message data stored in the auxiliary node of the first sub-database to a second database at intervals of preset first time;
The third storage module acquires time information corresponding to each message data stored in the second database at intervals of preset second time; when detecting that the time information of the message data exceeds the preset time, backing up the message data with the time information exceeding the preset time in the message data into a file database.
8. The apparatus of claim 7, wherein the first database comprises at least a first sub-database and a second sub-database, and wherein the first storage module is further configured to store the message data to the first sub-database when the first sub-database is successfully connected; and when the second sub-database is successfully connected, storing the message data to the second sub-database after the preset time.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811122763.8A CN109460438B (en) | 2018-09-26 | 2018-09-26 | Message data storage method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811122763.8A CN109460438B (en) | 2018-09-26 | 2018-09-26 | Message data storage method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109460438A CN109460438A (en) | 2019-03-12 |
| CN109460438B true CN109460438B (en) | 2024-04-12 |
Family
ID=65606968
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811122763.8A Active CN109460438B (en) | 2018-09-26 | 2018-09-26 | Message data storage method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109460438B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114138520B (en) * | 2020-05-13 | 2022-06-28 | 浙江创邻科技有限公司 | Distributed message processing method and system |
| CN111966533A (en) * | 2020-07-23 | 2020-11-20 | 招联消费金融有限公司 | Electronic file management method and device, computer equipment and storage medium |
| CN112395270A (en) * | 2020-11-27 | 2021-02-23 | 网络通信与安全紫金山实验室 | Data management method, device, equipment and storage medium |
| WO2022111733A1 (en) * | 2020-11-30 | 2022-06-02 | 百果园技术(新加坡)有限公司 | Message processing method and apparatus, and electronic device |
| CN113468154A (en) * | 2021-06-25 | 2021-10-01 | 浙江数链科技有限公司 | MySQL-based large table volume reduction method and device, electronic device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794246A (en) * | 2010-02-24 | 2010-08-04 | 成都市华为赛门铁克科技有限公司 | Backup storage system, backup system, data backup method and recovery method |
| CN103176860A (en) * | 2011-12-21 | 2013-06-26 | 腾讯科技（深圳）有限公司 | Data backup method and data backup system |
| CN104572838A (en) * | 2014-12-11 | 2015-04-29 | 北京京东尚科信息技术有限公司 | Improved system and method for writing data to database |
| CN106708658A (en) * | 2016-11-29 | 2017-05-24 | 维沃移动通信有限公司 | Data backup method, mobile terminal and storage server |
| CN108196979A (en) * | 2017-12-28 | 2018-06-22 | 北京小度信息科技有限公司 | Data back up method and device |
-
2018
- 2018-09-26 CN CN201811122763.8A patent/CN109460438B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794246A (en) * | 2010-02-24 | 2010-08-04 | 成都市华为赛门铁克科技有限公司 | Backup storage system, backup system, data backup method and recovery method |
| CN103176860A (en) * | 2011-12-21 | 2013-06-26 | 腾讯科技（深圳）有限公司 | Data backup method and data backup system |
| CN104572838A (en) * | 2014-12-11 | 2015-04-29 | 北京京东尚科信息技术有限公司 | Improved system and method for writing data to database |
| CN106708658A (en) * | 2016-11-29 | 2017-05-24 | 维沃移动通信有限公司 | Data backup method, mobile terminal and storage server |
| CN108196979A (en) * | 2017-12-28 | 2018-06-22 | 北京小度信息科技有限公司 | Data back up method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109460438A (en) | 2019-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109460438B (en) | Message data storage method, device, computer equipment and storage medium | |
| US10261719B2 (en) | Volume and snapshot replication | |
| WO2016086552A1 (en) | File processing method and client | |
| CN105843702B (en) | A kind of method and device for data backup | |
| US20080243878A1 (en) | Removal | |
| EP2330519A1 (en) | Distributed file system and data block consistency managing method thereof | |
| CN109407977B (en) | Big data distributed storage management method and system | |
| US11914554B2 (en) | Adaptable multi-layered storage for deduplicating electronic messages | |
| US11194669B2 (en) | Adaptable multi-layered storage for generating search indexes | |
| US11392460B2 (en) | Adaptable multi-layer storage with controlled restoration of protected data | |
| US20110082835A1 (en) | Periodic file system checkpoint manager | |
| WO2019020081A1 (en) | Distributed system and fault recovery method and apparatus thereof, product, and storage medium | |
| WO2016029744A1 (en) | Metadata recovery method and relevant device | |
| US11080142B2 (en) | Preservation of electronic messages between snapshots | |
| US11681586B2 (en) | Data management system with limited control of external compute and storage resources | |
| CN104077380A (en) | Method and device for deleting duplicated data and system | |
| CN103761165A (en) | Log backup method and log backup device | |
| CN113360322A (en) | Method and equipment for recovering data based on backup system | |
| CN103186418B (en) | A kind of task distribution method and system | |
| US10990312B2 (en) | Method, apparatus, device and storage medium for processing data location of storage device | |
| CN114900532A (en) | Power data disaster tolerance method, system, device, computer equipment and storage medium | |
| JP6251965B2 (en) | Information system and database recovery method | |
| CN112818021B (en) | Data request processing method, device, computer equipment and storage medium | |
| CN113312309B (en) | Snapshot chain management method, device and storage medium | |
| CN114416689A (en) | Data migration method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |