CN115550380A - Data synchronization method, device, equipment and storage medium - Google Patents
Data synchronization method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115550380A CN115550380A CN202211127109.2A CN202211127109A CN115550380A CN 115550380 A CN115550380 A CN 115550380A CN 202211127109 A CN202211127109 A CN 202211127109A CN 115550380 A CN115550380 A CN 115550380A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- data set
- synchronization
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
- H04L43/106—Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data synchronization method, a data synchronization device, data synchronization equipment and a storage medium. Relates to the technical field of big data resource scheduling. The method comprises the following steps: receiving a data bitmap code sent by a second server; determining a target data set to be synchronized in a first server according to the coding information at each position in the data bitmap coding; determining a target synchronization mode for performing data synchronization on the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode; and sending the target data set to the second server according to the target synchronization mode. According to the data synchronization method and device, the data are synchronized in the form of the messages and/or the file packets according to the data volume of the data to be synchronized, the real-time performance and the high efficiency of the data synchronization can be considered, and therefore the requirements of timeliness and the high efficiency of the data synchronization are met at the same time.
Description
Technical Field
The present application relates to the field of big data resource scheduling technologies, and in particular, to a data synchronization method, apparatus, device, and storage medium.
Background
In general, an Information Technology (IT) system used by each enterprise, such as a government, a bank, a hospital, a school, an enterprise, etc., is composed of a plurality of subsystems, data stored between the subsystems is dependent and related, and when processing business, synchronization processing of various data is required between the subsystems.
In the prior art, data synchronization is performed in a message form, and when the data quantity needing to be synchronized is large, the data synchronization efficiency is low; data synchronization is performed in a file packet mode, and timeliness of data synchronization cannot be guaranteed. Therefore, the prior art cannot give consideration to both real-time performance and high efficiency of data synchronization, and cannot simultaneously meet the requirements of real-time performance and high efficiency of data synchronization.
Disclosure of Invention
The application provides a data synchronization method, a data synchronization device, data synchronization equipment and a storage medium, which are used for solving the problem that the prior art cannot give consideration to the real-time performance and the high efficiency of data synchronization and cannot simultaneously meet the requirements of the real-time performance and the high efficiency of the data synchronization.
In a first aspect, the present application provides a data synchronization method, performed by a first server, the method including: receiving a data bitmap code sent by a second server; determining a target data set to be synchronized in the first server according to the coding information at each position in the data bitmap coding; determining a target synchronization mode for performing data synchronization on the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode; and sending the target data set to the second server according to the target synchronization mode.
In an embodiment of the present application, the determining a target data set to be synchronized in the first server includes: receiving data synchronization information sent by the second server, wherein the data synchronization information at least comprises the data bitmap coding; and determining the target data set according to the data synchronization information.
In one embodiment of the present application, the data synchronization information includes a user identifier, the data bitmap encoding, and a first timestamp, and the determining the target data set according to the data synchronization information includes: determining a first data set from the candidate data sets of the first server based on the user identification; determining a second data set from the first data set based on the data bitmap encoding; determining, from the second data set, a data set satisfying a data synchronization condition as the target data set based on the first timestamp and a second timestamp of the second data set.
In one embodiment of the present application, the determining, from the second data set, a data set satisfying a data synchronization condition based on the first timestamp and a second timestamp of the second data set includes: comparing the first timestamp with the second timestamp to obtain a candidate second timestamp which is greater than the first timestamp; and taking the data set corresponding to the candidate second timestamp as the data set meeting the data synchronization condition.
In an embodiment of the present application, the determining a target data set to be synchronized in the first server further includes: obtaining a third timestamp of the data set in the first server; determining whether the data set is updated based on the third timestamp; in response to determining that the data set has been updated, determining whether the data set satisfies a data synchronization condition; in response to the data synchronization condition being satisfied, determining the data set to be the target data set.
In an embodiment of the application, the determining a target synchronization mode for data synchronization to the second server according to the data volume of the target data set includes: determining that the message mode is the target synchronization mode in response to the data volume of the target data set being less than or equal to a set data volume threshold; and in response to the data volume of the target data set being greater than a set data volume threshold, determining that the packet mode and the bundle mode are the target synchronization mode, or determining that the bundle mode is the target synchronization mode.
In one embodiment of the present application, the method further comprises: and acquiring a target timestamp of the target data set, and sending the target timestamp to the second server, wherein the target timestamp is used for updating the first timestamp.
In a second aspect, the present application provides another data synchronization method, performed by a second server, the method comprising: generating a data bitmap code, and sending the data bitmap code to a first server, wherein the data bitmap code is used for determining a target data set to be synchronized in the first server; and receiving the target data set sent by the first server according to a target synchronization mode, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode.
In one embodiment of the present application, the method further comprises: acquiring a user identifier, and determining a data set of current service requirements based on the user identifier and a current service scene; acquiring a data bitmap code and a time stamp of the data set; generating data synchronization information based on the user identification, the data bitmap encoding and the timestamp; sending the data synchronization information to the first server, wherein the data synchronization information is used for determining the target data set.
In one embodiment of the present application, the method further comprises: and receiving a target timestamp of the target data set sent by the first server, and updating the timestamp to be the target timestamp.
In a third aspect, the present application provides a data synchronization apparatus, including: the receiving module is used for receiving the data bitmap code sent by the second server; the first determining module is used for determining a target data set to be synchronized in the first server according to the coding information at each position in the data bitmap coding; a second determining module, configured to determine, according to the data size of the target data set, a target synchronization mode for performing data synchronization to a second server, where the target synchronization mode includes at least one of a packet mode and a package mode; a first sending module, configured to send the target data set to the second server according to the target synchronization mode.
In an embodiment of the application, the first determining module is further configured to: receiving data synchronization information sent by the second server, wherein the data synchronization information at least comprises the data bitmap coding; and determining the target data set according to the data synchronization information.
In an embodiment of the application, the data synchronization information includes a user identifier, the data bitmap coding, and a first timestamp, and the first determining module is further configured to: determining a first data set from the candidate data sets of the first server based on the user identification; determining a second data set from the first data set based on the data bitmap encoding; determining a data set satisfying a data synchronization condition from the second data set as the target data set based on the first time stamp and a second time stamp of the second data set.
In an embodiment of the application, the first determining module is further configured to: comparing the first timestamp with the second timestamp to obtain a candidate second timestamp which is greater than the first timestamp; and taking the data set corresponding to the candidate second timestamp as the data set meeting the data synchronization condition.
In an embodiment of the application, the first determining module is further configured to: obtaining a third timestamp of the data set in the first server; determining whether the data set is updated based on the third timestamp; in response to determining that the data set has been updated, determining whether the data set satisfies a data synchronization condition; in response to the data synchronization condition being satisfied, determining the data set to be the target data set.
In an embodiment of the application, the second determining module is further configured to: determining that the message mode is the target synchronization mode in response to the data volume of the target data set being less than or equal to a set data volume threshold; and in response to the fact that the data volume of the target data set is larger than a set data volume threshold value, determining that the message mode and the file packet mode are the target synchronization mode, or determining that the file packet mode is the target synchronization mode.
In one embodiment of the present application, the apparatus further comprises: and the second sending module is used for obtaining a target timestamp of the target data set and sending the target timestamp to the second server, wherein the target timestamp is used for updating the first timestamp.
In a fourth aspect, the present application provides another data synchronization apparatus, including: the system comprises a first generation module, a second generation module and a synchronization module, wherein the first generation module is used for generating data bitmap codes and sending the data bitmap codes to a first server, and the data bitmap codes are used for determining a target data set to be synchronized in the first server; a first receiving module, configured to receive the target data set sent by the first server according to a target synchronization mode, where the target synchronization mode includes at least one of a packet mode and a file package mode.
In an embodiment of the present application, the apparatus further includes a second generating module, configured to: acquiring a user identifier, and determining a data set of current service requirements based on the user identifier and a current service scene; acquiring a data bitmap code and a time stamp of the data set; generating data synchronization information based on the user identification, the data bitmap encoding and the timestamp; sending the data synchronization information to the first server, wherein the data synchronization information is used for determining the target data set.
In one embodiment of the present application, the apparatus further comprises: and the second receiving module is used for receiving the target timestamp of the target data set sent by the first server and updating the timestamp to the target timestamp.
In a fifth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor; the memory stores computer-executable instructions; the processor executes computer-executable instructions stored by the memory to implement a data synchronization method as in the embodiments of the first or second aspects described above.
In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the data synchronization method as described in the first aspect or the second aspect.
In a seventh aspect, the present application provides a computer program product, which is characterized by comprising a computer program, and the computer program is used for implementing the data synchronization method according to the embodiment of the first aspect or the second aspect when being executed by a processor.
The technical scheme provided by the embodiment of the application at least has the following beneficial effects: according to the data volume of the data to be synchronized, the data is synchronized in the form of messages and/or file packets, and the real-time performance and the high efficiency of data synchronization can be considered, so that the requirements of timeliness and the high efficiency of the data synchronization are met at the same time.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a first flowchart of a data synchronization method according to an embodiment of the present application;
fig. 2 is a second flowchart of a data synchronization method according to an embodiment of the present application;
fig. 3 is a flow chart of a data synchronization method according to an embodiment of the present application;
fig. 4 is a fourth flowchart of a data synchronization method provided in the embodiment of the present application;
fig. 5 is a fifth flowchart of a data synchronization method according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of another data synchronization apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. The drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the concepts of the application by those skilled in the art with reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of the present application. As used in the examples of this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the embodiments of the present application. The words "if" and "if," as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination," depending on the context.
Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the like or similar elements throughout. The embodiments described below with reference to the accompanying drawings are illustrative and intended to explain the present application and should not be construed as limiting the present application.
According to the technical scheme, the data acquisition, storage, use, processing and the like meet the relevant regulations of national laws and regulations.
A data synchronization method, an apparatus, an electronic device, and a storage medium according to embodiments of the present application are described below with reference to the accompanying drawings.
Fig. 1 is a first flowchart of a data synchronization method according to an embodiment of the present disclosure. It should be noted that the main execution body of the data synchronization method provided by the embodiment of the present application is the first server.
As shown in fig. 1, the data synchronization method includes:
and S101, receiving the data bitmap code sent by the second server.
Wherein the bitmap coding of the data comprises multi-bit coding, and the coding of each position corresponds to a data set. Alternatively, the code may be 0 or 1, where 0 indicates that synchronization is not required and 1 indicates that synchronization is required.
The second server can generate corresponding data bitmap coding according to the data set required by the current service, and send the data bitmap coding to the first server.
S102, determining a target data set to be synchronized in the first server according to the coding information at each position in the data bitmap coding.
In the implementation of the present application, the data in the first server may be divided into a plurality of candidate data sets in advance. Alternatively, the data of each user in the first server may be used as a candidate data set. Since users often use only specific data when transacting business, the candidate data set can be further divided into a plurality of data sets, wherein each data set corresponds to a position code in the data bitmap code.
Optionally, the coding information comprises 0 or 1.
And after receiving the data bitmap code sent by the second server, the first server takes the data set corresponding to the position coded as 1 in the data bitmap code as a target data set to be synchronized.
It should be noted that, when the data set is divided, the dividing manner of the data in the first server is consistent with that of the data in the second server, where the dividing manner may be set according to actual requirements, and is not limited here.
S103, determining a target synchronization mode for performing data synchronization to the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode.
The message mode is a data synchronization mode for transmitting data by a message.
The package mode is a data synchronization mode in which data is transmitted through a package.
Optionally, in response to that the data amount of the target data set is less than or equal to the set data amount threshold, the packet mode is determined to be the target synchronization mode, and in response to that the data amount of the target data set is greater than the set data amount threshold, the packet mode and the bundle mode are determined to be the target synchronization mode, or the bundle mode is determined to be the target synchronization mode.
When the synchronous data volume is small, the data is synchronized through the message, so that the real-time synchronization of the data is realized, and when the synchronous data volume is large, the data is synchronized through the message and the file packet, or the data is synchronized through the file packet, so that the data synchronization efficiency can be ensured, and the real-time performance and the high efficiency of the data synchronization can be considered.
And S104, sending the target data set to the second server according to the target synchronization mode.
If the target synchronization mode is a message mode, the first server sends a target data set to the second server through a message; if the target synchronization mode is a message mode and a file packet mode, the first server sends a target data set to the second server through a message and a file packet; and if the target synchronization mode is a file packet mode, the first server sends the target data set to the second server through a file packet.
In the embodiment of the application, a data bitmap code sent by a second server is received, a target data set to be synchronized in a first server is determined according to coding information at each position in the data bitmap code, a target synchronization mode for performing data synchronization to the second server is determined according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode, and the target data set is sent to the second server according to the target synchronization mode. In the embodiment of the application, the data is determined to be synchronized through the message and/or the file packet according to the data volume of the data to be synchronized, and the real-time performance and the high efficiency of data synchronization can be considered, so that the requirements of timeliness and the high efficiency of the data synchronization are met at the same time.
Fig. 2 is a second flowchart of a data synchronization method according to an embodiment of the present application. It should be noted that the main execution body of the data synchronization method provided by the embodiment of the present application is the first server.
As shown in fig. 2, the data synchronization method includes the steps of:
s201, receiving data synchronization information sent by a second server. The data synchronization information at least comprises data bitmap coding.
The second server may generate data synchronization information according to the data synchronization requirement and send the data synchronization information to the first server.
Optionally, the second server sends the data synchronization information to the first server through a message.
S202, determining a target data set according to the data synchronization information.
Optionally, the data synchronization information comprises a user identification, a data bitmap encoding and a first timestamp.
The user identifier includes an Identity Document (ID).
A timestamp is a complete verifiable piece of data that can indicate that a piece of data already exists at a particular point in time. The first timestamp is a timestamp of a data set indicated by the bitmap encoding of the data in the second server.
On the basis of the above embodiment, determining a target data set according to data synchronization information includes:
step a1, determining a first data set from the candidate data set of the first server based on the user identification.
A first data set for the user may be looked up from a candidate data set in the first server with the user identification as an index.
And a2, determining a second data set from the first data set based on data bitmap coding.
Optionally, the first data set may be divided into a plurality of data sets according to a service scenario, the data sets are arranged according to a set sequence to obtain a data set sequence, and then the data set sequence is represented by data bitmap coding, where each bit in the data bitmap coding corresponds to one data set in the data set sequence.
For example, assuming that the first data set is divided into 10 data sets, the number of bits of the data bitmap encoding is 10 bits, such as 0000000010, 0100000000, and so on.
Since the code 1 indicates the requirement synchronization, the data set corresponding to the position coded as 1 in the data bitmap coding can be used as the second data set, for example, if the data bitmap coding is 0000000010, the 9 th data set in the data set sequence is the second data set, and if the data bitmap coding is 0100000000, the 2 nd data set in the data set sequence is the second data set.
And a3, determining a data set meeting the data synchronization condition from the second data set as a target data set based on the first time stamp and the second time stamp of the second data set.
Optionally, the first timestamp is compared with the second timestamp, a candidate second timestamp greater than the first timestamp is obtained, and a data set corresponding to the candidate second timestamp is used as the data set meeting the data synchronization condition.
For example, suppose that a user (ID: u 00001) enters a system of a second server for transaction for the first time, user data is stored in a first server, and at this time, the second server does not have the user data, and in order to perform normal transaction for the user, the second server needs to synchronize the user data with the first server to perform normal transaction.
In this case, the second server may send the data synchronization information to the first server in a message format, where the message format is as follows:
assuming that the first data set corresponding to user u00001 is divided into 10 second data sets, the bitmap encoding Scope:1111111111 indicates that all data in the first data set corresponding to user u00001 needs to be synchronized, i.e. the full amount of data corresponding to user u00001 needs to be synchronized. In addition, since the user u00001 transacts the business in the system of the second service for the first time, the data of the user u00001 does not exist in the second server at this time, and the first timestamp is null (null).
After receiving the message sent by the second server, the first server may parse the message to obtain UserID: u00001 and Scope:1111111111, and search the first data set from its own candidate data set using u00001 as an index, and after searching the first data set, obtain a second timestamp of each second data set in the first server because Scope:1111111111 corresponds to each second data set in the first data set. Since the first timestamp of the second data set in the second server is empty, the first timestamp may be considered to be less than the second timestamp, at which point each of the second data sets in the first data set may be determined to be the target data set.
Assuming that the user (ID: u 00001) does not enter the system of the second server for the first time to transact business, the data of the user exists in the second server. The message format is as follows:
1000000001 indicates that the 1 st data set and the 10 th data set in the data set sequence of the first data set need to be synchronized.
S203, determining a target synchronization mode for performing data synchronization to the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode.
And S204, sending the target data set to the second server according to the target synchronization mode.
For example, assuming that the target synchronization mode is the message mode, the message format is as follows:
the fact that Same:0 indicates that the data set in the first server indicated by Scope:1000000001 is inconsistent with the data set indicated by Scope:1000000001 and data synchronization is required, and correspondingly, the fact that the data set in the first server indicated by Scope:1000000001 is consistent with the data set indicated by Scope:1000000001 and data synchronization is not required is achieved. TimeStamp _2 represents a target TimeStamp of the target data set. Null indicates that no portfolio is included. Data1 and Data10 represent the target Data set synchronized to the second server.
Assuming that the target synchronization mode is a message mode and a file packet mode, the message format is as follows:
wherein fileName: fileNameXXX indicates that the file packet with the file name of fileNameXXX returns synchronous data.
For specific descriptions of steps S203-S204, reference may be made to the descriptions of relevant contents in the above embodiments, and details are not repeated here.
Further, the data synchronization method further comprises: and acquiring a target timestamp of the target data set, and sending the target timestamp to the second server, wherein the target timestamp is used for updating the first timestamp.
A target timestamp of the target data set may be obtained and the target data set and the target timestamp may be sent to the second server together to update the first timestamp to the target timestamp.
In the embodiment of the application, data synchronization information sent by a second server is received, wherein the data synchronization information at least comprises data bitmap codes, a target data set is determined according to the data synchronization information, a target synchronization mode for performing data synchronization to the second server is determined according to the data volume of the target data set, the target synchronization mode comprises at least one of a message mode and a file packet mode, and the target data set is sent to the second server according to the target synchronization mode. In the embodiment of the application, the data is synchronized in the form of the message and/or the file packet according to the data volume of the data to be synchronized, and the real-time performance and the high efficiency of data synchronization can be considered, so that the requirements of timeliness and the high efficiency of the data synchronization are met at the same time.
When data in the first server is updated, the first server may actively synchronize the updated data to the second server, as explained below with reference to fig. 3. Fig. 3 is a third flowchart of a data synchronization method according to an embodiment of the present application. It should be noted that the main execution body of the data synchronization method provided by the embodiment of the present application is the first server.
As shown in fig. 3, the data synchronization method includes the following steps:
s301, acquiring a third time stamp of the data set in the first service.
And S302, determining whether the data set is updated or not based on the third time stamp.
And S303, in response to determining that the data set is updated, determining whether the data set meets a data synchronization condition.
And S304, in response to the data synchronization condition being met, determining the data set as a target data set.
And after receiving the data bitmap code, the second server determines the data set corresponding to the data bitmap code from the data set of the second server, and sends a fourth timestamp of the data set corresponding to the data bitmap code to the first server. And after receiving the fourth timestamp, the first server determines whether the fourth timestamp is smaller than the third timestamp, if so, determines that the data set meets the data synchronization condition, and determines the data set as a target data set.
S305, determining a target synchronization mode for performing data synchronization to the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file package mode.
S306, sending the target data set to the second server according to the target synchronization mode.
For specific descriptions of steps S305-S306, reference may be made to the descriptions of relevant contents in the above embodiments, and details are not repeated here.
In the embodiment of the application, a third timestamp of a data set in a first server is obtained, whether the data set is updated is determined based on the third timestamp, whether the data set meets a data synchronization condition is determined in response to determining that the data set is updated, the data set is determined to be a target data set in response to meeting the data synchronization condition, a target synchronization mode for performing data synchronization on a second server is determined according to the data volume of the target data set, wherein the target synchronization mode includes at least one of a message mode and a file package mode, and the target data set is sent to the second server according to the target synchronization mode. In the embodiment of the application, when the data in the first server is updated, the first server can actively synchronize the updated data with the second server, so that the first consistency of the data in the first server and the data in the second server can be ensured, and the normal business handling of a user can be ensured.
Fig. 4 is a fourth flowchart of a data synchronization method according to an embodiment of the present application. It should be noted that the execution subject of the data synchronization method provided in the embodiment of the present application is the second server.
As shown in fig. 4, the data synchronization method includes the steps of:
s401, generating data bitmap codes, and sending the data bitmap codes to the first server, wherein the data bitmap codes are used for determining a target data set to be synchronized in the first server.
The second server can generate data bitmap coding according to the data synchronization requirement, and send the data bitmap coding to the first server.
Optionally, the second server sends the bitmap code of the data to the first server through a message.
S402, receiving a target data set sent by a first server according to a target synchronization mode, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode.
Optionally, after receiving the bitmap code sent by the second server, the first server determines the target data set according to the bitmap code, and obtains the data volume of the target data set, if the data volume of the target data set is less than or equal to a set data volume threshold, it is determined that the packet mode is the target synchronization mode, and if the data volume of the target data set is greater than the set data volume threshold, it is determined that the packet mode and the bundle mode are the target synchronization mode, or it is determined that the bundle mode is the target synchronization mode.
If the target synchronization mode is a message mode, the first server sends a target data set to the second server through a message; if the target synchronization mode is a message mode and a file packet mode, the first server sends a target data set to the second server through a message and a file packet; and if the target synchronization mode is a file packet mode, the first server sends the target data set to the second server through a file packet.
In the embodiment of the application, a data bitmap code is generated and sent to a first server, and the data bitmap code is used for determining a target data set to be synchronized in the first server and receiving the target data set sent by the first server according to a target synchronization mode, wherein the target synchronization mode includes at least one of a message mode and a file packet mode. In the embodiment of the application, the data is determined to be synchronized through the message and/or the file packet according to the data volume of the data to be synchronized, and the real-time performance and the high efficiency of data synchronization can be considered, so that the requirements of timeliness and the high efficiency of the data synchronization are met at the same time.
Fig. 5 is a fifth flowchart of a data synchronization method according to an embodiment of the present application. It should be noted that an execution subject of the data synchronization method provided in the embodiment of the present application is the second server.
As shown in fig. 5, the data synchronization method includes the steps of:
s501, obtaining a user identifier, and determining a data set of current service requirements based on the user identifier and a current service scene.
Optionally, the user identification comprises a user ID.
When the user transacts the service on the system of the second server, the user identification can be entered into the second server through the system.
In the embodiment of the present application, all data in the second server may be divided into a plurality of candidate data sets in advance. Alternatively, the data of each user in the first server may be used as a candidate data set.
Further, each candidate data set may be divided into a plurality of data sets according to a traffic scenario.
In some embodiments, a target candidate data set corresponding to the user is determined from the candidate data set of the second server according to the user identifier, and then a data set of the current service requirement is determined from the target candidate data set according to the current service scenario, where the data set is a data set required to be synchronized in the second server.
And S502, acquiring data bitmap coding and time stamp of the data set.
Since each data set in the target candidate data set corresponds to one bit of the data bitmap code, the data bitmap code can be determined according to the data set synchronization requirements in the target candidate data set.
For example, assuming that the target candidate data set is divided into 10 data sets, the number of bits of the bitmap encoding of the data is 10. If the 10 th data set and the 10 th data set in the target candidate data set need data synchronization, the data bitmap is encoded to be 1000000001, wherein the code 1 indicates that the data synchronization is needed, and the code 0 indicates that the data synchronization is not needed.
S503, generating data synchronization information based on the user identification, the data bitmap coding and the time stamp.
S504, sending data synchronization information to the first server, wherein the data synchronization information is used for determining a target data set to be synchronized in the first server.
After the user identifier, the data bitmap code and the timestamp are obtained, data synchronization information can be generated according to the data bitmap code, the data synchronization information is sent to the first server, and after the first server receives the data synchronization information, a target data set to be synchronized is determined according to the data synchronization information.
Optionally, the data synchronization information is sent to the first server by a message.
Illustratively, the message format is as follows:
the UserID u00001 represents a user identifier, the Type 1 represents that a data sending mode is a message, the data bitmap coding Scope 1000000001 represents that the 1 st data set and the 10 th data set in the candidate data set need to be synchronized, and the TimeStamp represents a TimeStamp.
S505, receiving a target data set sent by the first server according to a target synchronization mode, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode.
For specific descriptions of steps S504-S505, reference may be made to the descriptions of relevant contents in the above embodiments, and details are not described here.
And after receiving the target data set sent by the first server, the second server updates the data set required by the current service into the target data set, namely, the data set required to be updated in the second server is updated into the target data set, and the data set required by the current service in the second server is deleted.
Further, a target timestamp of the target data set sent by the first server is received, and the timestamp is updated to the target timestamp.
When the first server sends the target data set to the second server, the target timestamp of the target data set may be sent to the second server, and after receiving the target timestamp, the second server updates the timestamp of the data set of the current service demand to the target timestamp.
In the embodiment of the application, a user identifier is obtained, a data set of a current service requirement is determined based on the user identifier and a current service scene, a data bitmap code and a timestamp of the data set are obtained, data synchronization information is generated based on the user identifier, the data bitmap code and the timestamp, and the data synchronization information is sent to a first server. In the embodiment of the application, the data is determined to be synchronized through the message and/or the file packet according to the data volume of the data to be synchronized, and the real-time performance and the high efficiency of data synchronization can be considered, so that the requirements of timeliness and the high efficiency of the data synchronization are met at the same time.
Fig. 6 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application. As shown in fig. 6, the data synchronization apparatus 600 includes: a receiving module 610, a first determining module 620, a second determining module 630, and a first transmitting module 640.
The receiving module 610 is configured to receive a bitmap code of data sent by a second server;
a first determining module 620, configured to determine, according to coding information at each position in the bitmap coding, a target data set to be synchronized in the first server;
a second determining module 630, configured to determine a target synchronization mode for performing data synchronization on the second server according to a data amount of the target data set, where the target synchronization mode includes at least one of a packet mode and a file package mode;
a first sending module 640, configured to send the target data set to the second server according to the target synchronization mode.
In some embodiments, the first determining module 610 is further configured to: receiving data synchronization information sent by a second server, wherein the data synchronization information at least comprises data bitmap coding; and determining a target data set according to the data synchronization information.
In some embodiments, the data synchronization information includes a user identification, a data bitmap encoding, and a first timestamp, and the first determination module 620 is further configured to: determining a first data set from the candidate data sets of the first server based on the user identification; determining a second data set from the first data set based on the data bitmap encoding; and determining the data set meeting the data synchronization condition from the second data set as a target data set based on the first time stamp and the second time stamp of the second data set.
In some embodiments, the first determination module 620 is further configured to: comparing the first time stamp with the second time stamp to obtain a candidate second time stamp larger than the first time stamp; and taking the data set corresponding to the candidate second timestamp as the data set meeting the data synchronization condition.
In some embodiments, the first determination module 620 is further configured to: obtaining a third timestamp of the candidate data set; determining whether the candidate data set is updated based on the third timestamp; in response to determining that the candidate data set has been updated, determining whether the candidate data set satisfies a data synchronization condition; in response to the data synchronization condition being satisfied, the candidate data set is determined to be the target data set.
In some embodiments, the second determination module 630 is further configured to: determining that the message mode is a target synchronization mode in response to the data volume of the target data set being less than or equal to a set data volume threshold; and in response to the data volume of the target data set being larger than the set data volume threshold, determining that the message mode and the file packet mode are the target synchronization mode, or determining that the file packet mode is the target synchronization mode.
In some embodiments, the data synchronization apparatus 600 further comprises a second sending module 650 for obtaining a target timestamp of the target data set and sending the target timestamp to the second server, the target timestamp being used for updating the first timestamp.
The data synchronization apparatus provided in the embodiment of the present application may be used to implement the technical solution of the data synchronization method in the first aspect of the foregoing embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 7 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application. As shown in fig. 7, the data synchronization apparatus 700 includes: a first generating module 710 and a first receiving module 720.
The first generating module 710 is configured to generate a data bitmap code, and send the data bitmap code to the first server, where the data bitmap code is used to determine a target data set to be synchronized in the first server;
the first receiving module 720 is configured to receive a target data set sent by a first server according to a target synchronization mode, where the target synchronization mode includes at least one of a packet mode and a file package mode.
In some embodiments, the data synchronization apparatus 700 further includes a second generating module 730, configured to obtain a user identifier, and determine a data set of a current service requirement based on the user identifier and a current service scenario; acquiring data bitmap coding and a time stamp of a data set; generating data synchronization information based on the user identification, the data bitmap coding and the timestamp; and sending data synchronization information to the first server, wherein the data synchronization information is used for determining the target data set.
In some embodiments, the data synchronization apparatus 700 further includes: the second receiving module 740 is configured to receive a target timestamp of the target data set sent by the first server, and update the timestamp to the target timestamp.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 8, the electronic device may include: transceiver 121, processor 122, memory 123.
The processor 122 executes computer-executable instructions stored in the memory, causing the processor 122 to perform the aspects of the embodiments described above. The processor 122 may be a general-purpose processor including a central processing unit CPU, a Network Processor (NP), and the like; but also a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
The transceiver 121 may be used to acquire the task to be executed and the configuration information of the task to be executed.
The system bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The transceiver is used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). The memory may include Random Access Memory (RAM) and may also include non-volatile memory (non-volatile memory).
The electronic device provided by the embodiment of the application may be the terminal device of the above embodiment.
The embodiment of the present application further provides a chip for executing the instruction, where the chip is configured to execute the technical solution of the data synchronization method in the first aspect or the second aspect.
An embodiment of the present application further provides a computer-readable storage medium, where a computer instruction is stored in the computer-readable storage medium, and when the computer instruction runs on a computer, the computer is enabled to execute the technical solution of the data synchronization method in the first aspect or the second aspect.
The present application further provides a computer program product, where the computer program product includes a computer program, which is stored in a computer-readable storage medium, and at least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the at least one processor can implement the technical solution of the data synchronization method in the foregoing first aspect or second aspect.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (23)
1. A data synchronization method, performed by a first server, the method comprising:
receiving a data bitmap code sent by a second server;
determining a target data set to be synchronized in the first server according to the coding information at each position in the data bitmap coding;
determining a target synchronization mode for performing data synchronization on the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode;
and sending the target data set to the second server according to the target synchronization mode.
2. The method of claim 1, wherein determining the target data set to be synchronized in the first server comprises:
receiving data synchronization information sent by the second server, wherein the data synchronization information at least comprises the data bitmap coding;
and determining the target data set according to the data synchronization information.
3. The method of claim 2, wherein the data synchronization information includes a user identification, the data bitmap encoding, and a first timestamp, and wherein determining the target data set from the data synchronization information comprises:
determining a first data set from the candidate data sets of the first server based on the user identification;
determining a second data set from the first data set based on the data bitmap encoding;
determining a data set satisfying a data synchronization condition from the second data set as the target data set based on the first time stamp and a second time stamp of the second data set.
4. The method of claim 3, wherein determining the data set from the second data set that satisfies the data synchronization condition based on the first timestamp and a second timestamp of the second data set comprises:
comparing the first timestamp with the second timestamp to obtain a candidate second timestamp which is greater than the first timestamp;
and taking the data set corresponding to the candidate second timestamp as the data set meeting the data synchronization condition.
5. The method of claim 1, wherein determining the target data set to be synchronized in the first server further comprises:
obtaining a third timestamp of the data set in the first server;
determining whether the data set is updated based on the third timestamp;
in response to determining that the data set has been updated, determining whether the data set satisfies a data synchronization condition;
in response to the data synchronization condition being satisfied, determining the data set to be the target data set.
6. The method according to any one of claims 1-5, wherein the determining a target synchronization pattern for data synchronization to the second server according to the data volume of the target data set comprises:
determining that the message mode is the target synchronization mode in response to the data volume of the target data set being less than or equal to a set data volume threshold;
and in response to the data volume of the target data set being greater than a set data volume threshold, determining that the packet mode and the bundle mode are the target synchronization mode, or determining that the bundle mode is the target synchronization mode.
7. The method according to claim 3 or 5, characterized in that the method further comprises:
and acquiring a target timestamp of the target data set, and sending the target timestamp to the second server, wherein the target timestamp is used for updating the first timestamp.
8. A data synchronization method, performed by a second server, the method comprising:
generating a data bitmap code, and sending the data bitmap code to a first server, wherein the data bitmap code is used for determining a target data set to be synchronized in the first server;
and receiving the target data set sent by the first server according to a target synchronization mode, wherein the target synchronization mode comprises at least one of a message mode and a file packet mode.
9. The method of claim 8, further comprising:
acquiring a user identifier, and determining a data set of current service requirements based on the user identifier and a current service scene;
acquiring a data bitmap code and a time stamp of the data set;
generating data synchronization information based on the user identification, the data bitmap encoding and the timestamp;
sending the data synchronization information to the first server, wherein the data synchronization information is used for determining the target data set.
10. The method of claim 9, further comprising:
and receiving a target timestamp of the target data set sent by the first server, and updating the timestamp to the target timestamp.
11. A data synchronization apparatus, comprising:
the receiving module is used for receiving the data bitmap code sent by the second server;
the first determining module is used for determining a target data set to be synchronized in the first server according to the coding information at each position in the data bitmap coding;
a second determining module, configured to determine, according to the data size of the target data set, a target synchronization mode for performing data synchronization to a second server, where the target synchronization mode includes at least one of a packet mode and a package mode;
a first sending module, configured to send the target data set to the second server according to the target synchronization mode.
12. The apparatus of claim 11, wherein the first determining module is further configured to:
receiving data synchronization information sent by the second server, wherein the data synchronization information at least comprises the data bitmap coding;
and determining the target data set according to the data synchronization information.
13. The apparatus of claim 12, wherein the data synchronization information comprises a user identification, the data bitmap encoding, and a first timestamp, and wherein the first determining module is further configured to:
determining a first data set from the candidate data sets of the first server based on the user identification;
determining a second data set from the first data set based on the data bitmap encoding;
determining a data set satisfying a data synchronization condition from the second data set as the target data set based on the first time stamp and a second time stamp of the second data set.
14. The apparatus of claim 13, wherein the first determining module is further configured to:
comparing the first timestamp with the second timestamp to obtain a candidate second timestamp which is greater than the first timestamp;
and taking the data set corresponding to the candidate second timestamp as the data set meeting the data synchronization condition.
15. The apparatus of claim 11, wherein the first determining module is further configured to:
obtaining a third timestamp of the data set in the first server;
determining whether the data set is updated based on the third timestamp;
in response to determining that the data set has been updated, determining whether the data set satisfies a data synchronization condition;
in response to the data synchronization condition being satisfied, determining the data set to be the target data set.
16. The apparatus according to any of claims 11-15, wherein the second determining module is further configured to:
determining that the message mode is the target synchronization mode in response to the data volume of the target data set being less than or equal to a set data volume threshold;
and in response to the data volume of the target data set being greater than a set data volume threshold, determining that the packet mode and the bundle mode are the target synchronization mode, or determining that the bundle mode is the target synchronization mode.
17. The apparatus of claim 13 or 15, further comprising:
and the second sending module is used for obtaining a target timestamp of the target data set and sending the target timestamp to the second server, wherein the target timestamp is used for updating the first timestamp.
18. A data synchronization apparatus, comprising:
the system comprises a first generation module, a second generation module and a synchronization module, wherein the first generation module is used for generating data bitmap codes and sending the data bitmap codes to a first server, and the data bitmap codes are used for determining a target data set to be synchronized in the first server;
a first receiving module, configured to receive the target data set sent by the first server according to a target synchronization mode, where the target synchronization mode includes at least one of a packet mode and a file bundle mode.
19. The apparatus of claim 18, further comprising a second generation module configured to:
acquiring a user identifier, and determining a data set of current service requirements based on the user identifier and a current service scene;
acquiring a data bitmap code and a time stamp of the data set;
generating data synchronization information based on the user identification, the data bitmap encoding and the timestamp;
sending the data synchronization information to the first server, wherein the data synchronization information is used for determining the target data set.
20. The apparatus of claim 19, further comprising:
a second receiving module, configured to receive a target timestamp of the target data set sent by the first server, and update the timestamp to the target timestamp.
21. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored by the memory to implement the method of any of claims 1-7 or claims 8-10.
22. A computer-readable storage medium having computer-executable instructions stored therein, which when executed by a processor, are configured to implement the method of any one of claims 1-7 or claims 8-10.
23. A computer program product, comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-7 or claims 8-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211127109.2A CN115550380A (en) | 2022-09-16 | 2022-09-16 | Data synchronization method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211127109.2A CN115550380A (en) | 2022-09-16 | 2022-09-16 | Data synchronization method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115550380A true CN115550380A (en) | 2022-12-30 |
Family
ID=84726852
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211127109.2A Pending CN115550380A (en) | 2022-09-16 | 2022-09-16 | Data synchronization method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115550380A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117827957A (en) * | 2024-01-03 | 2024-04-05 | 广州安广电子科技股份有限公司 | Big data-based information input visualization method, system, medium and equipment |
-
2022
- 2022-09-16 CN CN202211127109.2A patent/CN115550380A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117827957A (en) * | 2024-01-03 | 2024-04-05 | 广州安广电子科技股份有限公司 | Big data-based information input visualization method, system, medium and equipment |
| CN117827957B (en) * | 2024-01-03 | 2024-06-11 | 广州安广电子科技股份有限公司 | Big data-based information input visualization method, system, medium and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110119292B (en) | System operation parameter query method, matching method, device and node equipment | |
| CN109218352B (en) | Consensus confirmation method and device for transaction information in blockchain network | |
| CN114500690B (en) | Interface data processing method and device, electronic equipment and storage medium | |
| CN112005523A (en) | Method and node for enabling intelligent contracts to be executed in blockchain networks | |
| CN110297944B (en) | Distributed XML data processing method and system | |
| CN111031135B (en) | Message transmission method and device and electronic equipment | |
| CN109617647A (en) | Message forwarding method, device, computer equipment and computer readable storage medium | |
| CN115550380A (en) | Data synchronization method, device, equipment and storage medium | |
| CN115470156A (en) | RDMA-based memory use method, system, electronic device and storage medium | |
| CN110069533A (en) | A kind of event subscription method and device based on block chain | |
| CN110297948B (en) | Relational network construction method and device | |
| CN111324645B (en) | Block chain data processing method and device | |
| CN115470302A (en) | Database bidirectional synchronization method, medium and equipment based on canal | |
| CN112860746B (en) | Cache reduction-based method, equipment and system | |
| CN115033599A (en) | Graph query method, system and related device based on multi-party security | |
| CN114549137A (en) | Request processing method and device | |
| CN113986828A (en) | Method and device for storing mass files, electronic equipment and storage medium | |
| CN113032820A (en) | File storage method, access method, device, equipment and storage medium | |
| CN114598559B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN117635081B (en) | Data processing method, device, equipment and storage medium | |
| CN115757494A (en) | Message processing method, device, equipment and storage medium | |
| CN117235163A (en) | Service processing method, device, equipment and storage medium | |
| CN117312644A (en) | Data query method, device and equipment | |
| CN117851082A (en) | Micro-service architecture system control method and device based on data modeling | |
| CN115834492A (en) | Method, device, equipment and medium for processing credit service data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |