CN110941665A - Data synchronization method, data synchronization device and data synchronization equipment between nodes - Google Patents
Data synchronization method, data synchronization device and data synchronization equipment between nodes Download PDFInfo
- Publication number
- CN110941665A CN110941665A CN201911053648.4A CN201911053648A CN110941665A CN 110941665 A CN110941665 A CN 110941665A CN 201911053648 A CN201911053648 A CN 201911053648A CN 110941665 A CN110941665 A CN 110941665A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- incremental
- nodes
- synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for synchronizing data among nodes, which is realized based on monitoring nodes, and comprises the steps of firstly carrying out real-time incremental synchronous deployment on each node to realize real-time incremental synchronization among the nodes, realizing the minimization of bandwidth occupied by the data synchronization process, simultaneously monitoring the transmission process of incremental data among the nodes, and recording the information of incremental data transmission failure when the incremental data transmission fails, on the basis, the transmission failure incremental data is conveniently monitored by manpower or machines and is subsequently processed, so that the accuracy of the data synchronization among the nodes is ensured. Therefore, the method for synchronizing the data among the nodes provided by the invention not only ensures the accuracy of the data synchronization among the nodes, but also reduces the required network bandwidth. The invention also discloses a data synchronization device, data synchronization equipment and a computer readable storage medium among the nodes, which have the beneficial effects.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a method, a device, and a computer readable storage medium for data synchronization between nodes.
Background
With the rapid development of the internet, the appearance of mass data and the change of a data structure bring huge challenges to data processing problems of various industries. Synchronous processing of data is a very common and important data processing task involving the migration of data from one location to another. The data migration process of the data synchronization task is manually migrated at the beginning, which not only consumes manpower, but also has limited applicable places. Furthermore, a mode of data full-scale import by using an automation tool appears, for example, a specific folder is uploaded in batch by using a timed task to call an FTP (file transfer protocol), but along with the increase of data volume, the mode brings great pressure to network bandwidth, and has poor stability and higher requirements for service establishment.
Therefore, the concept of real-time incremental transmission arises, that is, whether incremental data exists at the source end of real-time monitoring data or not, and only incremental data is transmitted each time. However, this data synchronization method has a problem that when transmission fails, data difference occurs between the data source end and the data destination end. Therefore, the solution provided by the prior art is to combine real-time incremental transmission and timing full transmission, that is, after a period of real-time incremental transmission is performed, full transmission is performed again, so as to ensure the data consistency of the data source end and the data destination end.
It can be seen that, in the prior art, the accuracy of real-time incremental transmission cannot be guaranteed in terms of the problem of data synchronization, and the accuracy of data synchronization needs to be guaranteed by introducing full-volume transmission occupying a large bandwidth, and as the data volume increases, the network bandwidth still generates a large pressure, which causes waste of the network bandwidth.
Disclosure of Invention
The invention aims to provide a data synchronization method, a data synchronization device, data synchronization equipment and a computer readable storage medium among nodes, which are used for ensuring the accuracy of data synchronization among the nodes and reducing the required network bandwidth.
In order to solve the above technical problem, the present invention provides a method for synchronizing data between nodes, which is based on a monitoring node and includes:
carrying out real-time incremental synchronous deployment on each node in advance to realize real-time incremental synchronization among the nodes;
monitoring the transmission process of the incremental data between the nodes;
and when the incremental data transmission fails, recording the information of the incremental data transmission failure.
Optionally, the method further includes:
when the incremental data are monitored, storing the incremental data in a preset data space;
and controlling the incremental data which are failed to be transmitted in the preset data space to be transmitted again at preset time.
Optionally, the performing, in advance, real-time incremental synchronization deployment on each node to implement real-time incremental synchronization between each node specifically includes:
receiving a configuration of the real-time incremental synchronization deployment;
generating a configuration file according to the configuration and a preset configuration template;
and sending the configuration file to each node to carry out the real-time incremental synchronization deployment so as to realize the real-time incremental synchronization among the nodes.
Optionally, before sending the configuration file to each node for the real-time incremental synchronous deployment, the method further includes:
encrypting the configuration file to obtain an encrypted configuration file;
correspondingly, the sending the configuration file to each node to perform the real-time incremental synchronous deployment specifically includes:
and sending the encrypted configuration file to each node so that each node decrypts the encrypted configuration file and then executes the real-time incremental synchronous deployment.
Optionally, the configuration file includes an address of the monitoring node;
correspondingly, the real-time incremental synchronization among the nodes is realized, which specifically comprises the following steps:
when a real-time increment synchronous request sent by a first node is received, the address of a second node related to the increment data is sent to the first node, so that the first node sends the increment data to the second node according to the address of the second node.
Optionally, the monitoring the transmission process of the incremental data between the nodes specifically includes:
controlling each node to establish long connection with the monitoring node based on the address of the monitoring node in the configuration file;
and monitoring the transmission process of the incremental data through the long connection.
Optionally, the monitoring the transmission process of the incremental data through the long connection specifically includes:
receiving heartbeat signals of each node through the long connection;
and determining the transmission stage and the transmission result of the incremental data according to the heartbeat signal.
In order to solve the above technical problem, the present invention further provides an inter-node data synchronization apparatus, including:
the configuration unit is used for carrying out real-time incremental synchronous deployment on each node in advance so as to realize real-time incremental synchronization among the nodes;
the monitoring unit is used for monitoring the transmission process of the incremental data between the nodes;
and the recording unit is used for recording the information of the incremental data transmission failure when the incremental data transmission fails.
In order to solve the above technical problem, the present invention further provides an inter-node data synchronization apparatus, including:
a memory for storing instructions, said instructions comprising the steps of any of the above methods for data synchronization between said nodes;
a processor to execute the instructions.
To solve the above technical problem, the present invention further provides a computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the data synchronization method between the nodes according to any one of the above items.
The method for synchronizing the data among the nodes is realized based on the monitoring nodes, firstly, real-time incremental synchronization deployment is carried out on each node to realize the real-time incremental synchronization among the nodes, the minimum bandwidth occupation of the data synchronization process is realized, meanwhile, the transmission process of the incremental data among the nodes is monitored, and when the transmission of the incremental data fails, the information of the transmission failure of the incremental data is recorded, so that the method is convenient for monitoring the transmission failure of the incremental data by manpower or machines and carrying out subsequent processing on the incremental data, thereby ensuring the accuracy of the data synchronization among the nodes. Therefore, the method for synchronizing the data among the nodes provided by the invention not only ensures the accuracy of the data synchronization among the nodes, but also reduces the required network bandwidth. The invention also provides a data synchronization device, data synchronization equipment and a computer readable storage medium among nodes, which have the beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is an architecture diagram of a data transmission system between nodes according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for data synchronization between nodes according to an embodiment of the present invention;
fig. 3 is a flowchart of another method for data synchronization between nodes according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an inter-node data synchronization apparatus according to an embodiment of the present invention;
fig. 5 is a structural diagram of a data synchronization apparatus between nodes according to an embodiment of the present invention.
Detailed Description
The core of the invention is to provide a data synchronization method, a data synchronization device, data synchronization equipment and a computer readable storage medium among nodes, which are used for ensuring the accuracy of data synchronization among the nodes and reducing the required network bandwidth.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is an architecture diagram of a data transmission system between nodes according to an embodiment of the present invention.
For ease of understanding, a system architecture to which the present invention is applicable will first be described. The specific implementation provided by the embodiment of the invention can be applied to a system with multiple data nodes, such as a distributed file system. Each node may be the same type of device or different types of devices. And a node can be a source node (a node generating incremental data) or a destination node (a node receiving the incremental data), and the identity of the node can be converted in different types of data synchronization tasks. For the sake of understanding, the source node may also be defined as a client and the destination node as a server.
In the data transmission system between nodes provided in the embodiments of the present invention, each node in the file system is mounted on hardware, and one node is selected as a monitoring node to perform monitoring of a synchronization task between nodes. Of course, there may be a plurality of monitoring nodes, some of the monitoring nodes may be redundant nodes of monitoring tasks, and in some data synchronization tasks, the monitoring nodes may also be monitored nodes for performing data synchronization.
When the data synchronization task among the nodes is carried out, the monitoring nodes are used for realizing the real-time incremental synchronization deployment of all the nodes, realizing the real-time incremental synchronization among all the nodes, monitoring the transmission process of the incremental data among the nodes, and recording the incremental data with transmission failure for subsequent arrangement. To implement the above solution, a software architecture of a data transmission system between nodes according to an embodiment of the present invention is shown in fig. 1, where each node is provided with a synchronization module, each monitoring node is provided with a monitoring module 101, and after each node is initialized, the corresponding module is started. For a certain data transmission task, the first synchronization module 102 located at the source node solidifies the node as the client, and the second synchronization module 103 located at the destination node solidifies the node as the server. The first synchronization module 102 monitors data of the client, and when incremental data is monitored, the incremental data is sent to the second synchronization module 103 according to a server corresponding to a service of the incremental data, and the second synchronization module 103 synchronizes the incremental data to the server, so that active incremental synchronization is realized. The monitoring module 101 of the monitoring node communicates with each synchronization module to obtain the transmission stage and the transmission result of the incremental data. If the second synchronization module 103 does not receive the incremental data after a preset time elapses after the first synchronization module 102 sends the incremental data, it is determined that the transmission of the incremental data fails.
It is understood that besides the above nodes, other nodes in the system may also perform data synchronization according to the method, for example, a node serving as a server may serve as a client for the next data synchronization task to transmit incremental data to other nodes.
Based on the above framework, the data synchronization method between nodes provided by the embodiment of the present invention is further described below with reference to the accompanying drawings.
Fig. 2 is a flowchart of a method for synchronizing data between nodes according to an embodiment of the present invention.
As shown in fig. 2, based on the monitoring node, the method for synchronizing data between nodes according to the embodiment of the present invention includes:
s201: and carrying out real-time incremental synchronous deployment on each node in advance so as to realize real-time incremental synchronization among the nodes.
The real-time incremental synchronous deployment of each node can be automated deployment based on a configuration file, specifically, the configuration file is sent to each node through a monitoring node, or a technician installs the configuration file to each node. The configuration file may include, but is not limited to, information such as project name, module type, expiration date, gateway address, service address, etc. In order to increase the flexibility of configuration, a visual configuration interface may be provided for the user based on the monitoring node, and therefore, step S201 specifically includes:
receiving a configuration for a real-time incremental synchronous deployment;
generating a configuration file according to the configuration and a preset configuration template;
and sending the configuration file to each node to carry out real-time incremental synchronous deployment so as to realize real-time incremental synchronization among the nodes.
In order to improve the service security, part or all of the contents in the configuration file can be encrypted. Therefore, before sending the configuration file to each node for real-time incremental synchronous deployment, the method may further include:
encrypting the configuration file to obtain an encrypted configuration file;
correspondingly, the configuration file is sent to each node to perform real-time incremental synchronous deployment, specifically:
and sending the encrypted configuration file to each node so that each node decrypts the encrypted configuration file and then executes real-time incremental synchronous deployment.
In a specific implementation, the encryption mode can be symmetric encryption or asymmetric encryption. By adopting symmetric encryption, encryption failure and configuration file leakage can be caused in the transmission process of the encrypted configuration file, so that the method is suitable for technicians to copy and deploy the configuration file in the node. By adopting asymmetric encryption, a pair of public key and private key can be generated in advance, the public key is used for encrypting the configuration file, the monitoring node transmits the encrypted configuration file to the node, and the node decrypts the configuration file by using the private key.
And realizing automatic deployment of real-time incremental synchronization of each node by adopting the configuration file. Real-time incremental synchronization between nodes requires that the source node know the address of the destination node, which can be stored in a configuration file. In addition, in order to improve the safety of the data synchronization process, the monitoring node can inform the source node of the address of the destination node every time the real-time incremental synchronization is carried out. Specifically, the configuration file includes an address of the monitoring node;
correspondingly, in step S201, real-time incremental synchronization between the nodes is implemented, specifically:
and when a real-time incremental synchronization request sent by the first node is received, sending the address of the second node associated with the incremental data to the first node, so that the first node sends the incremental data to the second node according to the address of the second node.
In a data synchronization task, the first node is a source node, and the second node is a destination node.
S202: and monitoring the transmission process of the incremental data between the nodes.
The transmission process of the incremental data among the monitoring nodes can be monitored in various modes, and a better realization mode is to monitor through stable connection so as to avoid missing some information. Therefore, step S202 may specifically include:
controlling each node to establish long connection with the monitoring node based on the address of the monitoring node in the configuration file;
and monitoring the transmission process of incremental data through the long connection.
Further, monitoring the transmission process of the incremental data through the long connection specifically includes:
receiving heartbeat signals of each node through long connection;
and determining the transmission stage and the transmission result of the incremental data according to the heartbeat signal.
After a long connection mechanism is established between each node and the monitoring node, the state information of each node can be reported to the monitoring node through heartbeat signals. The monitoring node can obtain the transmission stage and the transmission result of the incremental data through the heartbeat signals of all the nodes.
S203: and when the incremental data transmission fails, recording the information of the incremental data transmission failure.
When the incremental data transmission fails, the monitoring node records information of the incremental data transmission failure, which may specifically include a source node, a destination node, a service to which the incremental data belongs, time when the incremental data is sent from the source node, time when the destination node does not receive the incremental data within a predetermined time, and the like, and may also include the incremental data itself.
The information of the failure of incremental data transmission can be pushed in real time or at regular time on a visual interface of the monitoring node so as to be used for subsequent processing by technical personnel.
The method for synchronizing data among nodes provided by the embodiment of the invention is realized based on the monitoring nodes, firstly, real-time incremental synchronization deployment is carried out on each node to realize the real-time incremental synchronization among the nodes, the minimization of the bandwidth occupied by the data synchronization process is realized, meanwhile, the transmission process of the incremental data among the nodes is monitored, and when the transmission of the incremental data fails, the information of the transmission failure of the incremental data is recorded, so that the method is convenient for monitoring the transmission failure of the incremental data by manpower or machines and carrying out subsequent processing on the incremental data, thereby ensuring the accuracy of the data synchronization among the nodes. It can be seen that the method for data synchronization between nodes provided by the embodiment of the present invention not only ensures the accuracy of data synchronization between nodes, but also reduces the required network bandwidth.
Fig. 3 is a flowchart of another method for synchronizing data between nodes according to an embodiment of the present invention.
On the basis of the above embodiment, in another embodiment, the method for synchronizing data between nodes further includes:
s301: and when the incremental data are monitored, storing the incremental data in a preset data space.
S302: and controlling the incremental data which are failed to be transmitted in the preset data space to be transmitted again at the preset time.
It should be noted that, since the real-time incremental synchronization process between the nodes can be performed synchronously, steps S301 and S302 have no sequential relationship with steps S201 to S203, and for one data synchronization task, step S301 is located between step S202 and step S203, and step S302 is located after step S203.
And on the monitoring node, a preset data space is divided in advance to serve as a transfer station for storing incremental data. When the incremental data transmission fails, the synchronization can be controlled to be performed again in a system idle time.
In a specific implementation, the preset time for retransmitting the incremental data may be around 0: 00. For the increment data which is successfully transmitted, the increment data can be deleted from the preset data space, and unnecessary space occupation is reduced. Before the incremental data which fails to be transmitted is transmitted again, if the incremental data which conflicts with the incremental data which fails to be transmitted appears, the incremental data and the incremental data are judged according to a preset judgment mechanism, or the incremental data which fails to be transmitted is not transmitted again in preset time, and the processing of technicians is waited.
On the basis of the above detailed description of the embodiments corresponding to the method for data synchronization between nodes, the present invention also discloses a device, an apparatus and a computer readable storage medium for data synchronization between nodes corresponding to the above method.
Fig. 4 is a schematic structural diagram of an inter-node data synchronization apparatus according to an embodiment of the present invention.
As shown in fig. 4, the apparatus for synchronizing data between nodes according to the embodiment of the present invention includes:
a configuration unit 401, configured to perform real-time incremental synchronization deployment on each node in advance, so as to implement real-time incremental synchronization between each node;
a monitoring unit 402, configured to monitor a transmission process of incremental data between nodes;
a recording unit 403, configured to record information of the incremental data transmission failure when the incremental data transmission fails.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
Further, the embodiment of the invention also provides a device for synchronizing data between nodes.
Fig. 5 is a structural diagram of a data synchronization apparatus between nodes according to an embodiment of the present invention.
As shown in fig. 5, the data synchronization apparatus provided in the embodiment of the present invention includes:
a memory 510 for storing instructions comprising the steps of the data synchronization method according to any of the above embodiments;
a processor 520 for executing the instructions.
Among other things, processor 520 may include one or more processing cores, such as a 3-core processor, an 8-core processor, and so on. The processor 520 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). Processor 520 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also referred to as a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 520 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 520 may further include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning.
Memory 510 may include one or more computer-readable storage media, which may be non-transitory. Memory 510 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 510 is at least used for storing a computer program 511, wherein after the computer program 511 is loaded and executed by the processor 520, the relevant steps in the method for synchronizing data between nodes disclosed in any of the foregoing embodiments can be implemented. In addition, the resources stored in the memory 510 may also include an operating system 512, data 513, and the like, and the storage manner may be a transient storage or a permanent storage. The operating system 512 may be Windows, among others. Data 513 may include, but is not limited to, data involved with the above-described methods.
In some embodiments, the data synchronization device between nodes may further comprise a display screen 530, a power source 540, a communication interface 550, an input/output interface 560, a sensor 570, and a communication bus 580.
Those skilled in the art will appreciate that the architecture shown in FIG. 5 does not constitute a limitation of data synchronization devices between nodes and may include more or fewer components than those shown.
The inter-node data synchronization device provided by the embodiment of the application comprises a memory and a processor, and when the processor executes a program stored in the memory, the inter-node data synchronization method can be realized, and the effect is the same as that of the inter-node data synchronization method.
It should be noted that the above-described embodiments of the apparatus and device are merely illustrative, for example, the division of modules is only one division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form. Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and performs all or part of the steps of the methods according to the embodiments of the present invention, or all or part of the technical solution.
To this end, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the data synchronization method between nodes.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The computer program contained in the computer-readable storage medium provided in this embodiment can implement the steps of the method for synchronizing data between nodes as described above when executed by the processor, and the effect is the same as above.
The present invention provides a method, an apparatus, a device and a computer readable storage medium for data synchronization between nodes. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device, the apparatus and the computer-readable storage medium disclosed in the embodiments correspond to the method disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A method for synchronizing data between nodes is characterized in that based on a monitoring node, the method comprises the following steps:
carrying out real-time incremental synchronous deployment on each node in advance to realize real-time incremental synchronization among the nodes;
monitoring the transmission process of the incremental data between the nodes;
and when the incremental data transmission fails, recording the information of the incremental data transmission failure.
2. The data synchronization method of claim 1, further comprising:
when the incremental data are monitored, storing the incremental data in a preset data space;
and controlling the incremental data which are failed to be transmitted in the preset data space to be transmitted again at preset time.
3. The data synchronization method according to claim 1, wherein the performing, in advance, real-time incremental synchronization deployment on each node to achieve real-time incremental synchronization between each node specifically includes:
receiving a configuration of the real-time incremental synchronization deployment;
generating a configuration file according to the configuration and a preset configuration template;
and sending the configuration file to each node to carry out the real-time incremental synchronization deployment so as to realize the real-time incremental synchronization among the nodes.
4. The data synchronization method according to claim 3, further comprising, before the sending the configuration file to each of the nodes for the real-time incremental synchronization deployment:
encrypting the configuration file to obtain an encrypted configuration file;
correspondingly, the sending the configuration file to each node to perform the real-time incremental synchronous deployment specifically includes:
and sending the encrypted configuration file to each node so that each node decrypts the encrypted configuration file and then executes the real-time incremental synchronous deployment.
5. The data synchronization method according to claim 3, wherein the configuration file includes an address of the monitoring node;
correspondingly, the real-time incremental synchronization among the nodes is realized, which specifically comprises the following steps:
when a real-time increment synchronous request sent by a first node is received, the address of a second node related to the increment data is sent to the first node, so that the first node sends the increment data to the second node according to the address of the second node.
6. The data synchronization method according to claim 5, wherein the monitoring the transmission process of the incremental data between the nodes specifically includes:
controlling each node to establish long connection with the monitoring node based on the address of the monitoring node in the configuration file;
and monitoring the transmission process of the incremental data through the long connection.
7. The data synchronization method according to claim 6, wherein the monitoring the transmission process of the incremental data through the long connection specifically includes:
receiving heartbeat signals of each node through the long connection;
and determining the transmission stage and the transmission result of the incremental data according to the heartbeat signal.
8. An apparatus for synchronizing data between nodes, comprising:
the configuration unit is used for carrying out real-time incremental synchronous deployment on each node in advance so as to realize real-time incremental synchronization among the nodes;
the monitoring unit is used for monitoring the transmission process of the incremental data between the nodes;
and the recording unit is used for recording the information of the incremental data transmission failure when the incremental data transmission fails.
9. An apparatus for synchronizing data between nodes, comprising:
a memory for storing instructions comprising the steps of the method for synchronizing data between nodes according to any one of claims 1 to 7;
a processor to execute the instructions.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for data synchronization between nodes according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911053648.4A CN110941665A (en) | 2019-10-31 | 2019-10-31 | Data synchronization method, data synchronization device and data synchronization equipment between nodes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911053648.4A CN110941665A (en) | 2019-10-31 | 2019-10-31 | Data synchronization method, data synchronization device and data synchronization equipment between nodes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110941665A true CN110941665A (en) | 2020-03-31 |
Family
ID=69906433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911053648.4A Withdrawn CN110941665A (en) | 2019-10-31 | 2019-10-31 | Data synchronization method, data synchronization device and data synchronization equipment between nodes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110941665A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111831749A (en) * | 2020-07-07 | 2020-10-27 | 深圳市欢太科技有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
| CN112398905A (en) * | 2020-09-28 | 2021-02-23 | 联想(北京)有限公司 | Node and information synchronization method |
| CN112685501A (en) * | 2020-12-30 | 2021-04-20 | 杭州雾联科技有限公司 | Data synchronization method, device and system for center node and edge node |
| CN113507497A (en) * | 2021-06-01 | 2021-10-15 | 常州皓鸣信息科技有限公司 | Multi-node asynchronous issuing and delay integration method for multi-type data |
| WO2023041073A1 (en) * | 2021-09-19 | 2023-03-23 | 苏州浪潮智能科技有限公司 | Method for data synchronisation between multiple nodes, and system, device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160294767A1 (en) * | 2015-03-31 | 2016-10-06 | Willie L. Donaldson | Secure dynamic address resolution and communication system, method, and device |
| CN106294713A (en) * | 2016-08-09 | 2017-01-04 | 深圳中兴网信科技有限公司 | The method of data synchronization resolved based on Incremental Log and data synchronization unit |
| CN107016014A (en) * | 2016-09-30 | 2017-08-04 | 阿里巴巴集团控股有限公司 | The method of data synchronization and device of heterogeneous database |
| CN108228397A (en) * | 2016-12-22 | 2018-06-29 | 深圳市优朋普乐传媒发展有限公司 | The method and apparatus that a kind of cluster span computer room synchronizes |
| CN109299193A (en) * | 2018-09-25 | 2019-02-01 | 深圳市佰仟金融服务有限公司 | Method of data synchronization and relevant device |
-
2019
- 2019-10-31 CN CN201911053648.4A patent/CN110941665A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160294767A1 (en) * | 2015-03-31 | 2016-10-06 | Willie L. Donaldson | Secure dynamic address resolution and communication system, method, and device |
| CN106294713A (en) * | 2016-08-09 | 2017-01-04 | 深圳中兴网信科技有限公司 | The method of data synchronization resolved based on Incremental Log and data synchronization unit |
| CN107016014A (en) * | 2016-09-30 | 2017-08-04 | 阿里巴巴集团控股有限公司 | The method of data synchronization and device of heterogeneous database |
| CN108228397A (en) * | 2016-12-22 | 2018-06-29 | 深圳市优朋普乐传媒发展有限公司 | The method and apparatus that a kind of cluster span computer room synchronizes |
| CN109299193A (en) * | 2018-09-25 | 2019-02-01 | 深圳市佰仟金融服务有限公司 | Method of data synchronization and relevant device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111831749A (en) * | 2020-07-07 | 2020-10-27 | 深圳市欢太科技有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
| CN111831749B (en) * | 2020-07-07 | 2024-03-12 | 深圳市欢太科技有限公司 | Data processing method and device, storage medium and electronic equipment |
| CN112398905A (en) * | 2020-09-28 | 2021-02-23 | 联想(北京)有限公司 | Node and information synchronization method |
| CN112685501A (en) * | 2020-12-30 | 2021-04-20 | 杭州雾联科技有限公司 | Data synchronization method, device and system for center node and edge node |
| CN113507497A (en) * | 2021-06-01 | 2021-10-15 | 常州皓鸣信息科技有限公司 | Multi-node asynchronous issuing and delay integration method for multi-type data |
| WO2023041073A1 (en) * | 2021-09-19 | 2023-03-23 | 苏州浪潮智能科技有限公司 | Method for data synchronisation between multiple nodes, and system, device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110941665A (en) | Data synchronization method, data synchronization device and data synchronization equipment between nodes | |
| CN106331098B (en) | Server cluster system | |
| WO2021203979A1 (en) | Operation and maintenance processing method and apparatus, and computer device | |
| CN110765744A (en) | Multi-person collaborative document editing method and system | |
| CN111625497B (en) | Deployment method, device and equipment of distributed file system and storage medium | |
| CN109769001A (en) | A kind of internet of things data transmission method and system | |
| CN105574109A (en) | Database synchronizing method and equipment and system | |
| CN111966465B (en) | Method, system, equipment and medium for modifying host configuration parameters in real time | |
| CN111338893A (en) | Process log processing method and device, computer equipment and storage medium | |
| CN111314416B (en) | Data synchronization method, device, system, equipment and storage medium | |
| CN109788251B (en) | Video processing method, device and storage medium | |
| CN115357662A (en) | Data synchronization method, device and medium | |
| CN114285695B (en) | Communication method, device, apparatus, system and storage medium | |
| CN111881209A (en) | Data synchronization method and device for heterogeneous database, electronic equipment and medium | |
| CN103327060A (en) | Information processing apparatus and information processing method | |
| CN108093077B (en) | Abnormality eliminating method, device and server | |
| CN116599589B (en) | Signal synchronization method, device, equipment and medium | |
| CN112243033A (en) | Method, device and equipment for acquiring memory information and computer readable storage medium | |
| CN107609197A (en) | A kind of method of data synchronization, data synchronization unit and mobile terminal | |
| CN109885258B (en) | Data deleting device and method based on block chain | |
| CN115509563B (en) | Mirror image file manufacturing method, device, equipment and medium | |
| CN110928707A (en) | Data exchange method and related device for host machine and virtual machine | |
| CN105871659A (en) | Monitoring method and device of newly added server | |
| CN113031569B (en) | Monitoring method, system, terminal and medium of nuclear fusion control system | |
| CN110288309B (en) | Data interaction method, device, system, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200331 |