CN110750594A - Mysql-based real-time cross-network database synchronization method for incremental logs - Google Patents
Mysql-based real-time cross-network database synchronization method for incremental logs Download PDFInfo
- Publication number
- CN110750594A CN110750594A CN201910938282.2A CN201910938282A CN110750594A CN 110750594 A CN110750594 A CN 110750594A CN 201910938282 A CN201910938282 A CN 201910938282A CN 110750594 A CN110750594 A CN 110750594A
- Authority
- CN
- China
- Prior art keywords
- data
- mysql
- database
- incremental
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention relates to a mysql-based real-time cross-network database synchronization method for incremental logs, which comprises the following steps: analyzing the incremental log generated by the mysql, and serially persisting the analyzed incremental log into a database as an event; a server side of a machine room where a database server is located transmits the persistent data to a client side in a remote machine room in a socket long connection mode; the client side sequentially plays back the received persistent data and writes the persistent data into a local database, and synchronization of the databases across the network is achieved. The method can solve the fundamental problem of mysql public network synchronization, thereby realizing data foundation for the target of multiple data centers.
Description
Technical Field
The invention relates to the technical field of information, in particular to a mysql-based incremental log real-time cross-network database synchronization method.
Background
At present, with the continuous development of internet technology, for an internet application, the bearing capacity of the internet application is continuously updated and improved along with the increase of the user quantity. Correspondingly, the method gradually expands the application from the original single server application to the application of multiple servers in a single machine room and even to the application of multiple machine rooms and multiple data centers. Based on such changes, the synchronization of databases across computer rooms is becoming one of the important basic links to be considered. As a model for an open source database, it is important to solve the cross-machine room synchronization of mysql. The traditional mysql master-slave synchronization is only suitable for a single network with stable and fluctuation-free network environment, and cannot be used when facing public network transmission with complex network environment.
Currently, synchronization of mysql itself has the following problems in a cross-network scenario:
1. problems caused by network delay instability: since the local area network is separated, network transmission and network delay and instability on the public network inevitably exist, and synchronization of mysql is not optimized aiming at a cross-network scene, various faults can be caused.
2. The problem caused by the inconsistent equipment load of the master-slave synchronization is as follows: due to different loads of the master device or the slave device, mutual influence can be caused.
3. Synchronization failure problems caused by various conflicts: in the master-slave synchronization, if the slave database is changed, the mysql synchronization mechanism is a strong consistency principle, and therefore, the master-slave failure can be caused immediately.
4. Problem with synchronization strategy: there are conventionally 3 strategies for master-slave synchronization, (1) synchronous replication: transactionally handling synchronization, i.e. any transaction performed by the master library, involves the step of synchronizing all slave node data. The method has the advantages that the data synchronization consistency can be ensured, and the defects that the master-slave db performance is sharply reduced when the db load is large are overcome. (2) Asynchronous replication: the master library asynchronously synchronizes data to the slave libraries regardless of whether the slave libraries receive and process data. The advantage is that synchronization anomalies do not affect the master-slave db performance. A disadvantage is that when the master library fails, there may be situations where the data is not synchronized successfully. (3) Semi-synchronous replication: similar to synchronous replication, the difference is that any transaction executed by the master library contains at least one step of receiving binlog from the slave node (the slave node does not need to wait until execution is complete and then feeds back to the master node). The method has the advantages of giving consideration to the performance of asynchronous replication and the accuracy of synchronous replication to a certain extent. The disadvantage is that the performance is not completely asynchronous and therefore it will still load db. In terms of accuracy, the master cannot know the actual execution condition of the slave node, so that the slave node still fails to execute.
5. Problems with synchronization efficiency and synchronization delay: the replay strategy adopted by the slave node is single-thread sequential operation, and the method has the advantages of ensuring the sequentiality of data replay and ensuring the data synchronization consistency, but has the defects of insufficient synchronization efficiency and certain synchronization delay caused by single-thread processing.
Disclosure of Invention
The invention provides a real-time cross-network database synchronization method based on mysql incremental logs, which can solve the fundamental problem of mysql public network synchronization, thereby realizing data foundation for the target of multiple data centers.
The technical scheme adopted by the invention for solving the technical problems is as follows: the real-time cross-network database synchronization method based on the mysql incremental log comprises the following steps:
(1) analyzing the incremental log generated by the mysql, and serially persisting the analyzed incremental log into a database as an event;
(2) a server side of a machine room where a database server is located transmits the persistent data to a client side in a remote machine room in a socket long connection mode;
(3) the client side sequentially plays back the received persistent data and writes the persistent data into a local database, and synchronization of the databases across the network is achieved.
The step (1) is specifically as follows: analyzing the incremental logs in each database in batches, then persisting the analysis results into a table in batches, after the persistence is successful, saving the position of the current persisted incremental logs, and subsequently, if the program is restarted, starting processing by taking the position as an analysis point.
The fields of the table include: the method comprises the steps of incremental log identification, incremental log position, database name, type, analyzed original data and updating time.
If any abnormal condition occurs in the persistence process, repeated retry is carried out, and if the number of times of the abnormal condition reaches a retry threshold value, an alarm is given.
The step (2) is specifically as follows: loading ip/port configuration of remote clients, starting a socket long connection for each client, loading the persistent data in batches in an infinite loop manner, and transmitting the persistent data to the corresponding clients through the socket long connection; and after the transmission is successful, writing the current synchronous increment log position of the client into a data table for the initial loading position during each starting.
The data table field includes: id. Client identification, processing incremental log location, and update time.
The socket long connection realizes disconnection reconnection processing by monitoring whether the connection is effective or not, and realizes packet sticking and unpacking processing by detecting the head and the tail of a protocol; when the persistent data is transmitted, serialization processing, encryption processing and compression processing are carried out on the transmitted data.
And (3) the client establishes a socket monitor for receiving the connection of the server and judging whether the request ip of the connection is in a white list, if so, receiving the data transmitted by the server and decoding the data, otherwise, rejecting the data.
The step (3) is specifically as follows: the client-side carries out data grouping on each database, converts the persistent data related to each database into sql, stores the sql in a list in sequence, and stores the sql in each database object; importing playback work for each database processing, waiting for all threads to finish execution through a future mechanism, and feeding back a processing result to the server; when the processing process is imported, the server side synchronizes the change to the client side in real time after the structure of the pre-synchronization database changes; during the import processing, batch import is implemented by adopting a batch mechanism of jdbc, and if the batch import fails, the import operation is executed one by one.
And (4) after the step (3), retrying each batch synchronization by adopting a failover mode and playing back based on the sequence of the incremental logs.
Advantageous effects
Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects: the incremental logs generated by mysql are analyzed and persisted, then transmitted in a public network through a socket, sent to a cross-machine-room client, and played back and stored in a warehouse through a processing mechanism, and finally the function of database synchronization is realized.
Drawings
Fig. 1 is an application architecture diagram of the present invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
The embodiment of the invention relates to a mysql-based incremental log real-time cross-network database synchronization method, the application architecture of which is shown in fig. 1, and the method adopts a high-availability strategy (a server is based on zk, and a client is based on nginx) to ensure that no single point risk exists, and comprises the following steps: analyzing the incremental log generated by the mysql, and serially persisting the analyzed incremental log into a database as an event; a server side of a machine room where a database server is located transmits the persistent data to a client side in a remote machine room in a socket long connection mode; the client side sequentially plays back the received persistent data and writes the persistent data into a local database, and synchronization of the databases across the network is achieved.
In the server program, there is an infinite loop program, after the mysql log is analyzed, the mysql log is used as an event to be persisted in db in serial for subsequent positioning and synchronization, and the method comprises the following steps: the batches parse binlog in each db, and then persist the parsed results into a specific table. Table field: the method comprises the steps of obtaining binlog identification, a binlog position, a db name, a type (dml (data structure change)/ddl (data structure change)), and original data obtained through analysis, and updating time.
After the server program is started, the method comprises the following steps: and loading ip/port configuration of the remote client, and starting a socket long connection for each client. Data are transmitted through the long connection, transmission performance is improved, and resource consumption caused by repeatedly opening and closing the connection is reduced. The socket long connection is processed aiming at the conditions of disconnection, reconnection, package sticking, unpacking and the like. The disconnection reconnection handling mechanism is implemented based on monitoring whether the connection is valid (heartbeat). The sticky packet unpacking processing mechanism is realized by detecting the head and the tail of a protocol based on a self-defined transmission protocol. In order to facilitate binary transmission, the transmitted data is serialized in a Protostuff mechanism. In order to ensure the transmission security, the AES encryption is carried out on the transmission data, and the specific mode is as follows: AES/CBC/PKCS5 Padding. In order to reduce the size of the transmission data, the transmission data is compressed by adopting a Snappy mechanism. And respectively starting a thread for each client, loading the persistent event log data in batches in an infinite loop manner, and transmitting the persistent event log data to the corresponding client through a socket. When an exception occurs, it will retry indefinitely until it is successful. After success, the binlog position currently synchronized by the client is written into the data table for the position initially loaded at each start. Data table field definition: id, client identification, binlog position processing, and update time.
After the client program is started, the method comprises the following steps: after starting, a socket monitor is established for receiving the connection of the server. And judging whether the connected request ip is in a white list or not, and if not, rejecting the request ip. And carrying out decoding check on the received binary data. And decoding the received batch synchronous data.
The method for realizing the database synchronization by playing back and warehousing the received synchronous data comprises the following steps: because each db needs to realize data playback, the execution sequence of the sql must be completely executed according to the original sequence, so that the basic requirements of performance and playback are considered, data grouping is performed on each db, the relevant persistent data of the db is converted into the sql, and the sql is stored in a list in sequence and is stored in each db object. At this point, one thread can be started for each db to process the import playback job. And then, after all the threads are executed completely through a future mechanism, feeding back a processing result to the server. The client performs import preprocessing on the received sql data to enable the sql data to meet the execution requirement. For sql preprocessing, the attributes of certain fields need to be known, such as whether the key is primary, whether the field is time, etc. These are synchronized in real time to the client by the server after the change in the pre-synchronization db structure. In consideration of import performance, batch import is implemented by adopting a batch mechanism of jdbc. And if the batch import fails, executing the import operation one by one. The import by import ignores conflicts to import (the probability of successful import is higher than that of batch import), and besides fails, the whole task fails. Aiming at the scenes that the dml execution speed is high and the ddl execution speed is possibly low, different lead-in timeout times are set, and balance is achieved on the two points that lead-in is successful and problems are found in time.
In the embodiment, a mechanism is required to ensure that the data in the database is subjected to binlog analysis, persistence, sending by the server, receiving by the client and warehousing by the client, so that the sequence of data synchronization is ensured, and the consistency of data synchronization is ensured. Therefore, retry processing is performed on each batch synchronization in a failover mode, namely, a failure in batch synchronization is considered to be a failure. Playback is performed based on binlog order, ensuring data consistency. And (4) neglecting treatment is carried out aiming at partial conflict, so that the synchronization failure probability is reduced.
Aiming at the problem of mysql synchronization in the prior art, the implementation method adopts the following measures to process:
1. the problem caused by unstable network delay is solved.
The implementation mode is optimized for cross-network transmission, can ensure high efficiency and stability of transmission, and can ensure that the accuracy and the orderliness of data synchronization are not influenced even if network fluctuation exists through a series of strategies.
A. And the long tpc connection transmission is adopted, so that data synchronization and feedback are implemented on a single channel, and a series of problems caused by multiple handshaking are prevented.
B. Aiming at the conditions of unstable network, instantaneous interruption and the like, a heartbeat mechanism is adopted. And if no data input and output exist within 30 seconds, a heartbeat packet is initiated through a thread detection, and if feedback is received within the specified time. The connection is considered normal. If no separation is received, the connection is considered abnormal. At this time, reconnection is initiated, and the data transmission tcp connection is reinitialized.
C. Aiming at the scenes of restart, exception, timeout and the like, after each node is successfully transmitted, the last successfully synchronized data event identifier of each node is persisted. This is used as the initial value for the synchronous data event. And data are ensured not to be transmitted in an omission way under any condition, and the final consistency of the synchronous data of the database is ensured.
2. The problem caused by inconsistent equipment loads of master-slave synchronization is solved.
A. The server side adopts a mode of asynchronously analyzing the binlog log of the main node. A queue despeckling synchronization approach is then used. That is, only a quantitative batch of data is synchronized for a single client at any one time. The method can ensure that the synchronous amount of the server is always within a controllable range, and can not cause burden on the main node, thereby influencing the external service quality.
B. The client side adopts a single db single-thread data playback mode, and the data playback of the client side is enabled not to cause burden on the slave nodes based on the controllable synchronous data volume of the server side, so that the external service quality is influenced.
3. And solving the problem of synchronization failure caused by various conflicts.
A. The avoidance processing is carried out on special conflict scenes such as main key conflict, column inserting conflict, key inserting conflict, column deleting/key deleting lack, table existing during table building and the like during data inserting.
B. Therefore, the avoidance processing is adopted for the conflict scene. This is because the collision generated in the above operation is negligible from a business point of view. No forced verification is required.
4. The problem of synchronization strategy is solved.
A. The strategy adopted by the embodiment is different from the existing synchronous replication, asynchronous replication and semi-synchronous replication. The method can ensure that in the data synchronization process, the master node and the slave node are not subjected to too large load (the data volume of batch execution can be adjusted at any time), and ensure that external services are normal. In addition, the final consistency of the data can be ensured, and after the slave nodes are successfully synchronized, the synchronization position is fed back to the master node for persistence. The master node takes the result as the initial basis of synchronization.
B. Firstly, the service end asynchronously analyzes the binlog of the main node and then discards the binlog into a queue. The client-side customer sends the data to the client-side in batches through tcp long connection. And after receiving the data to be synchronized, the client parses the data into executable sql, and then stores the sql in bulk. And feeding back the data synchronization result of the batch to the server. If the data synchronization is successful, the server will take the data of the next batch for synchronization, and the data synchronization will be completed until the synchronization is successful. If the failure occurs, the server will retry indefinitely, and the alarm will be given if the error reaches the threshold value. In the whole process, the master node and the slave node normally provide services to the outside, and synchronization only occupies a small part of resources.
5. The problems of synchronization efficiency and synchronization delay are solved.
A. Binlog is converted into sql that is actually executed.
B. Wherein insert is divided into 2 cases, if the main key field is included, replace intro is adopted to execute, thereby preventing the subsequent main key conflict. Otherwise, directly executing with insert intos.
C. Wherein update is divided into 2 cases, if the field of the primary key is contained, the primary key is used as the query condition to update. Otherwise, adopting the full field as the query condition. The use of the former can accelerate the execution efficiency of update without full field matching.
D. Where delete is divided into 2 cases, if the field contains the primary key, the primary key is used as the query condition for delete. Otherwise, adopting the full field as the query condition. The use of the former can speed up delete execution efficiency without full field matching.
E. And (3) data is played back and put in storage, and a way of executing sql by batch is adopted, so that the execution efficiency can be obviously improved.
F. In addition to the above-described dml operations, ddl operations also encompass those in which changes to the structure of the data table are synchronized together. The difference is that ddl operation sets a timeout greater than dml operation. The rest of the processing mechanisms are consistent.
G. The synchronization is carried out without depending on the transaction consistency, and the synchronization efficiency is improved.
H. During synchronization, batch synchronous data are grouped according to db, and multithreading batch playback and warehousing are performed on each db, so that the synchronization efficiency is improved.
The incremental logs generated by mysql are analyzed and persisted, transmitted in a public network through a socket and sent to a cross-machine-room client, and data are played back and put in a warehouse through a processing mechanism, so that the function of database synchronization is finally realized, the problems of instability and insecurity of mysql master-slave synchronization in public network transmission are solved, and the establishment of a cross-machine-room multi-data center is possible.
Claims (10)
1. A mysql-based incremental log real-time cross-network database synchronization method is characterized by comprising the following steps:
(1) analyzing the incremental log generated by the mysql, and serially persisting the analyzed incremental log into a database as an event;
(2) a server side of a machine room where a database server is located transmits the persistent data to a client side in a remote machine room in a socket long connection mode;
(3) the client side sequentially plays back the received persistent data and writes the persistent data into a local database, and synchronization of the databases across the network is achieved.
2. The mysql-based incremental log real-time cross-network database synchronization method according to claim 1, wherein the step (1) is specifically as follows: analyzing the incremental logs in each database in batches, then persisting the analysis results into a table in batches, after the persistence is successful, saving the position of the current persisted incremental logs, and subsequently, if the program is restarted, starting processing by taking the position as an analysis point.
3. The mysql-based incremental log real-time cross-network database synchronization method of claim 2, wherein the fields of the table comprise: the method comprises the steps of incremental log identification, incremental log position, database name, type, analyzed original data and updating time.
4. The real-time cross-network database synchronization method based on mysql incremental logs as claimed in claim 2, wherein if any abnormal condition occurs during the persistence process, the retry is repeated, and if the number of times of the abnormal condition reaches a retry threshold value, an alarm is given.
5. The mysql-based incremental log real-time cross-network database synchronization method according to claim 1, wherein the step (2) is specifically as follows: loading ip/port configuration of remote clients, starting a socket long connection for each client, loading the persistent data in batches in an infinite loop manner, and transmitting the persistent data to the corresponding clients through the socket long connection; and after the transmission is successful, writing the current synchronous increment log position of the client into a data table for the initial loading position during each starting.
6. The mysql-based incremental log real-time cross-network database synchronization method of claim 5, wherein the data table field comprises: id. Client identification, processing incremental log location, and update time.
7. The real-time cross-network database synchronization method based on mysql incremental logs is characterized in that the socket long connection realizes disconnection and reconnection processing by monitoring whether the connection is valid or not, and realizes sticky packet unpacking processing by detecting the head and the tail of a protocol; when the persistent data is transmitted, serialization processing, encryption processing and compression processing are carried out on the transmitted data.
8. The real-time cross-network database synchronization method based on mysql incremental logs as claimed in claim 1, wherein the client in step (3) establishes a socket monitor for receiving the connection of the server and determining whether the connected request ip is in a white list, if so, receiving the data transmitted by the server and decoding the data, otherwise, rejecting the data.
9. The mysql-based incremental log real-time cross-network database synchronization method according to claim 1, wherein the step (3) is specifically as follows: the client-side carries out data grouping on each database, converts the persistent data related to each database into sql, stores the sql in a list in sequence, and stores the sql in each database object; importing playback work for each database processing, waiting for all threads to finish execution through a future mechanism, and feeding back a processing result to the server; when the processing process is imported, the server side synchronizes the change to the client side in real time after the structure of the pre-synchronization database changes; during the import processing, batch import is implemented by adopting a batch mechanism of jdbc, and if the batch import fails, the import operation is executed one by one.
10. The real-time cross-network database synchronization method based on mysql incremental logs is characterized in that the step (3) is followed by a retry processing step for each batch synchronization in a failover mode and a playback step based on the incremental log sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910938282.2A CN110750594B (en) | 2019-09-30 | 2019-09-30 | Real-time cross-network database synchronization method based on mysql incremental log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910938282.2A CN110750594B (en) | 2019-09-30 | 2019-09-30 | Real-time cross-network database synchronization method based on mysql incremental log |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110750594A true CN110750594A (en) | 2020-02-04 |
| CN110750594B CN110750594B (en) | 2023-05-30 |
Family
ID=69277516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910938282.2A Active CN110750594B (en) | 2019-09-30 | 2019-09-30 | Real-time cross-network database synchronization method based on mysql incremental log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110750594B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182095A (en) * | 2020-08-31 | 2021-01-05 | 福州智象信息技术有限公司 | Method, device, equipment and medium for synchronizing service data among multiple machine rooms |
| CN112527903A (en) * | 2020-12-11 | 2021-03-19 | 北京首汽智行科技有限公司 | Data synchronization method based on canal platform |
| CN116155920A (en) * | 2023-02-16 | 2023-05-23 | 北京万里开源软件有限公司 | Remote transmission method and system for MySQL protocol database data |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377100A (en) * | 2012-04-26 | 2013-10-30 | 华为技术有限公司 | Data backup method, network nodes and system |
| CN103793749A (en) * | 2014-02-27 | 2014-05-14 | 四川物联恒安安全技术研究有限公司 | Safety production comprehensive supervisory platform data transmission method |
| CN104281506A (en) * | 2014-07-10 | 2015-01-14 | 中国科学院计算技术研究所 | Data maintenance method and system for file system |
| CN104679841A (en) * | 2015-02-11 | 2015-06-03 | 北京京东尚科信息技术有限公司 | Consumption terminal data flow copying method and system |
| WO2015106656A1 (en) * | 2014-01-20 | 2015-07-23 | 浪潮电子信息产业股份有限公司 | Cross-data-center data synchronization method |
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| US20160294605A1 (en) * | 2014-07-07 | 2016-10-06 | Symphony Teleca Corporation | Remote Embedded Device Update Platform Apparatuses, Methods and Systems |
| CN107145403A (en) * | 2017-04-20 | 2017-09-08 | 浙江工业大学 | The relevant database data retrogressive method of web oriented development environment |
| US20180143884A1 (en) * | 2016-11-22 | 2018-05-24 | Sap Se | Database system recovery using preliminary and final slave node replay positions |
| CN109189860A (en) * | 2018-10-19 | 2019-01-11 | 山东浪潮云信息技术有限公司 | A kind of active and standby increment synchronization method of MySQL based on Kubernetes system |
-
2019
- 2019-09-30 CN CN201910938282.2A patent/CN110750594B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377100A (en) * | 2012-04-26 | 2013-10-30 | 华为技术有限公司 | Data backup method, network nodes and system |
| WO2015106656A1 (en) * | 2014-01-20 | 2015-07-23 | 浪潮电子信息产业股份有限公司 | Cross-data-center data synchronization method |
| CN103793749A (en) * | 2014-02-27 | 2014-05-14 | 四川物联恒安安全技术研究有限公司 | Safety production comprehensive supervisory platform data transmission method |
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| US20160294605A1 (en) * | 2014-07-07 | 2016-10-06 | Symphony Teleca Corporation | Remote Embedded Device Update Platform Apparatuses, Methods and Systems |
| CN104281506A (en) * | 2014-07-10 | 2015-01-14 | 中国科学院计算技术研究所 | Data maintenance method and system for file system |
| CN104679841A (en) * | 2015-02-11 | 2015-06-03 | 北京京东尚科信息技术有限公司 | Consumption terminal data flow copying method and system |
| US20180143884A1 (en) * | 2016-11-22 | 2018-05-24 | Sap Se | Database system recovery using preliminary and final slave node replay positions |
| CN107145403A (en) * | 2017-04-20 | 2017-09-08 | 浙江工业大学 | The relevant database data retrogressive method of web oriented development environment |
| CN109189860A (en) * | 2018-10-19 | 2019-01-11 | 山东浪潮云信息技术有限公司 | A kind of active and standby increment synchronization method of MySQL based on Kubernetes system |
Non-Patent Citations (2)
| Title |
|---|
| 何小川;: "MySQL数据库主从复制的实现" * |
| 倪泳智;王珊;: "一种基于虚拟日志的数据复制解决方案" * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182095A (en) * | 2020-08-31 | 2021-01-05 | 福州智象信息技术有限公司 | Method, device, equipment and medium for synchronizing service data among multiple machine rooms |
| CN112527903A (en) * | 2020-12-11 | 2021-03-19 | 北京首汽智行科技有限公司 | Data synchronization method based on canal platform |
| CN116155920A (en) * | 2023-02-16 | 2023-05-23 | 北京万里开源软件有限公司 | Remote transmission method and system for MySQL protocol database data |
| CN116155920B (en) * | 2023-02-16 | 2023-10-03 | 北京万里开源软件有限公司 | Remote transmission method and system for MySQL protocol database data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110750594B (en) | 2023-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10255343B2 (en) | Initialization protocol for a peer-to-peer replication environment | |
| CN110209726B (en) | Distributed database cluster system, data synchronization method and storage medium | |
| CN110750594A (en) | Mysql-based real-time cross-network database synchronization method for incremental logs | |
| US10895996B2 (en) | Data synchronization method, system, and apparatus using a work log for synchronizing data greater than a threshold value | |
| US9304868B2 (en) | Scalable relational database replication | |
| US8676749B2 (en) | Statement logging in databases | |
| CN106599061B (en) | SQLite-based embedded database synchronization method | |
| US8380664B2 (en) | Transactionally consistent replay of application workloads | |
| US20220207036A1 (en) | Data access method and apparatus, and data storage method and apparatus | |
| US10021182B2 (en) | Method and apparatus for data synchronization | |
| US10795910B2 (en) | Robust communication system for guaranteed message sequencing with the detection of duplicate senders | |
| US20070174315A1 (en) | Compressing state in database replication | |
| CN112367149A (en) | Message acquisition method, device, equipment and storage medium | |
| CN113626416A (en) | Data verification method and device, computing equipment and storage medium | |
| CN115185787A (en) | Method and device for processing transaction log | |
| US20160269335A1 (en) | Method and apparatus for identifying changed mailboxes in an internet message access protocol (imap) list | |
| CN113779094B (en) | Batch-flow-integration-based data processing method and device, computer equipment and medium | |
| CN106897316B (en) | Method and device for processing signaling data | |
| CN113282643B (en) | Table comparison method and system under ORACLE database synchronization environment | |
| CN116186082A (en) | Data summarizing method based on distribution, first server and electronic equipment | |
| CN115422286A (en) | Data synchronization method and device for distributed database | |
| CN113032477B (en) | Long-distance data synchronization method and device based on GTID and computing equipment | |
| US11392574B2 (en) | Mitigating race conditions across two live datastores | |
| CN113407638A (en) | Method for realizing real-time relational database data synchronization | |
| CN109710688A (en) | A kind of real-time Inspection method of data and message-oriented middleware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |