CN104462483A - Data synchronizing method, device and system - Google Patents
Data synchronizing method, device and system Download PDFInfo
- Publication number
- CN104462483A CN104462483A CN201410797462.0A CN201410797462A CN104462483A CN 104462483 A CN104462483 A CN 104462483A CN 201410797462 A CN201410797462 A CN 201410797462A CN 104462483 A CN104462483 A CN 104462483A
- Authority
- CN
- China
- Prior art keywords
- data
- base cluster
- base
- cluster
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data synchronizing method, device and system. The data synchronizing method is used for synchronizing data in a first database cluster to a second database cluster, wherein the first database cluster and the second database cluster are independent of each other. The data synchronizing method includes the steps that a data write operation of the first database cluster is monitored; when it is monitored that data are written in the first database cluster, the written data are read; the data are written in the second database cluster. By means of the technical scheme, data synchronization between the first database cluster and the second database cluster is achieved, the problem that data synchronization among clusters cannot be achieved by an independent cluster is solved, the second database cluster provides data access services to the outside, and thus the pressure of accessed data of the first database cluster is effectively shared.
Description
Technical field
The present invention relates to field of computer technology, particularly a kind of method of data synchronization, Apparatus and system.
Background technology
Mongodb, distribution type file stored data base, is write by C Plus Plus, is intended to for WWW (WEB) application provides extendible high-performance data storage solution.Mongodb is a high-performance, increases income, the Document image analysis of non-mode, is one more popular in current non-relational database.It can be used for alternative traditional relevant database or key/value storage mode under many scenes.
Mongodb itself only allows to hold 7 nodes in a cluster, each node receives the request of exogenous data, ask as increased, deleting, change, make a thorough investigation of, and then calculate data and distributed the in the cluster node of copy thereof according to request content, be responsible for the distribution of request and gathering of response.Along with the increase gradually of portfolio, occur that 7 nodes cannot bear the problem of existing request amount.
Summary of the invention
In view of the above problems, the present invention is proposed to provide a kind of overcoming the problems referred to above or the method for data synchronization solved the problem at least in part, Apparatus and system.
According to one aspect of the present invention, provide a kind of method of data synchronization, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, described first data-base cluster and described second data-base cluster separate, described method comprises: the data write operation monitoring described first data-base cluster; When monitor exist data be written into described first data-base cluster time, read write described data; And by described second data-base cluster of described data write.
Alternatively, described first data-base cluster comprises master data base and at least one from database, and the step of the data write operation of described first data-base cluster of described monitoring comprises further: the data write operation of the master data base in real time in described first data-base cluster of monitoring.
Alternatively, the step of the data write operation of the master data base in described first data-base cluster of described real-time monitoring comprises further: monitor in real time in the oplog journal file of described master data base and represent whether the parameter of write operation is assigned, if so, then determine that there are data is written into described first data-base cluster.
Alternatively, described first data-base cluster comprises master data base and at least one from database, and the step of the data write operation of described first data-base cluster of described monitoring comprises further: at least one data write operation from database in real time in described first data-base cluster of monitoring.
Alternatively, the step of at least one data write operation from database in described first data-base cluster of described real-time monitoring comprises further: in real time monitoring is described at least onely represents from the oplog journal file of database whether the parameter of write operation is assigned, if so, then determine that there are data is written into described first data-base cluster.
Alternatively, described second data-base cluster comprises master data base and at least one from database, and the described step described data being write the second data-base cluster comprises further: by the described master data base of described second data-base cluster of described data write.
Alternatively, after the described master data base of described second data-base cluster of described data write, also comprise: described data are synchronized to the described at least one from database of described second data-base cluster from the described master data base of described second data-base cluster.
Alternatively, described first data-base cluster and described second data-base cluster are Mongodb cluster.
Alternatively, the described described data reading write, comprising: the described data being read write by Mongodb interface.
According to another aspect of the present invention, additionally provide a kind of data synchronization unit, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, described first data-base cluster and described second data-base cluster separate, described data synchronization unit comprises:
Watch-dog, is suitable for the data write operation monitoring described first data-base cluster;
Reader, be suitable for when monitor exist data be written into described first data-base cluster time, read write described data; And
Write device, is suitable for described second data-base cluster of described data write.
Alternatively, described first data-base cluster comprises master data base and at least one from database, and described watch-dog is also suitable for: the data write operation of the master data base in real time in described first data-base cluster of monitoring.
Alternatively, described watch-dog is also suitable for: monitor in real time in the oplog journal file of described master data base and represent whether the parameter of write operation is assigned, if so, then determine that there are data is written into described master first data-base cluster.
Alternatively, described first data-base cluster comprises master data base and at least one from database, and described watch-dog is also suitable for: at least one data write operation from database in real time in described first data-base cluster of monitoring.
Alternatively, described watch-dog is also suitable for: in real time monitoring is described at least onely represents from the oplog journal file of database whether the parameter of write operation is assigned, and if so, then determines that there are data is written into described first data-base cluster.
Alternatively, described second data-base cluster comprises master data base and at least one from database, and said write device is also suitable for: by the described master data base of described second data-base cluster of described data write.
Alternatively, described device also comprises:
Synchronizer, be suitable for said write device by after the described master data base of described second data-base cluster of described data write, described data are synchronized to the described at least one from database of described second data-base cluster from the described master data base of described second data-base cluster.
Alternatively, described first data-base cluster and described second data-base cluster are Mongodb cluster.
Alternatively, described reader is also suitable for: the described data being read write by Mongodb interface.
According to another aspect of the present invention, additionally provide a kind of data synchronous system, comprise the first data-base cluster, the second data-base cluster and above-mentioned data synchronization unit.
Alternatively, described data synchronization unit is independent of described first data-base cluster and described second data-base cluster; Or described data synchronization unit resides in described first data-base cluster or described second data-base cluster.
Technical scheme provided by the invention is applied to cluster self and cannot realizes at least two data-base clusters of inter-cluster synchronization.Because of a certain data-base cluster cannot bear user ask time, need the collaborative burden of another one data-base cluster, but if another data-base cluster cannot realize data syn-chronization, so just cannot realize the object of cooperation service.For solving this technical matters, embodiments provide a kind of method of integration across database cluster synchronization data, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, namely the data write operation of the first data-base cluster is monitored, when monitor exist data be written into the first data-base cluster time, read the data of write, subsequently the data of write are write the second data-base cluster, so that the second data-base cluster externally provides data access service.Thus achieve the data syn-chronization of the first data-base cluster and the second data-base cluster, solve the problem that self contained data base cluster self cannot realize the data syn-chronization between cluster.And, after data syn-chronization to the second data-base cluster of the first data-base cluster, externally data access service can be provided by the second data-base cluster, to solve in correlation technique when request of access amount reaches the extreme value that the first data-base cluster can bear, first data-base cluster cannot continue the problem providing data access service, and then effectively shares the pressure of the accessed data of the first data-base cluster.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to technological means of the present invention can be better understood, and can be implemented according to the content of instructions, and can become apparent, below especially exemplified by the specific embodiment of the present invention to allow above and other objects of the present invention, feature and advantage.
According to hereafter by reference to the accompanying drawings to the detailed description of the specific embodiment of the invention, those skilled in the art will understand above-mentioned and other objects, advantage and feature of the present invention more.
Accompanying drawing explanation
By reading hereafter detailed description of the preferred embodiment, various other advantage and benefit will become cheer and bright for those of ordinary skill in the art.Accompanying drawing only for illustrating the object of preferred implementation, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts by identical reference symbol.In the accompanying drawings:
Fig. 1 shows the schematic flow sheet of method of data synchronization according to an embodiment of the invention;
Fig. 2 shows the structural representation of data synchronization unit according to an embodiment of the invention;
Fig. 3 shows the structural representation of data synchronization unit in accordance with another embodiment of the present invention; And
Fig. 4 shows the structural representation of data synchronous system according to an embodiment of the invention.
Embodiment
Below with reference to accompanying drawings exemplary embodiment of the present disclosure is described in more detail.Although show exemplary embodiment of the present disclosure in accompanying drawing, however should be appreciated that can realize the disclosure in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the disclosure can be understood, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
For solving the problems of the technologies described above, embodiments provide a kind of method of data synchronization, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, first data-base cluster and the second data-base cluster separate, and data-base cluster self cannot realize the data syn-chronization between two data-base clusters.It should be noted that, the not quantitative restriction of first, second data-base cluster in the embodiment of the present invention, can have at least one first data-base cluster, at least one second data-base cluster.Fig. 1 shows the schematic flow sheet of method of data synchronization according to an embodiment of the invention.As shown in Figure 1, the method at least comprises the following steps S102, step S104 and step S106.
Step S102, monitor the data write operation of the first data-base cluster.
Step S104, when monitor exist data be written into the first data-base cluster time, read write data.
Step S106, the data of above-mentioned write are write the second data-base cluster.
Technical scheme provided by the invention is applied to cluster self and cannot realizes at least two data-base clusters of inter-cluster synchronization.Because of a certain data-base cluster cannot bear user ask time, need the collaborative burden of another one data-base cluster, but if another data-base cluster cannot realize data syn-chronization, so just cannot realize the object of cooperation service.For solving this technical matters, embodiments provide a kind of method of integration across database cluster synchronization data, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, namely the data write operation of the first data-base cluster is monitored, when monitor exist data be written into the first data-base cluster time, read the data of write, subsequently the data of write are write the second data-base cluster, so that the second data-base cluster externally provides data access service.Thus achieve the data syn-chronization of the first data-base cluster and the second data-base cluster, solve the problem that self contained data base cluster self cannot realize the data syn-chronization between cluster.And, after data syn-chronization to the second data-base cluster of the first data-base cluster, externally data access service can be provided by the second data-base cluster, to solve in correlation technique when request of access amount reaches the extreme value that the first data-base cluster can bear, first data-base cluster cannot continue the problem providing data access service, and then effectively shares the pressure of the accessed data of the first data-base cluster.
In the embodiment of the present invention, the first data-base cluster can comprise master data base and at least one from database, thus can monitor the master data base of the first data-base cluster or at least one one of any from database.That is, step S102 may be embodied as the data write operation monitoring the master data base in the first data-base cluster in real time above.In a preferred embodiment, oplog (the operationlog of master data base can be monitored in real time, dispatching log) represent whether the parameter of write operation is assigned in journal file, if, then determine that there are data is written into master data base (also namely, data are written into the first data-base cluster); Otherwise, then determine do not have data to be written into master data base.Such as, in the oplog journal file of real-time monitoring master data base, whether insert (insertion) or update (renewal) parameter are assigned, and if so, then determine to monitor to there are data and be written into master data base; If not, then determine not monitor exist data write master data base.In addition, when non-instant data, services, step S102 can also be embodied as at least one data write operation from database monitored in real time in the first data-base cluster.In another preferred embodiment, can monitor in real time and represent whether the parameter of write operation is assigned from the oplog journal file of database, if so, then determine that there are data is written into from database (also namely, data are written into the first data-base cluster); Otherwise, then determine do not have data to be written into from database.Such as, in real time whether monitoring insert (insertion) or update (renewal) parameter from the oplog journal file of database is assigned, and if so, then determines to monitor to there are data and be written into from database; If not, then determine not monitor exist data write from database.
Further, the second data-base cluster can comprise master data base and at least one from database, and step S106 may be embodied as master data base data being write the second data-base cluster.Afterwards, data are synchronized to it from database from the master data base of the second data-base cluster.
In one embodiment of the invention, first, second data-base cluster can be Mongodb cluster.Mongodb cluster is separate cluster, cluster self cannot realize the data syn-chronization between cluster, adopt technical scheme provided by the invention, by the data syn-chronization of Mongodb first data-base cluster to Mongodb second data-base cluster, thus externally can provide data access service by the second data-base cluster.Such as, the request of access amount that Mongodb first data-base cluster can be born is 1000, if when current request of access amount reaches 1001, can provide service by Mongodb second data-base cluster to the 1001st request of access.Again such as, the request of access amount that Mongodb first data-base cluster can be born is 1000, if request of access amount is 1500, service can be provided by Mongodb first data-base cluster to the request of access of 800 wherein, provide service by Mongodb second data-base cluster to remaining 700 request of access.Again such as, the request of access amount that Mongodb first data-base cluster can be born is 1000, the request of access amount that Mongodb second data-base cluster A can bear is 1000, if request of access amount is 2500, now two clusters can not meet current demand, need again to increase cluster, so by the data syn-chronization of Mongodb first data-base cluster to Mongodb second data-base cluster B, service can be provided by the second data-base cluster B to 500 request of access.It should be noted that, it is only schematic for illustrating herein, is not limited to the present invention.
Further, in step S104 when monitor exist data be written into the first data-base cluster time, read write data can by Mongodb interface read write data.In addition, in step S106, the data of above-mentioned write are write the second data-base cluster, by Mongodb interface, the data of above-mentioned write can be write the second data-base cluster.
Based on same inventive concept, the embodiment of the present invention additionally provides a kind of data synchronization unit, to realize above-mentioned method of data synchronization.
Fig. 2 shows the structural representation of data synchronization unit according to an embodiment of the invention.See Fig. 2, this device at least comprises: watch-dog 210, reader 220 and write device 230.
Now introduce the annexation between each composition of the data synchronization unit of the embodiment of the present invention or the function of device and each several part:
Watch-dog 210, is suitable for the data write operation of monitoring first data-base cluster;
Reader 220, is coupled with watch-dog 210, be suitable for when monitor exist data be written into the first data-base cluster time, read write data; And
Write device 230, is coupled with reader 220, is suitable for data to write the second data-base cluster.
In one embodiment of the invention, the first data-base cluster comprises master data base and at least one from database, and above-mentioned watch-dog 210 is also suitable for: the data write operation of the master data base in real time in monitoring the first data-base cluster.
In one embodiment of the invention, above-mentioned watch-dog 210 is also suitable for: monitor in real time in the oplog journal file of master data base and represent whether the parameter of write operation is assigned, if, then determine that there are data is written into master data base (also namely, data are written into the first data-base cluster).
In one embodiment of the invention, the first data-base cluster comprises master data base and at least one from database, and above-mentioned watch-dog 210 is also suitable for: at least one data write operation from database in real time in monitoring the first data-base cluster.
In one embodiment of the invention, above-mentioned watch-dog 210 is also suitable for: in real time monitoring is described at least onely represents from the oplog journal file of database whether the parameter of write operation is assigned, if, then determine that there are data is written into from database (also namely, data are written into the first data-base cluster).
In one embodiment of the invention, the second data-base cluster comprises master data base and at least one from database, and write device 230 is also suitable for: master data base data being write the second data-base cluster.
In one embodiment of the invention, Fig. 3 shows the structural representation of data synchronization unit in accordance with another embodiment of the present invention.See Fig. 3, this device can also comprise: synchronizer 310, be coupled with write device 230, be suitable for after data write the master data base of the second data-base cluster by write device 230, data being synchronized at least one from database of described second data-base cluster from the master data base of the second data-base cluster.The synchronous operation that note that herein is not limited to described synchronizer 310 and performs.Because Mongodb cluster itself also has by master data base data syn-chronization to it from the function of database, therefore described synchronization action also can have been come by Mongodb self when not having synchronizer 310, this not restriction of the present invention.
In one embodiment of the invention, the first data-base cluster and the second data-base cluster are Mongodb cluster.
In one embodiment of the invention, reader 220 is also suitable for: the data being read write by Mongodb interface.
The method of data synchronization provided based on each embodiment above and device, based on same inventive concept, the embodiment of the present invention additionally provides a kind of data synchronous system, shown in Figure 4, this system at least comprises: the first data-base cluster 410, second data-base cluster 420 and data synchronization unit 430 (as shown in Figure 2).
Data synchronization unit 430, is suitable for the data write operation of monitoring first data-base cluster 410; When monitor exist data be written into the first data-base cluster 410 time, read write data; The data of write are write the second data-base cluster 420;
Second data-base cluster 420, is suitable for receiving the data write request from data synchronization unit 430, performs the write operation of data.
In one embodiment of the invention, data synchronization unit 430 is independent of the first data-base cluster 410 and the second data-base cluster 420; Or,
Data synchronization unit 430 resides in the first data-base cluster 410 or the second data-base cluster 420.
In one embodiment of the invention, when receiving data access request, default distributing strategy can be adopted to shunt data access request, and part uses the first data-base cluster 410 to realize access, and part uses the second data-base cluster 420 to realize access.Here distributing strategy can comprise bandwidth load equilibrium, flow load balance or request average access time delay, etc., the present invention is not limited thereto.
According to the combination of any one preferred embodiment above-mentioned or multiple preferred embodiment, the embodiment of the present invention can reach following beneficial effect:
Technical scheme provided by the invention is applied to cluster self and cannot realizes at least two data-base clusters of inter-cluster synchronization.Because of a certain data-base cluster cannot bear user ask time, need the collaborative burden of another one data-base cluster, but if another data-base cluster cannot realize data syn-chronization, so just cannot realize the object of cooperation service.For solving this technical matters, embodiments provide a kind of method of integration across database cluster synchronization data, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, namely the data write operation of the first data-base cluster is monitored, when monitor exist data be written into the first data-base cluster time, read the data of write, subsequently the data of write are write the second data-base cluster, so that the second data-base cluster externally provides data access service.Thus achieve the data syn-chronization of the first data-base cluster and the second data-base cluster, solve the problem that self contained data base cluster self cannot realize the data syn-chronization between cluster.And, after data syn-chronization to the second data-base cluster of the first data-base cluster, externally data access service can be provided by the second data-base cluster, to solve in correlation technique when request of access amount reaches the extreme value that the first data-base cluster can bear, first data-base cluster cannot continue the problem providing data access service, and then effectively shares the pressure of the accessed data of the first data-base cluster.
The invention also discloses:
A1, a kind of method of data synchronization, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, described first data-base cluster and described second data-base cluster separate, described method comprises:
Monitor the data write operation of described first data-base cluster;
When monitor exist data be written into described first data-base cluster time, read write described data; And
By described second data-base cluster of described data write.
A2, method according to A1, wherein, described first data-base cluster comprises master data base and at least one from database, and the step of the data write operation of described first data-base cluster of described monitoring comprises further:
The data write operation of the master data base in described first data-base cluster of real-time monitoring.
A3, method according to A2, wherein, the step of the data write operation of the master data base in described first data-base cluster of described real-time monitoring comprises further:
Represent in the oplog journal file of the described master data base of real-time monitoring whether the parameter of write operation is assigned, if so, then determine that there are data is written into described first data-base cluster.
A4, method according to A1, wherein, described first data-base cluster comprises master data base and at least one from database, and the step of the data write operation of described first data-base cluster of described monitoring comprises further:
At least one data write operation from database in described first data-base cluster of real-time monitoring.
A5, method according to A4, wherein, the step of at least one data write operation from database in described first data-base cluster of described real-time monitoring comprises further:
Real-time monitoring is described at least onely represents from the oplog journal file of database whether the parameter of write operation is assigned, and if so, then determines that there are data is written into described first data-base cluster.
A6, method according to A1, wherein, described second data-base cluster comprises master data base and at least one from database, and the described step described data being write the second data-base cluster comprises further:
By the described master data base of described second data-base cluster of described data write.
A7, method according to A6, wherein, after the described master data base of described second data-base cluster of described data write, also comprise:
Described data are synchronized to the described at least one from database of described second data-base cluster from the described master data base of described second data-base cluster.
A8, method according to any one of A1-A7, wherein, described first data-base cluster and described second data-base cluster are Mongodb cluster.
A9, method according to A8, wherein, the described described data reading write, comprising:
The described data of write are read by Mongodb interface.
B10, a kind of data synchronization unit, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, described first data-base cluster and described second data-base cluster separate, described data synchronization unit comprises:
Watch-dog, is suitable for the data write operation monitoring described first data-base cluster;
Reader, be suitable for when monitor exist data be written into described first data-base cluster time, read write described data; And
Write device, is suitable for described second data-base cluster of described data write.
B11, device according to B10, wherein, described first data-base cluster comprises master data base and at least one from database, and described watch-dog is also suitable for:
The data write operation of the master data base in described first data-base cluster of real-time monitoring.
B12, device according to B11, wherein, described watch-dog is also suitable for:
Represent in the oplog journal file of the described master data base of real-time monitoring whether the parameter of write operation is assigned, if so, then determine that there are data is written into described first data-base cluster.
B13, device according to B10, wherein, described first data-base cluster comprises master data base and at least one from database, and described watch-dog is also suitable for:
At least one data write operation from database in described first data-base cluster of real-time monitoring.
B14, device according to B13, wherein, described watch-dog is also suitable for:
Real-time monitoring is described at least onely represents from the oplog journal file of database whether the parameter of write operation is assigned, and if so, then determines that there are data is written into described first data-base cluster.
B15, device according to B10, wherein, described second data-base cluster comprises master data base and at least one from database, and said write device is also suitable for:
By the described master data base of described second data-base cluster of described data write.
B16, device according to B15, wherein, also comprise:
Synchronizer, be suitable for said write device by after the described master data base of described second data-base cluster of described data write, described data are synchronized to the described at least one from database of described second data-base cluster from the described master data base of described second data-base cluster.
B17, device according to any one of B10-B16, wherein, described first data-base cluster and described second data-base cluster are Mongodb cluster.
B18, device according to B17, wherein, described reader is also suitable for:
The described data of write are read by Mongodb interface.
C19, a kind of data synchronous system, comprise the first data-base cluster, the second data-base cluster and the data synchronization unit described in any one of B10-B18.
C20, system according to C19, wherein, described data synchronization unit is independent of described first data-base cluster and described second data-base cluster; Or,
Described data synchronization unit resides in described first data-base cluster or described second data-base cluster.
In instructions provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in detail in the claims, the one of any of embodiment required for protection can use with arbitrary array mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that the some or all functions that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the some or all parts in the data synchronization unit of the embodiment of the present invention and system.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and does not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computing machine of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.
So far, those skilled in the art will recognize that, although multiple exemplary embodiment of the present invention is illustrate and described herein detailed, but, without departing from the spirit and scope of the present invention, still can directly determine or derive other modification many or amendment of meeting the principle of the invention according to content disclosed by the invention.Therefore, scope of the present invention should be understood and regard as and cover all these other modification or amendments.
Claims (10)
1. a method of data synchronization, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, described first data-base cluster and described second data-base cluster separate, described method comprises:
Monitor the data write operation of described first data-base cluster;
When monitor exist data be written into described first data-base cluster time, read write described data; And
By described second data-base cluster of described data write.
2. method according to claim 1, wherein, described first data-base cluster comprises master data base and at least one from database, and the step of the data write operation of described first data-base cluster of described monitoring comprises further:
The data write operation of the master data base in described first data-base cluster of real-time monitoring.
3. method according to claim 2, wherein, the step of the data write operation of the master data base in described first data-base cluster of described real-time monitoring comprises further:
Represent in the oplog journal file of the described master data base of real-time monitoring whether the parameter of write operation is assigned, if so, then determine that there are data is written into described first data-base cluster.
4. method according to claim 1, wherein, described first data-base cluster comprises master data base and at least one from database, and the step of the data write operation of described first data-base cluster of described monitoring comprises further:
At least one data write operation from database in described first data-base cluster of real-time monitoring.
5. method according to claim 4, wherein, the step of at least one data write operation from database in described first data-base cluster of described real-time monitoring comprises further:
Real-time monitoring is described at least onely represents from the oplog journal file of database whether the parameter of write operation is assigned, and if so, then determines that there are data is written into described first data-base cluster.
6. method according to claim 1, wherein, described second data-base cluster comprises master data base and at least one from database, and the described step described data being write the second data-base cluster comprises further:
By the described master data base of described second data-base cluster of described data write.
7. method according to claim 6, wherein, after the described master data base of described second data-base cluster of described data write, also comprises:
Described data are synchronized to the described at least one from database of described second data-base cluster from the described master data base of described second data-base cluster.
8. the method according to any one of claim 1-7, wherein, described first data-base cluster and described second data-base cluster are Mongodb cluster.
9. a data synchronization unit, for by the data syn-chronization in the first data-base cluster to the second data-base cluster, described first data-base cluster and described second data-base cluster separate, described data synchronization unit comprises:
Watch-dog, is suitable for the data write operation monitoring described first data-base cluster;
Reader, be suitable for when monitor exist data be written into described first data-base cluster time, read write described data; And
Write device, is suitable for described second data-base cluster of described data write.
10. a data synchronous system, comprises the first data-base cluster, the second data-base cluster and data synchronization unit according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410797462.0A CN104462483A (en) | 2014-12-18 | 2014-12-18 | Data synchronizing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410797462.0A CN104462483A (en) | 2014-12-18 | 2014-12-18 | Data synchronizing method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104462483A true CN104462483A (en) | 2015-03-25 |
Family
ID=52908518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410797462.0A Pending CN104462483A (en) | 2014-12-18 | 2014-12-18 | Data synchronizing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104462483A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912628A (en) * | 2016-04-07 | 2016-08-31 | 北京奇虎科技有限公司 | Synchronization method and device for master database and slave database |
| CN106331063A (en) * | 2016-08-15 | 2017-01-11 | 成都轻车快马网络科技有限公司 | Database deployment method |
| CN107124469A (en) * | 2017-06-07 | 2017-09-01 | 郑州云海信息技术有限公司 | A kind of clustered node communication means and system |
| CN107145562A (en) * | 2017-05-02 | 2017-09-08 | 北京奇艺世纪科技有限公司 | A kind of method of data synchronization, apparatus and system |
| CN107370809A (en) * | 2017-07-13 | 2017-11-21 | 广州市百果园信息技术有限公司 | Method of data synchronization and data search system |
| CN107451245A (en) * | 2017-07-27 | 2017-12-08 | 联想(北京)有限公司 | Method of data synchronization and server cluster |
| CN107644035A (en) * | 2016-07-21 | 2018-01-30 | 百度在线网络技术（北京）有限公司 | A kind of Database Systems and its dispositions method |
| CN108881379A (en) * | 2018-05-03 | 2018-11-23 | 网宿科技股份有限公司 | The synchronous method and apparatus of data between a kind of server cluster |
| CN112667698A (en) * | 2021-01-04 | 2021-04-16 | 山西云媒体发展有限公司 | MongoDB data synchronization method based on converged media platform |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101854400A (en) * | 2010-06-09 | 2010-10-06 | 中兴通讯股份有限公司 | Database synchronization deployment and monitoring method and device |
| CN102158540A (en) * | 2011-02-18 | 2011-08-17 | 广州从兴电子开发有限公司 | System and method for realizing distributed database |
| CN102693324A (en) * | 2012-01-09 | 2012-09-26 | 西安电子科技大学 | Distributed database synchronization system, synchronization method and node management method |
| CN102750283A (en) * | 2011-04-20 | 2012-10-24 | 阿里巴巴集团控股有限公司 | Massive data synchronization system and method |
| CN103150304A (en) * | 2011-12-06 | 2013-06-12 | 郑红云 | Cloud database system |
| US8548945B2 (en) * | 2011-09-23 | 2013-10-01 | International Business Machines Corporation | Database caching utilizing asynchronous log-based replication |
| CN103747060A (en) * | 2013-12-26 | 2014-04-23 | 惠州华阳通用电子有限公司 | Distributed monitor system and method based on streaming media service cluster |
-
2014
- 2014-12-18 CN CN201410797462.0A patent/CN104462483A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101854400A (en) * | 2010-06-09 | 2010-10-06 | 中兴通讯股份有限公司 | Database synchronization deployment and monitoring method and device |
| CN102158540A (en) * | 2011-02-18 | 2011-08-17 | 广州从兴电子开发有限公司 | System and method for realizing distributed database |
| CN102750283A (en) * | 2011-04-20 | 2012-10-24 | 阿里巴巴集团控股有限公司 | Massive data synchronization system and method |
| US8548945B2 (en) * | 2011-09-23 | 2013-10-01 | International Business Machines Corporation | Database caching utilizing asynchronous log-based replication |
| CN103150304A (en) * | 2011-12-06 | 2013-06-12 | 郑红云 | Cloud database system |
| CN102693324A (en) * | 2012-01-09 | 2012-09-26 | 西安电子科技大学 | Distributed database synchronization system, synchronization method and node management method |
| CN103747060A (en) * | 2013-12-26 | 2014-04-23 | 惠州华阳通用电子有限公司 | Distributed monitor system and method based on streaming media service cluster |
Non-Patent Citations (1)
| Title |
|---|
| 白坤娟: "认证***异构数据库数据同步方法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912628A (en) * | 2016-04-07 | 2016-08-31 | 北京奇虎科技有限公司 | Synchronization method and device for master database and slave database |
| CN105912628B (en) * | 2016-04-07 | 2019-05-28 | 北京奇虎科技有限公司 | The synchronous method and device of master-slave database |
| CN107644035A (en) * | 2016-07-21 | 2018-01-30 | 百度在线网络技术（北京）有限公司 | A kind of Database Systems and its dispositions method |
| CN106331063B (en) * | 2016-08-15 | 2019-04-09 | 成都轻车快马网络科技有限公司 | Database portion arranging method |
| CN106331063A (en) * | 2016-08-15 | 2017-01-11 | 成都轻车快马网络科技有限公司 | Database deployment method |
| CN107145562A (en) * | 2017-05-02 | 2017-09-08 | 北京奇艺世纪科技有限公司 | A kind of method of data synchronization, apparatus and system |
| CN107145562B (en) * | 2017-05-02 | 2020-07-28 | 北京奇艺世纪科技有限公司 | Data synchronization method, device and system |
| CN107124469A (en) * | 2017-06-07 | 2017-09-01 | 郑州云海信息技术有限公司 | A kind of clustered node communication means and system |
| CN107370809A (en) * | 2017-07-13 | 2017-11-21 | 广州市百果园信息技术有限公司 | Method of data synchronization and data search system |
| CN107451245A (en) * | 2017-07-27 | 2017-12-08 | 联想(北京)有限公司 | Method of data synchronization and server cluster |
| CN108881379A (en) * | 2018-05-03 | 2018-11-23 | 网宿科技股份有限公司 | The synchronous method and apparatus of data between a kind of server cluster |
| CN112667698A (en) * | 2021-01-04 | 2021-04-16 | 山西云媒体发展有限公司 | MongoDB data synchronization method based on converged media platform |
| CN112667698B (en) * | 2021-01-04 | 2023-02-28 | 山西云媒体发展有限公司 | MongoDB data synchronization method based on converged media platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104462483A (en) | Data synchronizing method, device and system | |
| CN103237046B (en) | Support distributed file system and the implementation method of mixed cloud storage application | |
| Benelallam et al. | Neo4EMF, a scalable persistence layer for EMF models | |
| CN104615606B (en) | A kind of Hadoop distributed file systems and its management method | |
| JP5376696B2 (en) | Document synchronization via stateless protocol | |
| CN102411637B (en) | Metadata management method of distributed file system | |
| CN110362381A (en) | HDFS cluster High Availabitity dispositions method, system, equipment and storage medium | |
| CN101272313B (en) | Intermediate device for achieving virtualization of file level, file server system and relay method | |
| CN104537076A (en) | File reading and writing method and device | |
| US20170147602A1 (en) | Allocating file system metadata to storage nodes of distributed file system | |
| CN102882985A (en) | File sharing method based on cloud storage | |
| CN104220987A (en) | Application installation | |
| KR101545626B1 (en) | System for interoperation between dds and dbms | |
| CN109684282A (en) | A kind of method and device constructing metadata cache | |
| CN105740469B (en) | Storage server and metadata access method | |
| CN104156381A (en) | Copy access method and device for Hadoop distributed file system and Hadoop distributed file system | |
| CN104270412A (en) | Three-level caching method based on Hadoop distributed file system | |
| CA3137742A1 (en) | Blockchain cache system | |
| CN103414762A (en) | Cloud backup method and cloud backup device | |
| CN105049524B (en) | A method of the large-scale dataset based on HDFS loads | |
| CN105335450B (en) | Data storage processing method and device | |
| CN106708636A (en) | Cluster-based data caching method and apparatus | |
| CN102281312A (en) | Data loading method and system and data processing method and system | |
| US20150295984A1 (en) | Multi Web Application Management Framework System and Method | |
| CN108243207A (en) | A kind of date storage method of network cloud disk |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150325 |