CN117112690A - Data synchronization method of distributed cluster and related equipment thereof - Google Patents
Data synchronization method of distributed cluster and related equipment thereof Download PDFInfo
- Publication number
- CN117112690A CN117112690A CN202310576529.7A CN202310576529A CN117112690A CN 117112690 A CN117112690 A CN 117112690A CN 202310576529 A CN202310576529 A CN 202310576529A CN 117112690 A CN117112690 A CN 117112690A
- Authority
- CN
- China
- Prior art keywords
- storage management
- management unit
- data
- master
- slave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000003860 storage Methods 0.000 claims abstract description 303
- 230000001360 synchronised effect Effects 0.000 claims abstract description 89
- 238000007726 management method Methods 0.000 claims description 268
- 238000013500 data storage Methods 0.000 claims description 8
- 230000002688 persistence Effects 0.000 claims description 2
- 230000005012 migration Effects 0.000 abstract description 17
- 238000013508 migration Methods 0.000 abstract description 17
- 238000004590 computer program Methods 0.000 description 14
- 238000012545 processing Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000002085 persistent effect Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 238000007251 Prelog reaction Methods 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Hardware Redundancy (AREA)
Abstract
The application relates to a data synchronization method of a distributed cluster and related equipment thereof. Wherein the method comprises the following steps: determining a synchronous time slice based on master-slave state switching of a storage management unit group in the distributed cluster; generating a data operation log of a main storage management unit in the storage management unit group in a synchronous time slice; the data operation log is synchronized to the slave storage management units in the storage management unit group so that the slave storage management units perform data synchronization based on the data operation log. The application solves the problems of service unavailability and increment data synchronization consistency caused by shutdown migration among different isolated sites, realizes non-shutdown migration data among the isolated sites, and realizes increment data synchronization among the isolated sites.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data synchronization method for a distributed cluster and related devices thereof.
Background
This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. It is not admitted to be prior art by inclusion of this description in this section.
The product warehouse needs to carry out redundant backup at different sites and product synchronization among different sites, so that when the regional problem occurs, master-slave switching can be carried out to achieve the purpose of high availability, and meanwhile, network delay caused by regional factors can be solved through multi-site deployment. In addition, there is also a need for product synchronization between different deployment environments, where unverified products need to be verified in the development environment first, and then synchronized to a stable production environment by a site synchronization function after passing the verification.
The different sites of the product warehouse are isolated, so that the conventional online data synchronization method cannot realize the data synchronization between the different sites of the product warehouse. Migration of data between different sites isolated in the related art usually adopts a shutdown migration manner, however, the shutdown migration will cause temporary unavailability of services, and real-time synchronization cannot be achieved for incremental data.
Disclosure of Invention
The data synchronization method and the related device of the distributed cluster at least solve the problem that service is unavailable due to shutdown migration among different isolated sites.
A method of data synchronization for a distributed cluster, comprising:
Determining a synchronous time slice based on master-slave state switching of a storage management unit group in the distributed cluster;
generating a data operation log of a main storage management unit in the storage management unit group in a synchronous time slice;
and synchronizing the data operation log to a slave storage management unit in the storage management unit group so that the slave storage management unit performs data synchronization based on the data operation log.
In some of these embodiments, the method further comprises:
generating metadata of the master storage management unit so that when the data operation log is synchronized to a slave storage management unit in the storage management unit group, determining a data operation log to be synchronized in the data operation log based on the metadata, wherein the metadata comprises a synchronization state of the data operation log;
and updating the metadata based on the synchronization result of the data operation log.
In some of these embodiments, generating a data operation log for a master storage management unit in the storage management unit group within a synchronization time slice includes:
persistence of data operation information of clusters in the distributed clusters according to an execution sequence of data operation, wherein the data operation information comprises information of an operation object, operation content and a storage management unit for executing the data operation;
And acquiring the data operation logs of the main storage management unit in the synchronous time slices according to the data operation information of the clusters, wherein the data operation logs are ordered according to the execution sequence of the data operation.
In some of these embodiments, determining the synchronization time slices based on master-slave state switching of storage management unit groups in the distributed cluster includes:
and acquiring a master-slave state switching logic clock of the storage management unit group, and determining the synchronous time slices and the serial numbers of the synchronous time slices according to the master-slave state switching logic clock.
In some of these embodiments, the method further comprises:
when the master-slave state of the storage management unit of the cluster in the distributed cluster is switched, the cluster reports the master-slave state switching logic clock to a storage system, and the storage system sequentially and automatically increases the serial numbers of the synchronous time slices according to the master-slave state switching logic clock and then sends the serial numbers to the distributed cluster.
In some of these embodiments, the method further comprises:
the distributed cluster periodically acquires the sequence numbers of the synchronous time slices from the storage system.
In some of these embodiments, synchronizing the data operation log to a slave storage management unit in the storage management unit group comprises:
And determining the data operation log to be synchronized in the master storage management unit according to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the serial number of the synchronization time slice, and synchronizing the data operation log to be synchronized to the slave storage management unit.
In some of these embodiments, the data operation log to be synchronized is determined and the synchronization of the data operation log to be synchronized is performed in a case where the sequence numbers of the synchronization time slices of the master storage management unit and the slave storage management unit are the same.
In some embodiments, in the initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, the storage management units in the slave cluster in the distributed cluster are all slave storage management units, and master-slave switching of the storage management unit group is managed by a preset controller.
In some embodiments, the preset controller sets the storage management unit with the identifier as a main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.
In some of these embodiments, in the event that there is data to be written to the storage management unit group, the data is written to the primary storage management unit by querying the preset controller to determine the primary storage management unit in the storage management unit group.
In some of these embodiments, the method further comprises:
and taking the data storage state of the storage management unit as a finite state machine, respectively electing a leader node and at least one follower node in the master storage management unit and the slave storage management unit, and carrying out copy synchronization of the finite state machine in the storage management unit group by the leader node based on the data operation log.
A distributed cluster system comprising a plurality of clusters including a master cluster and at least one slave cluster, and a storage system connected to the plurality of clusters, respectively;
the storage system is used for storing the sequence numbers of the synchronous time slices; the clusters are all subjected to data synchronization through the data synchronization method.
In some embodiments, the distributed cluster system further includes a preset controller, where the preset controller is configured to control a master-slave state switching of the storage management unit, and record a master-slave state of the storage management unit.
An electronic device, comprising: a processor, and a memory storing a program, wherein the program comprises instructions that, when executed by the processor, cause the processor to perform the data synchronization method described above.
A non-transitory machine readable medium storing computer instructions for causing the computer to perform the data synchronization method described above.
The data synchronization method and the related equipment of the distributed cluster provided by the embodiment of the invention determine the synchronization time slices by switching the master-slave states based on the storage management unit groups in the distributed cluster; generating a data operation log of a main storage management unit in the storage management unit group in a synchronous time slice; the data operation log is synchronized to the slave storage management units in the storage management unit group, so that the slave storage management units perform data synchronization based on the data operation log, the problems of service unavailability and incremental data synchronization consistency caused by shutdown migration among different isolated sites are solved, non-shutdown migration data among the isolated sites is realized, and incremental data synchronization among the isolated sites is realized.
The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is evident that the drawings in the following description are only some embodiments of the invention, from which other embodiments can be obtained for a person skilled in the art without inventive effort.
Fig. 1 is a flow chart of a data synchronization method of a distributed cluster according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of generating data oplogs and metadata for a storage management unit in accordance with an embodiment of the invention.
FIG. 3 is a flow chart of a storage management unit sending a data oplog request in accordance with an embodiment of the invention.
FIG. 4 is a flow chart of a storage management unit receiving a data oplog request according to an embodiment of the invention.
FIG. 5 is a flow chart of a storage management unit as a Master sending a snapshot of data according to an embodiment of the present invention.
FIG. 6 is a schematic diagram of a multi-storage management unit synchronization control according to an embodiment of the present invention.
FIG. 7 is a schematic diagram of synchronous task processing based on a distributed consistency algorithm according to an embodiment of the present invention.
Fig. 8 is a schematic diagram of a distributed cluster according to an embodiment of the present invention.
Fig. 9 is a schematic structural diagram of the electronic apparatus of the present embodiment.
Detailed Description
Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present embodiments are illustrated in the accompanying drawings, it is to be understood that the present embodiments may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided to provide a more thorough and complete understanding of the present embodiments. It should be understood that the drawings and the embodiments of the present embodiments are presented for purposes of illustration only and are not intended to limit the scope of the embodiments.
The library of artifacts, as the name implies, is a warehouse of artifacts, which are the resulting products of software delivery, typically in a binary form that is operable, and thus the library of artifacts is also commonly referred to as a binary warehouse of artifacts. The artifact library provides a unique portal for dependent parsing for micro-service developers using various development languages during the development phase. The method provides unique dependency analysis sources and unified management libraries of various language products for construction tools of various languages in the construction stage. After the testing stage is entered, all the testing environment deployment tools pull the products meeting the testing conditions from the product library to deploy, and after the testing is finished, the testing result data is fed back to the product library and is associated with the products. And in the deployment stage, whether the product meets deployment conditions or not is checked according to the quality checkpoints, and if the product meets the deployment conditions, the deployment tool pulls the product from the product warehouse to deploy the product docking environment.
In order to ensure the safety of data synchronization, the real-time data synchronization scheme needs to have uniform synchronization clocks among different devices, so that data synchronization errors are avoided, and rollback of data is ensured. However, the product warehouse adopts multi-region isolation, and because the isolated regions have no synchronous clock, the conventional real-time data synchronization method in the related technology cannot be adopted, but only a shutdown migration mode can be adopted.
For example, in the related art, the repository replication data migration scheme of jfork uses an http calling method to upload the product by calling the interface of the existing product system, but this method cannot avoid the data synchronization security and stability under the non-bayer problem, only supports product migration but does not support data migration of other formats, and cannot perform isolation operation on the multiple region data.
In the related art, an open source data migration scheme of the Rsync is used for carrying out data transmission by establishing ssh connection with a target system and calling a data interface on a remote system for carrying out data transmission. This approach requires manual migration and fails to synchronize incremental data automatically, and simply file synchronization, fails to establish a relationship between the file and the system.
The distributed cluster system of the present embodiment includes one Master cluster (Master cluster) and at least one slave cluster (slave cluster). Each cluster includes a plurality of nodes, which in turn are divided by regions into a plurality of storage management units, also known as regions, that are isolated from each other. The master storage management unit and the slave storage management units which perform data synchronization are used as a storage management unit group, and one master storage management unit and at least one or a plurality of slave storage management units are arranged in the storage management unit group. The master-slave state of the storage management units within the storage management unit group may be switched. In the initial state, the regions in the Master cluster are Master roles, namely in a Master state, and the regions in the slave clusters are slave roles, namely in a slave state. The main cluster is used as a cluster for initiating data synchronization in the data synchronization process; the slave cluster is the cluster receiving the data sync during the data sync process. But after a master-slave state switch of a region within a master-slave cluster, data synchronization may also be initiated by the slave cluster and received by the master cluster.
For example, when data is synchronized, in an initial state, the data is written into a region serving as a master role in a master cluster and then synchronized to a region serving as a slave role in a slave cluster. After the master-slave state of two or more regions which are master-slave, the data is written into the region serving as the master role in the slave cluster and then synchronized to the region serving as the slave role in the master cluster.
In order to achieve data synchronization between isolated sites and avoid service outage unavailability, the embodiment of the invention provides a data synchronization method of a distributed cluster. Fig. 1 is a flowchart of a data synchronization method of a distributed cluster according to an embodiment of the present invention, as shown in fig. 1, the flowchart includes the following steps:
step S101, determining a synchronous time slice based on master-slave state switching of a storage management unit group in a distributed cluster.
Step S102, generating a data operation log of the main storage management unit in the storage management unit group in the synchronous time slice.
Step S103, synchronizing the data operation log to the slave storage management unit in the storage management unit group so that the slave storage management unit performs data synchronization based on the data operation log.
Through the steps, based on the master-slave state switching of the storage management unit groups in the distributed cluster, the synchronous time slices are determined, and the data storage states of the storage management units in the synchronous time slices are synchronized based on the data operation logs, so that the problem of real-time data synchronization among the isolated multiple storage management units is solved. The data synchronization method not only can realize the real-time data synchronization of the stock data, but also can realize the real-time data synchronization of the incremental data without shutdown migration, and is also suitable for any type of data synchronization.
Wherein the synchronization status of the data operation log, for example, the synchronized data operation log, the data operation log submitted to the slave storage management unit, and the like, which can be stored in metadata, is updated according to the synchronization result of the data operation log to determine the data storage status of each storage management unit at the time of synchronizing the data. In some of these embodiments, the method further comprises: generating metadata of the master storage management unit so that when the data operation log is synchronized to a slave storage management unit in the storage management unit group, determining a data operation log to be synchronized in the data operation log based on the metadata, wherein the metadata comprises a synchronization state of the data operation log; and updating the metadata based on the synchronization result of the data operation log.
In some of these embodiments, the data storage state of the storage management unit is used as a finite state machine, so that the data synchronization of the storage management unit can be implemented based on a copy synchronization mechanism of the finite state machine. In the copy synchronization mechanism of the finite state machine, the copy synchronization is performed based on a synchronization log (log), and in this embodiment, the data operation log is used as the synchronization log for the copy synchronization of the finite state machine.
In this embodiment, the predefined data objects include cluster objects, node objects, and finite state machines. Wherein the cluster object comprises the state of each cluster node. The node object includes metadata (metadata) of the node, which is used to record status information of a cluster in the data synchronization process, including a sequence number (term) of a synchronization time slice of the storage management unit, an index (logIndex) of a data operation log for marking location information of the data operation log, an index number (snapshotIndex) of a last data operation log included in the snapshot, a sequence number (snapshotTerm) of a synchronization time slice of a last data operation log included in the snapshot, and an index number (preLogIndex) of a maximum log that has been applied to the slave. The node object also includes node location information (endpoint) that represents coordinates of the node, typically an IP address of the terminal in the network environment. The node object may also include the state of the master node that initiated the data synchronization cluster during the data synchronization process. The finite state machine comprises data objects that are executed, actions that are executed (i.e., data operations). In this embodiment, the storage management unit having a master-slave relationship is regarded as a finite state machine.
The data storage state comprises stock data and data operations such as adding, deleting and modifying based on the stock data. Wherein the stock data is usually stored in the form of a data snapshot (snapshot), and the data operation is recorded in an operation log. If the stock data is synchronized between the storage management units in the master-slave relationship, that is, the stock data of each storage management unit is the same, the data storage state of the storage management unit may be represented by only the data operation on the stock data.
The finite state machine has a finite state, and starts at a given starting state (e.g., an initial state, or a state with the same stock data), and each input received by the state machine produces a new state and corresponding output via the transition equation and the output equation. This new state will remain until the next input arrives, and the resulting output will be passed to the corresponding recipient. Multiple copies of the same state machine may be synchronized among multiple storage management units based on a replicated state machine approach, starting with a start state, and receiving the same input in the same order will arrive at the same state that has generated the same output, i.e., replicated state machine.
The above-mentioned synchronization time slice sequence number (term) is a synchronization clock for all clusters in the distributed cluster, and the synchronization time slice sequence number is strictly and sequentially self-increased from a set value (for example, 0). The sequence numbers of the synchronized time slices are buffered in a storage system having a high performance storage medium.
In the above step, the synchronization time slices of the finite state machine are updated by switching between the master state and the slave state of any group of storage management units. Since the update of the sequence numbers of the synchronous time slices takes the master-slave state switching of the storage management units as a mark, the states of the finite state machines of any storage management unit after the master-slave state switching fall into at least one synchronous time slice, namely, the correspondence between the finite state machines of the storage management units and the sequence numbers of the synchronous time slices can be established. Therefore, the cluster can determine the sequence number of the synchronous time slice which is started to enter from the master-slave state switching logic clock by acquiring the master-slave state switching logic clock of any group of storage management units in the cluster. Since the sequence numbers of the synchronization time slices are also self-increased by the influence of master-slave state switching of other groups of storage management units in the distributed cluster, for a certain storage management unit, the state change of the finite state machine may span multiple synchronization time slices and correspond to the sequence numbers of the multiple synchronization time slices. In this embodiment, for the data operation log and metadata generated in each synchronization time slice, the serial number of the corresponding synchronization time slice is marked.
When the master-slave state of any group of storage management units in any one of the distributed clusters is switched, any one of the clusters reports the master-slave state switching logic clock to the storage system, the serial numbers of the synchronous time slices in the storage system are increased by one, and the storage system sequentially and automatically increases the serial numbers of the synchronous time slices according to the master-slave state switching logic clock and then sends the serial numbers to the distributed clusters. In addition, the clusters in the distributed clusters can regularly pull the serial numbers of the synchronous time slices so as to prevent the serial number issuing failure of the synchronous time slices.
In this embodiment, the input data of the finite state machine is data operation information. In order to meet the same input requirement required by the replication state machine, the data operation information needs to be identical in operation content and execution sequence. Thus, in any one of the distributed clusters, the data operation information of the clusters in the distributed cluster is persisted in an execution order of the data operations, wherein the data operation information includes information of an operation object, operation contents, and a storage management unit that performs the data operations.
Specifically, in a cluster (typically a Master cluster) receiving data writing, a buried point record can be added by writing operation at each place to record information of an operation object, operation content and a storage management unit executing data operation, the operation log (pendingLog) of an increment generated by a data synchronization initiator after encapsulation is used, the increment operation log is then persisted into a database, and the database is used for ensuring that the increment operation log is stored according to an execution sequence.
Since in the cluster receiving the data write, the operation logs of all the increments performed by the storage management unit are stored together in the order of execution. For data synchronization in units of storage management units, incremental operation logs may be further subjected to grouping processing in units of storage management units. For example, as shown in fig. 2, a node in a cluster receiving data writing selects a node starting thread for processing a synchronization task to read incremental operation logs from a database in batches, then the operation logs are divided according to a storage management unit, and each log is marked with a serial number of a synchronization time slice and index (logIndex) information of the data operation log in the same storage management unit to generate the data operation log for data synchronization.
If the current storage management unit is generating a synchronization log (log) for the first time, the index number of the data operation log sequentially self-increases from 0, and meanwhile, the metadata (metadata) of the storage management unit is initialized, and the index number (snapshontddex) of the data snapshot in the metadata is a random value n (for example, may be any integer between 1 and 100). The metadata of the storage management unit further includes data for performing data synchronization management, for example, an index number (lastappliedbdex) of a data operation log of the last slave node to which the storage management unit has been applied, and an index number (compttedbdex) of a data operation log of the storage management unit to which the data operation log has been submitted. At initialization, lastappiiedindex and comittedindex are set to the same value n as snapshontdindex. The sequence number of the synchronization time slice in the metadata is set as the sequence number of the current synchronization time slice. The data operation log and the metadata are persistent, namely the data preparation work of data synchronization is completed.
If the current storage management unit does not generate the synchronization log (log) for the first time, that is, there is a history of data operation logs (including the data operation log that has been synchronized and the data operation log that has not been synchronized), the index number of the newly generated data operation log is increased by one from the index number of the last data operation log generated last time, the metadata snapshotIndex, lastAppliedIndex, committedIndex is also increased by one from snapshotIndex, lastAppliedIndex, committedIndex generated last time, and the serial number of the synchronization time slice is also set to be the serial number of the current synchronization time slice. The data operation log and the metadata are persistent, namely the data preparation work of data synchronization is completed.
And when the data synchronization is carried out, the data synchronization is realized based on the copy synchronization of the finite state machine. For the storage management unit, the cluster receiving the data writing determines the data operation log to be synchronized in the master storage management unit according to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the serial number of the synchronization time slice, and synchronizes the data operation log to be synchronized to the slave storage management unit.
Referring to fig. 3, a master storage management unit selects a node for processing a synchronization task, and the node processes the synchronization task to screen out a storage management unit requiring synchronization. In the synchronization task, firstly, the index number of the next log to be synchronized is obtained in a cache, if the index number does not exist, the index number of the last submitted log in metadata is queried, then the index number preLogIndex of the last data operation log is set as the submitted index number minus one, whether the snapshot is sent or not is determined by judging the sizes of the preLogIndex and the snapshot Index, if the preLogIndex is not smaller than the snapshot Index, the data is migrated, otherwise, a batch of data operation logs to be synchronized are fetched from a database and are sent to a slave storage management unit for processing.
The flow of processing the data operation log from the storage management unit is shown in fig. 4, and the node for processing data synchronization is also selected from the storage management unit. The node is responsible for processing the data oplog.
In some embodiments, in the same storage management unit group, if the serial numbers of the synchronization time slices of the master storage management unit and the slave storage management unit are the same, determining the data operation log to be synchronized and synchronizing the data operation log to be synchronized, and if the serial numbers of the synchronization time slices are different, retrying.
If the sequence number of the synchronization time slice of the received data operation log is inconsistent with the sequence number of the synchronization time slice of the current node, it may be that the sequence number of the synchronization time slice of the current node is not updated or the synchronization request of the data operation log has expired, and the node returns the sequence number of the synchronization time slice of the current node to the main storage management unit. If the serial numbers of the synchronous time slices are consistent, the synchronous request of the data operation log is effective, whether the current storage management unit is in a slave role (slave) state in the cluster is checked again, and if the current storage management unit is in the slave state, log checking is performed.
The state of the current FSM of the slave storage management unit is determined by the commit index of the submitted data operation log and the sequence number of the synchronous time slice, if the current FSM can be matched with the synchronous request, the current FSM is updated by directly applying the log sent by the master storage management unit, the result is returned to the master storage management unit, otherwise, the current FSM is returned to the master storage management unit, and the master storage management unit is expected to send the matched log in the next synchronous request. The master storage management unit may not receive the return value of the slave storage management unit in response to the timeout, and retry at this time, and if the return value of the slave storage management unit is received, update the next transmitted synchronization request according to the state returned from the storage management unit.
The synchronization method can be used for synchronizing incremental data and can also be used for synchronizing stock data. With continued reference to fig. 3, when a data operation log request is sent for the first time, the data operation log of the storage management unit is not written in the slave storage management unit, the conflict position returned to the master storage management unit is 0, and the master storage management unit continues to send the data operation log after updating the next log index, at this time, the pre log index of the data operation log is necessarily smaller than the randomly generated snapshotIndex, so that the master storage management unit sends snapshot to migrate the storage data. The snapshot is a snapshot of the FSM of the current storage management unit generated by the primary storage management unit, where an index of all data (e.g., artifacts) and related data are maintained. In some embodiments, the snapshot sent by the main storage management unit may be a URL of the snapshot, and the actually generated snapshot may be uploaded to the file storage system, so that the data size of the snapshot request sent can be reduced. The flow of sending the snapshot by the main storage management unit is as shown in fig. 5, firstly, a storage management unit needing stock migration is determined in a set needing to send the snapshot of the storage management unit, after an FSM snapshot of the storage management unit is generated, a request for installing the snapshot is sent to a slave storage management unit, the installation state of the slave storage management unit is judged, and if the serial numbers of the synchronous time slices of the request and the slave storage management unit are not matched or the installation of the snapshot by the slave storage management unit fails, the request for installing the snapshot is added into the set sending the snapshot to retry.
In some embodiments, in the initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, the storage management units in the slave cluster in the distributed cluster are all slave storage management units, and master-slave switching of the storage management units is managed through a preset controller. The preset controller sets the storage management unit with the identifier as a main storage management unit in a manner of adding the identifier of the storage management unit into a gray list so as to realize the master-slave state switching of the storage management units in the distributed cluster.
For example, referring to fig. 6, a master-slave relationship of different storage management units in different clusters is set, and is stored in a master-slave controller available for inquiry. When the storage management units are not in the gray list, the storage management units of the cluster 1 (for example, the Master cluster) are in a Master state, the storage management units of the cluster 2 (for example, the slave cluster) are in a slave state, and the data flow is that the slave cluster 1 flows into the cluster 2. If the storage management unit is in the gray list, the storage management unit of the cluster 2 is in the Master state, and data flows from the cluster 2 into the cluster 1. Because of the bidirectional writing of data, the Finite State Machines (FSMs) of the master storage management unit and the slave storage management units in the same storage management unit group always keep the consistent state in a safe state, and at the moment, the roles of different storage management units in different clusters are controlled by controlling whether the storage management units are in a gray list or not so as to control the flow direction of data synchronization. The newly added data of the user in any cluster cannot be lost, and the newly added data can be synchronized in the cluster 1 and the cluster 2 in a mode of a synchronization log (log). The above-mentioned secure state means that the FSM states of the master storage management unit and the slave storage management unit in the same storage management unit are kept consistent, and there is no data operation log that has been applied to the master storage management unit but has not been applied to the slave storage management unit.
In the case where a preset controller is provided for querying the master-slave state of the storage management unit, in some of these embodiments, in the case where new data is written to the storage management unit, the master storage management unit is determined by querying the preset controller, and the new data is written to the master storage management unit. For example, when a user needs to perform a data operation, the user may directly access cluster 1 or cluster 2; the user confirms whether the master node or the slave node is accessed by querying the identity of the storage management unit stored in the master-slave controller.
In order to ensure the reliability of data synchronization and avoid data synchronization failure caused by single node failure, in some embodiments, in a master storage management unit and a slave storage management unit, a leader node and at least one follower node are respectively elected, and the leader node in each storage management unit performs copy synchronization of a finite state machine of the storage management unit with a master-slave relationship. Referring to fig. 7, in this embodiment, a leader node may be elected in each cluster or storage management unit to process a synchronization task based on a raft (a distributed consistency algorithm); and when the leader node of the main storage management unit is in abnormal conditions such as downtime, the data in the main storage management unit is persistent and is not lost, and after a new leader is selected in the follower node, the data can be continuously synchronized to the auxiliary storage management unit according to the persistent log and metadata, so that the auxiliary storage management unit can still keep consistent with the main storage management unit.
The embodiment of the invention can realize stock data migration and incremental data synchronization of multiple storage management units based on a shift log synchronization technology, can be applied to any format file, is not limited to Jfork product files, and ensures the data synchronization safety and reliability under the non-Bayesian problem through shift. All synchronous data are applied to the system through the FSM, and the association relationship between the data can be synchronized among different systems. The data are isolated through the multiple storage management units, so that the synchronous directions of different storage management units among different sites can be controlled.
The embodiment also provides a distributed cluster system. Fig. 8 is a schematic diagram of a distributed cluster system according to the present embodiment, and as shown in fig. 8, the system includes a cluster 81, at least one cluster 82, and a storage system 83, where the cluster 81 is used as a master role, the cluster 82 is used as a slave role, and the storage system 83 is connected to the above-mentioned clusters, respectively.
The storage system 83 is used for storing the sequence numbers of the synchronous time slices; the clusters are all subjected to data synchronization by the data synchronization method of the distributed clusters.
In some of these embodiments, the cluster determines a synchronization time slice based on master-slave state switching of storage management unit groups in the distributed cluster; generating a data operation log of a main storage management unit in the storage management unit group in a synchronous time slice; the data operation log is synchronized to the slave storage management units in the storage management unit group so that the slave storage management units perform data synchronization based on the data operation log.
In some of these embodiments, the cluster generates metadata of the master storage management units such that, upon synchronizing the data oplogs to the slave storage management units in the storage management unit group, a data oplog to be synchronized in the data oplogs is determined based on the metadata, wherein the metadata includes a synchronization state of the data oplogs; updating metadata based on synchronization results of data operation logs
In some embodiments, the clusters persist data operation information of the clusters in the distributed clusters according to an execution sequence of the data operations, wherein the data operation information comprises information of an operation object, operation content and a storage management unit for executing the data operations; and acquiring the data operation logs of the main storage management unit in the synchronous time slices according to the data operation information of the clusters, wherein the data operation logs are ordered according to the execution sequence of the data operation.
In some embodiments, the cluster obtains a master-slave state switching logic clock of the storage management unit group, and determines the synchronization time slices and sequence numbers of the synchronization time slices according to the master-slave state switching logic clock.
In some embodiments, when the master-slave state of the storage management unit of the cluster in the distributed cluster is switched, the cluster reports the master-slave state switching logic clock to the storage system, and the storage system sequentially and automatically increases the serial numbers of the synchronous time slices according to the master-slave state switching logic clock and then sends the serial numbers to the distributed cluster.
In some of these embodiments, the distributed cluster periodically obtains the sequence numbers of the synchronized slots from the storage system.
In some embodiments, the cluster determines the data operation log to be synchronized in the master storage management unit according to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the serial number of the synchronization time slice, and synchronizes the data operation log to be synchronized to the slave storage management unit.
In some embodiments, the cluster determines the data operation log to be synchronized and performs synchronization of the data operation log to be synchronized under the condition that the serial numbers of the synchronization time slices of the master storage management unit and the slave storage management unit are the same.
In some embodiments, the distributed cluster system further includes a preset controller, wherein the preset controller is configured to control the master-slave state switching of the storage management unit, and record the master-slave state of the storage management unit.
In some embodiments, in the initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, the storage management units in the slave cluster in the distributed cluster are all slave storage management units, and master-slave switching of the storage management unit group is managed by a preset controller.
In some embodiments, the preset controller sets the storage management unit with the identifier as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.
In some of these embodiments, in the event that there is data to be written to the storage management unit group, the data is written to the primary storage management unit by querying a preset controller to determine the primary storage management unit in the storage management unit group.
In some embodiments, the data storage state of the storage management unit is used as a finite state machine, a leader node and at least one follower node are respectively elected in the master storage management unit and the slave storage management unit, and the leader node performs copy synchronization of the finite state machine in the storage management unit group based on the data operation log.
The embodiment of the invention also provides electronic equipment, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor. The memory stores a computer program executable by the at least one processor, which when executed by the at least one processor is adapted to cause an electronic device to perform a method of an embodiment of the invention.
The embodiments of the present invention also provide a non-transitory machine-readable medium storing a computer program, wherein the computer program is configured to cause a computer to perform the method of the embodiments of the present invention when executed by a processor of the computer.
The embodiments of the present invention also provide a computer program product comprising a computer program, wherein the computer program, when being executed by a processor of a computer, is for causing the computer to perform the method of the embodiments of the present invention.
With reference to fig. 9, a block diagram of an electronic device that may be a server or a client of an embodiment of the present invention will now be described, which is an example of a hardware device that may be applied to aspects of the present invention. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 9, the electronic device includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM903, various programs and data required for the operation of the electronic device can also be stored. The computing unit 901, the ROM 902, and the RAM903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.
A number of components in the electronic device are connected to the I/O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 may be any type of device capable of inputting information to an electronic device, and the input unit 906 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device. The output unit 907 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 908 may include, but is not limited to, magnetic disks, optical disks. The communication unit 909 allows the electronic device to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.
The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a CPU, a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above. For example, in some embodiments, method embodiments of the present invention may be implemented as a computer program tangibly embodied on a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device via the ROM 902 and/or the communication unit 909. In some embodiments, the computing unit 901 may be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).
A computer program for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of embodiments of the present invention, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It should be noted that the term "comprising" and its variants as used in the embodiments of the present invention are open-ended, i.e. "including but not limited to". The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. References to "one or more" modifications in the examples of the invention are intended to be illustrative rather than limiting, and it will be understood by those skilled in the art that "one or more" is intended to be interpreted as "one or more" unless the context clearly indicates otherwise.
User information (including but not limited to user equipment information, user personal information and the like) and data (including but not limited to data for analysis, stored data, presented data and the like) according to the embodiment of the invention are information and data authorized by a user or fully authorized by all parties, and the collection, use and processing of related data are required to comply with related laws and regulations and standards of related countries and regions, and are provided with corresponding operation entrances for users to select authorization or rejection.
The steps described in the method embodiments provided in the embodiments of the present invention may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the invention is not limited in this respect.
The term "embodiment" in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive. The various embodiments in this specification are described in a related manner, with identical and similar parts being referred to each other. In particular, for apparatus, devices, system embodiments, the description is relatively simple as it is substantially similar to method embodiments, see for relevant part of the description of method embodiments.
The above examples merely represent a few embodiments of the present invention, which are described in more detail and are not to be construed as limiting the scope of the patent claims. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of the invention should be assessed as that of the appended claims.
Claims (16)
1. A method of data synchronization for a distributed cluster, comprising:
determining a synchronous time slice based on master-slave state switching of a storage management unit group in the distributed cluster;
generating a data operation log of a main storage management unit in the storage management unit group in a synchronous time slice;
and synchronizing the data operation log to a slave storage management unit in the storage management unit group so that the slave storage management unit performs data synchronization based on the data operation log.
2. The method of claim 1, wherein the method further comprises:
generating metadata of the master storage management unit so that when the data operation log is synchronized to a slave storage management unit in the storage management unit group, determining a data operation log to be synchronized in the data operation log based on the metadata, wherein the metadata comprises a synchronization state of the data operation log;
And updating the metadata based on the synchronization result of the data operation log.
3. The method of claim 1, wherein generating a data operation log for a master storage management unit in the storage management unit group within a synchronization time slice comprises:
persistence of data operation information of clusters in the distributed clusters according to an execution sequence of data operation, wherein the data operation information comprises information of an operation object, operation content and a storage management unit for executing the data operation;
and acquiring the data operation logs of the main storage management unit in the synchronous time slices according to the data operation information of the clusters, wherein the data operation logs are ordered according to the execution sequence of the data operation.
4. The method of claim 1, wherein determining a synchronization time slice based on master-slave state switching of storage management unit groups in a distributed cluster comprises:
and acquiring a master-slave state switching logic clock of the storage management unit group, and determining the synchronous time slices and the serial numbers of the synchronous time slices according to the master-slave state switching logic clock.
5. The method of claim 1, wherein the method further comprises:
When the master-slave state of the storage management unit of the cluster in the distributed cluster is switched, the cluster reports the master-slave state switching logic clock to a storage system, and the storage system sequentially and automatically increases the serial numbers of the synchronous time slices according to the master-slave state switching logic clock and then sends the serial numbers to the distributed cluster.
6. The method of claim 5, wherein the method further comprises:
the distributed cluster periodically acquires the sequence numbers of the synchronous time slices from the storage system.
7. The method of claim 2, wherein synchronizing the data operation log to a slave storage management unit in the storage management unit group comprises:
and determining the data operation log to be synchronized in the master storage management unit according to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the serial number of the synchronization time slice, and synchronizing the data operation log to be synchronized to the slave storage management unit.
8. The method of claim 7, wherein the data oplog to be synchronized is determined and synchronization of the data oplog to be synchronized is performed in the event that sequence numbers of synchronization time slices of the master storage management unit and the slave storage management unit are the same.
9. The method of claim 1, wherein in an initial state, storage management units in a master cluster in the distributed cluster are all master storage management units, storage management units in a slave cluster in the distributed cluster are all slave storage management units, and master-slave switching of the storage management unit group is managed by a preset controller.
10. The method according to claim 9, wherein the preset controller sets the storage management unit with the identifier as a master storage management unit by adding the identifier of the storage management unit to a gray list, so as to realize master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.
11. The method of claim 9, wherein, in the event that there is data to be written to the storage management unit group, determining a primary storage management unit in the storage management unit group by querying the preset controller writes data to the primary storage management unit.
12. The method of claim 1, wherein the method further comprises:
and taking the data storage state of the storage management unit as a finite state machine, respectively electing a leader node and at least one follower node in the master storage management unit and the slave storage management unit, and carrying out copy synchronization of the finite state machine in the storage management unit group by the leader node based on the data operation log.
13. A distributed cluster system comprising a plurality of clusters including a master cluster and at least one slave cluster, and a storage system connected to the plurality of clusters, respectively;
the storage system is used for storing the sequence numbers of the synchronous time slices; the plurality of clusters are each data synchronized by the method of any one of claims 1 to 12.
14. The distributed cluster system of claim 13, wherein the distributed cluster system further comprises a preset controller for controlling the master-slave state switching of the storage management unit and recording the master-slave state of the storage management unit.
15. An electronic device, comprising: a processor, and a memory storing a program, wherein the program comprises instructions that when executed by the processor cause the processor to perform the method of any one of claims 1 to 12.
16. A non-transitory machine readable medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310576529.7A CN117112690A (en) | 2023-05-19 | 2023-05-19 | Data synchronization method of distributed cluster and related equipment thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310576529.7A CN117112690A (en) | 2023-05-19 | 2023-05-19 | Data synchronization method of distributed cluster and related equipment thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117112690A true CN117112690A (en) | 2023-11-24 |
Family
ID=88795436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310576529.7A Pending CN117112690A (en) | 2023-05-19 | 2023-05-19 | Data synchronization method of distributed cluster and related equipment thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117112690A (en) |
-
2023
- 2023-05-19 CN CN202310576529.7A patent/CN117112690A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109558215B (en) | Backup method, recovery method and device of virtual machine and backup server cluster | |
| US10895996B2 (en) | Data synchronization method, system, and apparatus using a work log for synchronizing data greater than a threshold value | |
| US11221995B2 (en) | Data replication from a cloud-based storage resource | |
| US9934242B2 (en) | Replication of data between mirrored data sites | |
| WO2013152058A2 (en) | Telemetry system for a cloud synchronization system | |
| CN107018185B (en) | Synchronization method and device of cloud storage system | |
| US10489378B2 (en) | Detection and resolution of conflicts in data synchronization | |
| CN112955874A (en) | System and method for self-healing in decentralized model building using machine learning of blockchains | |
| US8612799B2 (en) | Method and apparatus of backing up subversion repository | |
| CN111338834B (en) | Data storage method and device | |
| CN114238495A (en) | Method and device for switching main cluster and standby cluster of database, computer equipment and storage medium | |
| WO2021212493A1 (en) | Data synchronization method and device, data storage system and computer-readable medium | |
| CN112069152A (en) | Database cluster upgrading method, device, equipment and storage medium | |
| CN108881452B (en) | Data synchronization method, device and storage medium | |
| CN113515574B (en) | Data synchronization method and device | |
| WO2022227719A1 (en) | Data backup method and system, and related device | |
| CN117112690A (en) | Data synchronization method of distributed cluster and related equipment thereof | |
| WO2022121387A1 (en) | Data storage method and apparatus, server, and medium | |
| CN111147226A (en) | Data storage method, device and storage medium | |
| CN111522688B (en) | Data backup method and device for distributed system | |
| CN114981796A (en) | Data synchronization method, data synchronization device, data storage system and computer readable medium | |
| US7644306B2 (en) | Method and system for synchronous operation of an application by a purality of processing units | |
| CN111142921A (en) | Software upgrading method and device | |
| CN112732809B (en) | ETL system and data processing method based on ETL system | |
| US20230138900A1 (en) | Systems and methods for managing cloud environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |