CN108055343B - Data synchronization method and device for computer room - Google Patents
Data synchronization method and device for computer room Download PDFInfo
- Publication number
- CN108055343B CN108055343B CN201711431177.7A CN201711431177A CN108055343B CN 108055343 B CN108055343 B CN 108055343B CN 201711431177 A CN201711431177 A CN 201711431177A CN 108055343 B CN108055343 B CN 108055343B
- Authority
- CN
- China
- Prior art keywords
- updating
- request
- update
- message
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004806 packaging method and process Methods 0.000 claims abstract description 11
- 238000004891 communication Methods 0.000 claims description 13
- 238000005538 encapsulation Methods 0.000 claims description 8
- 230000001360 synchronised effect Effects 0.000 abstract description 7
- 230000002457 bidirectional effect Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000003139 buffering effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a data synchronization method and device for a computer room. Wherein, the method comprises the following steps: the first machine room executes a plurality of update operations corresponding to the received update request; packaging a plurality of operation records corresponding to a plurality of updating operations into a piece of message data according to an execution sequence; and storing the message data into a preset message queue so that the second machine room can carry out data synchronization according to each message data stored in the message queue. Therefore, the scheme provided by the invention realizes the bidirectional data synchronous operation among a plurality of computer rooms. In addition, a plurality of operation records corresponding to the same updating request can be encapsulated into a piece of message data according to the execution sequence, thereby preventing the disorder condition and ensuring the correctness of the synchronization process.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data synchronization method and device for a computer room.
Background
With the increasing popularization of network services, the access volume of users is larger and larger, if all the service data are stored in one server, the server often fails due to the large access volume, and due to the limitation of the storage space of the server, in an application scenario with a large service data volume, one server often cannot bear all the service data, so that a plurality of servers are required to provide services for the users together. For this purpose, a plurality of servers commonly used for providing a certain service may be provided in the same machine room, and the machine room provides the service to the user. In addition, in practical applications, it is often necessary to store some common service data in multiple computer rooms simultaneously.
In the above situation, the data in multiple rooms needs to be kept consistent, and once the data in each room is inconsistent, the normal use of the user is affected. Since the service data is changed frequently, once the data in a certain machine room is changed, the data is inconsistent because other machine rooms are not updated synchronously in time. In the prior art, data synchronization among a plurality of computer rooms is generally realized by adopting a one-way synchronization mode, in which data can be synchronized only from a main computer room to a slave computer room, but data cannot be synchronized from the computer room to the main computer room.
However, the inventor finds that the above mode in the prior art has at least the following defects in the process of implementing the invention: since data can be synchronized only from the master room to the slave room, the updated data corresponding to the slave room cannot be synchronized to the master room, thereby bringing about the hidden danger of data inconsistency. In order to eliminate this risk, all the update data has to be written into the master room and then synchronized to the respective slave rooms via the master room. However, this tends to cause a drastic increase in the writing pressure of the main room, thereby increasing latency and also increasing the probability of write failure in an intangible manner.
Disclosure of Invention
In view of the above, the present invention is proposed in order to provide a data synchronization method and apparatus for a room that overcomes or at least partially solves the above problems.
According to an aspect of the present invention, there is provided a data synchronization method for a computer room, including: the first machine room executes a plurality of update operations corresponding to the received update request; packaging a plurality of operation records corresponding to a plurality of updating operations into a piece of message data according to an execution sequence; and storing the message data into a preset message queue so that the second machine room can carry out data synchronization according to each message data stored in the message queue.
According to another aspect of the present invention, there is provided a data synchronization apparatus for a computer room, the data synchronization apparatus being provided on a first computer room, comprising:
an update module adapted to perform a plurality of update operations corresponding to the received update request;
the encapsulation module is suitable for encapsulating a plurality of operation records corresponding to a plurality of updating operations into a piece of message data according to an execution sequence;
and the storage module is suitable for storing the message data into a preset message queue so that the second machine room can carry out data synchronization according to each message data stored in the message queue.
According to still another aspect of the present invention, there is provided an electronic apparatus including: the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the data synchronization method for the computer room.
According to still another aspect of the present invention, there is provided a computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the data synchronization method for a computer room as described above.
According to the data synchronization method and device for the machine rooms, when the first machine room executes a plurality of updating operations corresponding to the received updating request, a plurality of operation records corresponding to the updating operations are further packaged into a message data according to the execution sequence, and the message data is stored in a preset message queue so that the second machine room can perform data synchronization according to the message data stored in the message queue. Therefore, in the embodiment, a message queue for caching messages is additionally arranged between the first computer room and the second computer room, so that bidirectional data synchronization operation among a plurality of computer rooms is realized. In addition, because a plurality of update operations corresponding to one update request have a strict sequence relationship, in order to prevent the execution sequence of the plurality of update operations corresponding to the same update request from being influenced by the disorder of the message data in the message queue, a plurality of operation records corresponding to the same update request are packaged into one message data according to the execution sequence, thereby preventing the disorder and ensuring the correctness of the synchronization process.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 shows a flow chart of a data synchronization method for a computer room according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a data synchronization method for a computer room according to another embodiment of the present invention;
fig. 3 is a diagram illustrating an apparatus structure of a data synchronization apparatus for a computer room according to another embodiment of the present invention;
fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Fig. 1 is a schematic flow chart illustrating a data synchronization method for a computer room according to an embodiment of the present invention. As shown in fig. 1, the method includes:
step S110: the first machine room performs a plurality of update operations corresponding to the received update request.
The update request received by the first machine room may be sent by an end user or an operation and maintenance user. Moreover, the type of the update request may include a plurality of types, which depend on the service type, and the invention is not limited to the specific details. For example, the update request may be a comment-type update request for adding a comment of a user, or may be an approval-type update request for implementing an approval function.
Wherein, one update request often corresponds to a plurality of update operations. Specifically, a plurality of update operations corresponding to one update request may be directly included in the request parameters of the update request, so that the corresponding plurality of update operations are determined by analyzing the update request. Alternatively, a plurality of update operations corresponding to one update request may also be preset and stored in the request execution rule, so that the corresponding plurality of update operations are determined by querying the preset request execution rule.
In the two manners, the user only needs to trigger the update request once, and the background automatically triggers a plurality of corresponding update operations, for example, taking the like as an example, the user only needs to trigger the like update request once, and the background automatically executes the following two update operations according to the preset request execution rule: (1) modifying the display state of the network content corresponding to the praise operation; (2) an approval notification message is sent to the user associated with the network content. Accordingly, in this step, the two update operations are sequentially executed according to the execution order of the two update operations, so as to complete the update process for the like-approved update request.
In addition, in other implementations, an update request may require a user to perform multiple trigger operations, each trigger operation corresponding to an update sub-request, and each update sub-request corresponding to one or more update operations. For example, taking a comment class update request as an example, a user needs to sequentially perform the following trigger operations: (1) clicking a comment entrance below the network content to enable the first machine room to receive a first updating sub-request contained in the comment type updating request; (2) inputting corresponding comment content in the comment input box so that the first machine room receives a second updating sub-request contained in the comment type updating request; (3) and clicking a submission inlet for confirming submission of the comment to submit the comment, and enabling the first machine room to receive a third update sub-request contained in the comment type update request. Each of the update sub-requests described above corresponds to at least one update operation. For example, the first update sub-request corresponds to an update operation of the display comment input box; the second update sub-request corresponds to an update operation of the display comment content; the third update sub-request corresponds to an update operation of modifying a comment status of the web content, and an update operation of sending a notification message to a user terminal related to the web content. In this manner, a plurality of update sub-requests having the same execution purpose are determined as one update request.
Therefore, the present invention does not limit the specific connotation and implementation manner of the update request, nor the specific type and implementation manner of each update operation corresponding to one update request.
Step S120: and packaging a plurality of operation records corresponding to a plurality of updating operations into a piece of message data according to the execution sequence.
Among them, there is often a strict execution order among a plurality of update operations corresponding to one update request, and in step S110, it is necessary to execute each update operation in turn according to the execution order. Accordingly, in this step, it is also necessary to encapsulate a plurality of operation records into one piece of message data in the execution order.
Specifically, when an update request only needs to be triggered by a user once, the execution sequence between multiple update operations corresponding to the update request may be directly included in the request parameter of the update request, or may be preset and stored in the request execution rule; when an update request requires a user to trigger multiple operations, multiple update operations corresponding to the update request may be determined according to the trigger time of the multiple trigger operations. It can be seen that the present invention does not limit the specific determination manner of the execution order between the respective update operations. In addition, each updating operation corresponds to an operation record for recording the operation object, the operation type and other related information of the updating operation, so that the updating operation is synchronously implemented according to the operation record. In summary, the present invention needs to encapsulate each operation record corresponding to each update operation belonging to the same update request into one message data. And, ensure each operation record in this message data and carry on the encapsulation according to the execution sequence, in order to prevent the emergence of the out-of-order phenomenon.
Step S130: and storing the message data into a preset message queue so that the second machine room can carry out data synchronization according to each message data stored in the message queue.
The message queue may be a message queue in various forms as long as the message data buffering function can be implemented. And storing each message data generated in the above steps into the message queue, wherein the second machine room is used as a consumption end of the message queue and is responsible for consuming each message data in the message queue, and realizing data synchronization with the first machine room according to the consumed message data.
According to the above description, whenever there is data update in the first machine room, the message queue is updated correspondingly, and the second machine room implements the data synchronization function by consuming the message data in the message queue. The message queue serves as an intermediary between the first computer room and the second computer room, so that a bidirectional synchronization function among a plurality of computer rooms can be realized.
Therefore, in the embodiment, a message queue for caching messages is additionally arranged between the first computer room and the second computer room, so that bidirectional data synchronization operation among a plurality of computer rooms is realized. In addition, because a plurality of update operations corresponding to one update request have a strict sequence relationship, in order to prevent the execution sequence of the plurality of update operations corresponding to the same update request from being influenced by the disorder of the message data in the message queue, a plurality of operation records corresponding to the same update request are packaged into one message data according to the execution sequence, thereby preventing the disorder and ensuring the correctness of the synchronization process.
Fig. 2 is a schematic flow chart of a data synchronization method for a computer room according to another embodiment of the present invention. As shown in fig. 2, the method includes:
step S210: the first machine room determines the operation type and the execution sequence of each updating operation corresponding to the received updating request, and executes each updating operation in sequence according to the operation type and the execution sequence of each updating operation.
The update request received by the first machine room may be sent by an end user or an operation and maintenance user. Moreover, the type of the update request may include a plurality of types, which depend on the service type, and the invention is not limited to the specific details. For example, the update request may be a comment-type update request for adding a comment of a user, or may be an approval-type update request for implementing an approval function. Wherein, one update request often corresponds to a plurality of update operations. In this step, the operation type and execution sequence of each update operation corresponding to one update request need to be determined:
specifically, in the first implementation manner, the operation types and the execution orders of the plurality of update operations corresponding to one update request may be directly included in the request parameters of the update request, so that the corresponding plurality of update operations are determined by parsing the update request. For example, when one update request corresponds to three update operations, at least three request parameters respectively corresponding to the respective update operations are contained in the update request, and the operation type and execution order of the corresponding update operation can be determined by each request parameter.
Alternatively, in the second implementation manner, the operation types and the execution orders of the plurality of update operations corresponding to one update request may also be preset and stored in the request execution rule, so that the corresponding plurality of update operations are determined by querying the preset request execution rule. In a specific implementation, first, a request type of a received update request is determined, for example, the type of the update request may be determined according to a request packet header or a request parameter of the update request. Then, the operation type and the execution sequence of each update operation corresponding to the request type are determined according to the preset request execution rule. For example, request execution rules corresponding to various request types are stored in advance, and by querying the request execution rules of the corresponding types, the operation types and the execution order of the respective update operations corresponding to the request types can be determined. For example, it may be preset that the update operation corresponding to the like update request includes "modify the display state of the network content corresponding to the like operation" and "send a like notification message to the user related to the network content". Of course, the method for sending the approval notification message may further include two parallel implementation methods of the WeChat notification and the email notification. As can be seen, the number and types of update operations corresponding to various types of update requests may be preconfigured.
In the two manners, the user only needs to trigger the update request once, and the background automatically triggers a plurality of corresponding update operations, for example, taking the like as an example, the user only needs to trigger the like update request once, and the background automatically executes the following two update operations according to the preset request execution rule: (1) modifying the display state of the network content corresponding to the praise operation; (2) an approval notification message is sent to the user associated with the network content. The operation type of the first updating operation is a display type, and the execution sequence is executed in advance; the operation type of the second updating operation is a notification type, and the execution sequence is executed later. Accordingly, in this step, the update operation of the first display type is performed first, and then the update operation of the second notification type is performed, so as to complete the update process for the compliment update request.
In addition, in other implementations, an update request may require a user to perform multiple trigger operations, each trigger operation corresponding to an update sub-request, and each update sub-request corresponding to one or more update operations. For example, taking a comment class update request as an example, a user needs to sequentially perform the following trigger operations: (1) clicking a comment entrance below the network content to enable the first machine room to receive a first updating sub-request contained in the comment type updating request; (2) inputting corresponding comment content in the comment input box so that the first machine room receives a second updating sub-request contained in the comment type updating request; (3) and clicking a submission inlet for confirming submission of the comment to submit the comment, and enabling the first machine room to receive a third update sub-request contained in the comment type update request. Each of the update sub-requests described above corresponds to at least one update operation. For example, the first update sub-request corresponds to a first update operation of displaying a comment input box; the second update sub-request corresponds to a second update operation of displaying the comment content; the third update sub-request corresponds to a third update operation of modifying a comment status of the web content, and a fourth update operation of sending a notification message to a user terminal related to the web content. In this manner, a plurality of update sub-requests having the same execution purpose are determined as one update request. Also, in the present embodiment, in order to facilitate identification of the respective update requests, a unique request identification (e.g., request ID) is assigned to each update request in advance. Correspondingly, each update sub-request and/or each update operation corresponding to the update request carries a corresponding request identifier, so that subsequent encapsulation operation is performed according to the request identifier.
Step S220: and packaging a plurality of operation records corresponding to a plurality of updating operations into a piece of message data according to the execution sequence.
Because a unique request identifier is allocated to each update request in advance in the previous step, in the present step, each update operation corresponding to the same update request is determined directly according to the request identifier included in each update operation; and packaging a plurality of operation records corresponding to each updating operation of the same updating request into a message data according to the execution sequence.
Specifically, when an update request only needs to be triggered by a user once, the execution sequence between multiple update operations corresponding to the update request may be directly included in the request parameter of the update request, or may be preset and stored in the request execution rule; when one update request needs a user to trigger multiple operations, multiple update operations corresponding to the update request can be further determined by combining the trigger time of multiple trigger operations. For example, taking the above comment type update request as an example, the execution order of each update operation corresponding to each update sub-request is determined according to the trigger order of the corresponding update sub-request. Accordingly, the first update operation is performed in an order earlier than the second update operation, and the second update operation is performed in an order earlier than the third update operation. In addition, when the same update sub-request corresponds to a plurality of update operations, the execution order among the plurality of update operations may be determined according to a preset rule. For example, for the third update sub-request, which corresponds to a third update operation of modifying the comment status of the web content and a fourth update operation of sending a notification message to the user terminal related to the web content, the execution order between the two update operations may be determined according to a preset rule.
It can be seen that the present invention does not limit the specific determination manner of the execution order between the respective update operations. In addition, each updating operation corresponds to an operation record for recording the operation object, the operation type and other related information of the updating operation, so that the updating operation is synchronously implemented according to the operation record. In short, the present invention needs to encapsulate each operation record corresponding to each update operation belonging to the same update request into one message data according to the request identifier. And, ensure each operation record in this message data and carry on the encapsulation according to the execution sequence, in order to prevent the emergence of the out-of-order phenomenon.
Step S230: and storing the message data into a storage file contained in a preset storage position, and updating a preset message queue according to the storage file contained in the preset storage position so as to store the message data into the message queue.
Specifically, the message data may be stored in a preset folder under the preset path, each piece of message data may be recorded by a storage file in the folder, and the preset message queue may be updated according to the storage file. The message queue may be a message queue in various forms as long as the message data buffering function can be implemented. For example, the method can be a distributed message queue to realize the parallelized storage of data and improve the storage efficiency and the storage capacity. And storing each message data generated in the above steps into the message queue so that the second machine room can perform data synchronization according to each message data stored in the message queue. The second machine room is used as a consuming end of the message queue and is responsible for consuming each message data in the message queue, and data synchronization is realized with the first machine room according to the consumed message data.
In specific implementation, it is considered that the updating operation in the machine room is continuously performed, which leads to the continuous increase of the size of the storage file, so that the storage file contained in the preset storage location needs to be sorted according to the preset file sorting rule at preset time intervals to avoid the occurrence of insufficient storage space, so as to save storage space. In this embodiment, assuming that the storage file included in the preset storage location is a first file (specifically, includes a plurality of files) corresponding to a first file name at a first time (for example, at the beginning of the morning of the first day), the first file corresponding to the first file name is renamed to a second file name, so as to obtain a second file corresponding to the second file name, where the file content of the second file corresponding to the second file name is the same as the content of the first file corresponding to the first file name. And then, generating a third file corresponding to the first file name at a preset storage position, wherein the file name of the third file is the first file name, but the third file does not contain any content when being generated and is an empty file. After the sorting operation is executed, when the message data are stored to the preset storage position again, the message data to be stored are written into the third file, and therefore the message data are prevented from being lost due to insufficient file storage space. Similarly, at a second time (for example, at the beginning of the morning on the next day), the storage files included in the preset storage location include a third file corresponding to the first file name, the third file corresponding to the first file name is renamed to the third file name, a fourth file corresponding to the third file name is obtained, and the file content of the fourth file corresponding to the third file name is the same as the content of the third file corresponding to the first file name. And then, generating a fifth file corresponding to the first file name at a preset storage position, wherein the file name of the fifth file is the first file name, but the fifth file does not contain any content when being generated and is an empty file. After the sorting operation is executed, when the message data are stored to the preset storage position again, the message data to be stored are written into the fifth file, and therefore the message data are prevented from being lost due to insufficient file storage space. Therefore, through the operation, the original storage file is renamed continuously, and a new storage file is generated, so that the subsequent message data can be written into the new storage file. In addition, after a period of time, each renamed file can be deleted or removed, so that the disk space is saved. For example, assuming that an original file stored in a preset storage location is an a.log file, after a preset time interval, the original a.log file is renamed to be a b.log file, and an empty a.log file is regenerated, so that subsequent message data is stored in the empty a.log file. Accordingly, in this step, when the message queue is updated according to the storage file included in the preset storage location, the system notification message for indicating the file change condition needs to be read, the sorted storage file included in the preset storage location is determined according to the system notification message, and the message queue is updated according to the sorted storage file. That is, after the name of the file is changed, in order to prevent a reading error, a system notification message for indicating the file change condition needs to be subscribed so as to know the change condition of the file name in time, so that the message queue is updated according to the file corresponding to the changed file name. Of course, in addition to the way of obtaining the file name change condition through the system notification message, the file name change condition of each storage file contained in the preset storage location may be regularly checked.
Step S240: and the second computer room performs data synchronization according to each message data stored in the message queue.
Specifically, first, the second machine room obtains a piece of message data in a message queue, and determines a plurality of operation records contained in the message data; then, an update transaction corresponding to the message data is created; wherein, the updating affair comprises a plurality of updating operations which are in one-to-one correspondence with a plurality of operation records contained in the message data; finally, a data synchronization operation corresponding to the message data is performed through the update transaction.
For example, assuming that two operation records of "modify the display state of the network content corresponding to the approval operation" and "send the approval notification message to the user related to the network content" are contained in one piece of message data, accordingly, an update transaction corresponding to the message data is created, and the update transaction contains a first update operation for performing "modify the display state of the network content corresponding to the approval operation" and a second update operation for performing "send the approval notification message to the user related to the network content". When the updating operation is executed, the transaction is submitted only when the first updating operation and the second updating operation contained in the updating transaction are executed successfully; and rolling back the transaction when the first updating operation and/or the second updating operation contained in the updating transaction fails to execute. By means of the transaction mode, the consistency of the data in the database is ensured, and data errors caused by the fact that one operation in the transaction is successfully executed and the other operation is failed are avoided.
In this embodiment, the first computer room and/or the second computer room may be a Redis storage computer room, and the number of the second computer rooms may be plural. When the number of the second machine rooms is multiple, the consumption ends equivalent to the message queues are multiple, and the consumption ends consume the data in the message queues independently, so that data synchronization among the machine rooms is realized. In addition, the first machine room and the second machine room can be interchanged. For example, assuming that three machine rooms are provided, when data update occurs in the machine room a, the machine room a serves as a first machine room, and the machine rooms B and C serve as second machine rooms; when data updating occurs in the machine room B, the machine room B serves as a first machine room, and the machine room A and the machine room C serve as a second machine room; when data updating occurs in the machine room C, the machine room C serves as a first machine room, and the machine room B and the machine room A serve as a second machine room. In the above case, the message queue may be one or multiple, and in short, the function of synchronizing data in two directions among multiple computer rooms can be realized through the message queue. As can be seen from this, the message queue stores not only the message data generated when the first machine room is updated, but also the message data generated when the second machine room is updated, and therefore, this embodiment may further include the following steps: and the first machine room acquires message data generated when the second machine room updates in the message queue, and performs data synchronization according to the acquired message data. Correspondingly, each computer room may include a synchronization module, configured to acquire message data written in the message queue by another computer room, so as to implement synchronization with the other computer rooms. At this time, in order to distinguish which computer room generates each message data, in this embodiment, a computer room identifier may be further carried in the message data, so that each computer room queries message data that is not the computer room and performs synchronization.
In summary, according to the method provided by the invention, when updating is needed, one machine room can be selected nearby for updating, and then the updated data is synchronized to other machine rooms through the machine room. All updating operations are not required to be completed in one host computer room in a centralized mode like the prior art, so that data writing pressure of all computer rooms is dispersed, operation delay is reduced, and writing errors caused by data congestion are prevented. In addition, because a plurality of update operations corresponding to one update request have a strict sequence relationship, in order to prevent the execution sequence of the plurality of update operations corresponding to the same update request from being influenced by the disorder of the message data in the message queue, a plurality of operation records corresponding to the same update request are packaged into one message data according to the execution sequence, thereby preventing the disorder and ensuring the correctness of the synchronization process.
Fig. 3 is a device structure diagram of a data synchronization device for a computer room according to another embodiment of the present invention, where the data synchronization device is disposed in a first computer room, and includes:
an update module 31 adapted to perform a plurality of update operations corresponding to the received update request;
an encapsulation module 32, adapted to encapsulate a plurality of operation records corresponding to the plurality of update operations into a piece of message data according to an execution sequence;
the storage module 33 is adapted to store the message data in a preset message queue, so that the second machine room performs data synchronization according to each message data stored in the message queue.
Optionally, the updating module 31 is specifically adapted to: determining the operation type and the execution sequence of each updating operation corresponding to the updating request; and executing the updating operations in sequence according to the operation types and the execution sequence of the updating operations.
Optionally, the updating module 31 is specifically adapted to: determining the request type of the updating request, and determining the operation type and the execution sequence of each updating operation corresponding to the request type according to a preset request execution rule.
Optionally, the encapsulation module 32 is specifically adapted to: allocating a unique request identifier for each updating request in advance; determining each updating operation corresponding to the same updating request according to the request identifier contained in each updating operation; and packaging the operation records corresponding to the updating operations corresponding to the same updating request into a message data according to the execution sequence.
Optionally, the storage module 33 is specifically adapted to: storing the message data into a storage file contained in a preset storage position, and updating the message queue according to the storage file contained in the preset storage position;
and, the apparatus further comprises:
the sorting module 34 is adapted to sort the storage files contained in the preset storage location at preset time intervals according to a preset file sorting rule;
the storage module 33 is specifically adapted to: reading a system notification message for indicating a file change condition, determining a sorted storage file contained in the preset storage position according to the system notification message, and updating the message queue according to the sorted storage file.
Optionally, the first computer room and/or the second computer room are Redis storage computer rooms, and the number of the second computer rooms is multiple.
The specific structure and the working principle of each module may refer to the description of the corresponding step in the method embodiment, and are not described herein again.
The embodiment of the application provides a nonvolatile computer storage medium, wherein at least one executable instruction is stored in the computer storage medium, and the computer executable instruction can execute the data synchronization method for a computer room in any method embodiment.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.
As shown in fig. 4, the electronic device may include: a processor (processor)402, a Communications Interface 404, a memory 406, and a Communications bus 408.
Wherein:
the processor 402, communication interface 404, and memory 406 communicate with each other via a communication bus 408.
A communication interface 404 for communicating with network elements of other devices, such as clients or other servers.
The processor 402 is configured to execute the program 410, and may specifically execute relevant steps in the above-described data synchronization method embodiment for a computer room.
In particular, program 410 may include program code comprising computer operating instructions.
The processor 402 may be a central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 406 for storing a program 410. Memory 406 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 410 may specifically be configured to cause the processor 402 to perform the following operations:
executing a plurality of update operations corresponding to the received update request;
packaging a plurality of operation records corresponding to the plurality of updating operations into a piece of message data according to an execution sequence;
and storing the message data into a preset message queue so that the second machine room can carry out data synchronization according to each message data stored in the message queue.
In an alternative manner, the program 410 may be further specifically configured to cause the processor 402 to perform the following operations:
the first machine room determines the operation type and the execution sequence of each updating operation corresponding to the updating request;
and executing the updating operations in sequence according to the operation types and the execution sequence of the updating operations.
In an alternative manner, the program 410 may be further specifically configured to cause the processor 402 to perform the following operations:
determining the request type of the updating request, and determining the operation type and the execution sequence of each updating operation corresponding to the request type according to a preset request execution rule.
In an alternative manner, the program 410 may be further specifically configured to cause the processor 402 to perform the following operations:
allocating a unique request identifier for each updating request in advance;
determining each updating operation corresponding to the same updating request according to the request identifier contained in each updating operation;
and packaging the operation records corresponding to the updating operations corresponding to the same updating request into a message data according to the execution sequence.
In an alternative manner, the program 410 may be further specifically configured to cause the processor 402 to perform the following operations:
the second computer room obtains a piece of message data in the message queue and determines a plurality of operation records contained in the message data;
creating an update transaction corresponding to the message data; wherein, the updating affair comprises a plurality of updating operations which are in one-to-one correspondence with a plurality of operation records contained in the message data;
performing a data synchronization operation corresponding to the message data through the update transaction.
In an alternative manner, the program 410 may be further specifically configured to cause the processor 402 to perform the following operations:
storing the message data into a storage file contained in a preset storage position, and updating the message queue according to the storage file contained in the preset storage position;
sorting the storage files contained in the preset storage positions at preset time intervals according to a preset file sorting rule;
reading a system notification message for indicating a file change condition, determining a sorted storage file contained in the preset storage position according to the system notification message, and updating the message queue according to the sorted storage file.
The first machine room and/or the second machine room are Redis storage machine rooms, and the number of the second machine rooms is multiple.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of a data synchronization apparatus for a room according to an embodiment of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (9)
1. A data synchronization method for a computer room comprises the following steps:
the first machine room executes a plurality of update operations corresponding to the received update request;
packaging a plurality of operation records corresponding to the plurality of updating operations into a piece of message data according to an execution sequence;
storing the message data into a preset message queue for a second machine room to perform data synchronization according to each message data stored in the message queue;
the step of encapsulating the plurality of operation records corresponding to the plurality of update operations into one piece of message data according to the execution sequence specifically includes:
allocating a unique request identifier for each updating request in advance;
determining each updating operation corresponding to the same updating request according to the request identifier contained in each updating operation;
packaging a plurality of operation records corresponding to each updating operation corresponding to the same updating request into a message data according to an execution sequence;
the step of the first machine room executing a plurality of update operations corresponding to the received update request specifically includes:
the first machine room determines the operation type and the execution sequence of each updating operation corresponding to the updating request;
executing each updating operation in sequence according to the operation type and the execution sequence of each updating operation;
the step of determining, by the first machine room, the operation type and the execution order of each update operation corresponding to the update request specifically includes:
determining the request type of the updating request, and determining the operation type and the execution sequence of each updating operation corresponding to the request type according to a preset request execution rule.
2. The method according to claim 1, wherein the step of performing data synchronization by the second computer room according to each message data stored in the message queue specifically includes:
the second computer room obtains a piece of message data in the message queue and determines a plurality of operation records contained in the message data;
creating an update transaction corresponding to the message data; wherein, the updating affair comprises a plurality of updating operations which are in one-to-one correspondence with a plurality of operation records contained in the message data;
performing a data synchronization operation corresponding to the message data through the update transaction.
3. The method according to claim 2, wherein the step of storing the message data in a preset message queue specifically comprises:
storing the message data into a storage file contained in a preset storage position, and updating the message queue according to the storage file contained in the preset storage position;
and, the method further comprises:
sorting the storage files contained in the preset storage positions at preset time intervals according to a preset file sorting rule;
the step of updating the message queue according to the storage file included in the preset storage location specifically includes: reading a system notification message for indicating a file change condition, determining a sorted storage file contained in the preset storage position according to the system notification message, and updating the message queue according to the sorted storage file.
4. A method according to claim 3, wherein the first and/or second room is a Redis storage room and the number of second rooms is plural.
5. A data synchronization device for a machine room, the data synchronization device being provided on a first machine room, comprising:
an update module adapted to perform a plurality of update operations corresponding to the received update request;
the encapsulation module is suitable for encapsulating a plurality of operation records corresponding to the plurality of updating operations into a piece of message data according to an execution sequence;
the storage module is suitable for storing the message data into a preset message queue so that the second machine room can carry out data synchronization according to each message data stored in the message queue;
the encapsulation module is particularly adapted to: allocating a unique request identifier for each updating request in advance; determining each updating operation corresponding to the same updating request according to the request identifier contained in each updating operation; packaging a plurality of operation records corresponding to each updating operation corresponding to the same updating request into a message data according to an execution sequence;
the update module is specifically adapted to: determining the operation type and the execution sequence of each updating operation corresponding to the updating request; executing each updating operation in sequence according to the operation type and the execution sequence of each updating operation;
the update module is specifically adapted to:
determining the request type of the updating request, and determining the operation type and the execution sequence of each updating operation corresponding to the request type according to a preset request execution rule.
6. The apparatus according to claim 5, wherein the storage module is specifically adapted to: storing the message data into a storage file contained in a preset storage position, and updating the message queue according to the storage file contained in the preset storage position;
and, the apparatus further comprises:
the sorting module is suitable for sorting the storage files contained in the preset storage positions at preset time intervals according to a preset file sorting rule;
the storage module is specifically adapted to: reading a system notification message for indicating a file change condition, determining a sorted storage file contained in the preset storage position according to the system notification message, and updating the message queue according to the sorted storage file.
7. The apparatus of claim 6, wherein the first and/or second room is a Redis storage room and the second room is plural in number.
8. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the data synchronization method for the computer room in any one of claims 1-4.
9. A computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the data synchronization method for a computer room according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711431177.7A CN108055343B (en) | 2017-12-26 | 2017-12-26 | Data synchronization method and device for computer room |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711431177.7A CN108055343B (en) | 2017-12-26 | 2017-12-26 | Data synchronization method and device for computer room |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108055343A CN108055343A (en) | 2018-05-18 |
| CN108055343B true CN108055343B (en) | 2021-07-13 |
Family
ID=62131935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711431177.7A Expired - Fee Related CN108055343B (en) | 2017-12-26 | 2017-12-26 | Data synchronization method and device for computer room |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108055343B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108833569A (en) * | 2018-06-27 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of method of data synchronization and relevant apparatus |
| CN109684406A (en) * | 2018-11-13 | 2019-04-26 | 北京达佳互联信息技术有限公司 | Method, apparatus, equipment and the storage medium of redis cluster backup data across computer room |
| TWI678087B (en) * | 2018-11-22 | 2019-11-21 | 財團法人工業技術研究院 | Method of message synchronization in message queue publish and subscriotion and system thereof |
| CN110149382A (en) * | 2019-05-08 | 2019-08-20 | 拉扎斯网络科技（上海）有限公司 | Data synchronous method, system, primary server, sync client and medium |
| CN110381146B (en) * | 2019-07-23 | 2021-09-03 | 腾讯科技（深圳）有限公司 | Batch operation processing method and device and storage medium |
| CN111309693A (en) * | 2020-02-07 | 2020-06-19 | 北京奇艺世纪科技有限公司 | Data synchronization method, device and system, electronic equipment and storage medium |
| WO2021212493A1 (en) * | 2020-04-24 | 2021-10-28 | 深圳市欢太科技有限公司 | Data synchronization method and device, data storage system and computer-readable medium |
| CN111680100B (en) * | 2020-04-30 | 2023-02-28 | 聚好看科技股份有限公司 | Redis data synchronization method and server |
| CN113407528A (en) * | 2020-11-20 | 2021-09-17 | 广东美云智数科技有限公司 | Authority data synchronization method, management device and storage medium |
| CN112764947B (en) * | 2021-01-15 | 2023-12-26 | 百果园技术(新加坡)有限公司 | Message data pulling method, device, equipment and storage medium |
| CN113407637A (en) * | 2021-07-13 | 2021-09-17 | 上海浦东发展银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| US11922026B2 (en) | 2022-02-16 | 2024-03-05 | T-Mobile Usa, Inc. | Preventing data loss in a filesystem by creating duplicates of data in parallel, such as charging data in a wireless telecommunications network |
| CN115174682B (en) * | 2022-06-17 | 2023-07-28 | ***股份有限公司 | Data distribution method, system, center, device and computer readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104113787A (en) * | 2014-05-29 | 2014-10-22 | 腾讯科技（深圳）有限公司 | Program-based commenting method, terminal, server, and program-based commenting system |
| CN104967536A (en) * | 2015-06-29 | 2015-10-07 | 北京奇虎科技有限公司 | Method and device for realizing data consistency of multiple machine rooms |
| CN105677849A (en) * | 2016-01-06 | 2016-06-15 | 北京京东尚科信息技术有限公司 | Data updating method and device |
| CN107357883A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107465767A (en) * | 2017-09-29 | 2017-12-12 | 网宿科技股份有限公司 | A kind of method and system of data syn-chronization |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101200492B1 (en) * | 2009-07-13 | 2012-11-16 | 한국전자통신연구원 | Multimedia Playing Apparatus and its Method |
-
2017
- 2017-12-26 CN CN201711431177.7A patent/CN108055343B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104113787A (en) * | 2014-05-29 | 2014-10-22 | 腾讯科技（深圳）有限公司 | Program-based commenting method, terminal, server, and program-based commenting system |
| CN104967536A (en) * | 2015-06-29 | 2015-10-07 | 北京奇虎科技有限公司 | Method and device for realizing data consistency of multiple machine rooms |
| CN105677849A (en) * | 2016-01-06 | 2016-06-15 | 北京京东尚科信息技术有限公司 | Data updating method and device |
| CN107357883A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107465767A (en) * | 2017-09-29 | 2017-12-12 | 网宿科技股份有限公司 | A kind of method and system of data syn-chronization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108055343A (en) | 2018-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108055343B (en) | Data synchronization method and device for computer room | |
| CN108874558B (en) | Message subscription method of distributed transaction, electronic device and readable storage medium | |
| JP2019503538A (en) | Method and apparatus for presenting interface data | |
| CN110741342A (en) | Blockchain transaction commit ordering | |
| US9590859B2 (en) | Discovering resources of a distributed computing environment | |
| CN109144619B (en) | Icon font information processing method, device and system | |
| CN109254854A (en) | Asynchronous invoking method, computer installation and storage medium | |
| US9262241B2 (en) | Computer system, computer-implemented method and computer program product for sequencing incoming messages for processing at an application | |
| CN111880948A (en) | Data refreshing method and device, electronic equipment and computer readable storage medium | |
| US11237761B2 (en) | Management of multiple physical function nonvolatile memory devices | |
| CN113794764A (en) | Request processing method and medium for server cluster and electronic device | |
| CN111913764A (en) | Service dependency analysis method, electronic device, and storage medium | |
| US11816163B2 (en) | Systems and methods for improved transactional mainframes | |
| CN114327948A (en) | Message processing method, device, equipment and storage medium | |
| US10581997B2 (en) | Techniques for storing or accessing a key-value item | |
| CN113468143A (en) | Data migration method, system, computing device and storage medium | |
| CN109032693A (en) | Method and device for loading display information, electronic equipment and readable storage medium | |
| CN108062224B (en) | Data reading and writing method and device based on file handle and computing equipment | |
| CN112052104A (en) | Message queue management method based on multi-computer-room realization and electronic equipment | |
| CN116303309A (en) | File mounting method and device and electronic equipment | |
| JP5884566B2 (en) | Batch processing system, progress confirmation device, progress confirmation method, and program | |
| CN112860746B (en) | Cache reduction-based method, equipment and system | |
| CN114116676A (en) | Data migration method and device, electronic equipment and computer readable storage medium | |
| CN114675954A (en) | Task scheduling method and device | |
| CN114968216A (en) | Application program development project generation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210713 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |