WO2018107460A1 - 对象复制方法、装置及对象存储设备 - Google Patents

对象复制方法、装置及对象存储设备 Download PDF

Info

Publication number
WO2018107460A1
WO2018107460A1 PCT/CN2016/110314 CN2016110314W WO2018107460A1 WO 2018107460 A1 WO2018107460 A1 WO 2018107460A1 CN 2016110314 W CN2016110314 W CN 2016110314W WO 2018107460 A1 WO2018107460 A1 WO 2018107460A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage device
backup
target
sequence number
objects
Prior art date
Application number
PCT/CN2016/110314
Other languages
English (en)
French (fr)
Inventor
张雷
罗庆超
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2016/110314 priority Critical patent/WO2018107460A1/zh
Priority to CN201680003361.9A priority patent/CN108475254A/zh
Publication of WO2018107460A1 publication Critical patent/WO2018107460A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions

Definitions

  • Embodiments of the present invention relate to the field of storage, and in particular, to an object copying method, apparatus, and object storage device.
  • the object-based storage system (English-based object-based storage system) includes several object-based storage devices (OSDs), which are interconnected through a network. Each object storage device is responsible for writing an object (English: object) to or reading an object from the storage medium.
  • An object is the basic unit of distributed object storage system operations, containing data and attribute information related to the data.
  • EC erasure codes
  • object copying between multiple object storage devices performs the same operation in the same order by controlling each object storage device, and finally obtains the same output with the same input, so that multiple object copies are consistent. status.
  • each object corresponds to an operation log
  • the operation log includes a sequence number.
  • the sequence of the operation log is used to achieve strong consistency of the system.
  • the operation log needs to be read and written separately, Causes disk input/output (I/O) operations More times. For example, before the object storage device writes the object to the storage medium, the operation log corresponding to the object is first written into the storage medium, and then the object is written into the storage medium, that is, the operation log corresponding to the object and the object respectively Write to the storage medium instead of writing to the storage medium at one time.
  • the application provides an object copying method, device and object storage device, which can reduce the number of disk I/O operations.
  • an object replication method is provided, the method being applied to a distributed object storage system, the distributed object storage system comprising a source storage device and a backup storage device.
  • the source storage device receives a plurality of write request messages sent by the client, where each write request message includes an object; the source storage device allocates a sequence number for each object in the write request message, and the sequence number of each object is sequentially incremented; the source storage device Each object is merged with the sequence number of the object to generate a plurality of target objects; the source storage device sends the plurality of target objects to the backup storage device.
  • the backup storage device when the source storage device performs object copying, the object and the sequence number of the object are first combined to obtain the target object, and then the target object is sent to the backup storage device, because the target object received by the backup storage device is a whole. Therefore, the backup storage device does not write the object and object serial number twice to the storage medium, but writes the storage medium once, thus reducing the number of disk I/O operations.
  • the source storage device stores a plurality of target objects.
  • the backup storage device stores a plurality of backup objects, and the plurality of backup objects are a subset of the plurality of target objects.
  • the source storage device receives the synchronization request message sent by the backup storage device, where the synchronization request message includes the sequence number of the target backup object saved in the backup storage device, and the sequence number of the target backup object is greater than the sequence number of any other backup object saved by the backup storage device;
  • the device sends the target object whose sequence number is greater than the sequence number of the target backup object to the backup storage device.
  • the source storage device and the backup storage device perform object synchronization, only the sequence number of the target backup object sent by the backup storage device needs to be greater than the target.
  • the target object of the serial number of the backup object is sent to the backup storage device without log synchronization, so the time required for synchronization is short, and the distributed object storage system performs well.
  • an object replication method is provided, the method being applied to a distributed object storage system, the distributed object storage system comprising a source storage device and a backup storage device.
  • the backup storage device receives multiple backup objects sent by the source storage device, where each backup object includes an object and a sequence number of the object; and the backup storage device stores multiple backup objects.
  • the backup storage device receives multiple backup objects sent by the source storage device, and writes the backup object as a whole to the storage medium at one time, thereby reducing the number of disk I/O operations.
  • the source storage device stores a plurality of target objects
  • the plurality of backup objects are a subset of the plurality of target objects
  • the backup storage device sends a synchronization request message to the source storage device, and the synchronization is performed.
  • the request message includes the sequence number of the target backup object saved in the backup storage device.
  • the sequence number of the target backup object is greater than the sequence number of any other backup object saved by the backup storage device.
  • the sequence number sent by the backup storage device to the source storage device is greater than the sequence number of the target backup object. Target object.
  • the backup storage device when the backup storage device performs object synchronization with the source storage device, it only needs to send the sequence number of the target backup object to the source storage device, and receive the target object sent by the source storage device that is greater than the sequence number of the target backup object, without performing the target object.
  • Log synchronization so the time required for synchronization is short, and the distributed object storage system performs well.
  • an object storage device is provided, the device being applied to a distributed object storage system, the distributed object storage system comprising a source storage device and a backup storage device, wherein the object storage device is located in the source storage device
  • the device includes: a communication module and a processing module; the communication module is configured to receive a plurality of write request messages sent by the client, where each of the write request messages includes an object; and the processing module is configured to: The object in each write request message is assigned a sequence number, and the sequence number of each object is sequentially incremented; each object is merged with the sequence number of the object to generate a plurality of target objects; and the communication module is further configured to The target object is sent to the backup storage device.
  • the device further includes a storage module, where the processing module is further configured to store the multiple target objects by using the storage module.
  • the backup storage device stores a plurality of backup objects, where the plurality of backup objects are a subset of the plurality of target objects
  • the communication module further And receiving, by the backup storage device, a synchronization request message, where the synchronization request message includes a sequence number of the target backup object saved in the backup storage device, where the sequence number of the target backup object is greater than other saved by the backup storage device
  • the serial number of any one of the backup objects; the communication module is further configured to send the target object whose serial number is greater than the sequence number of the target backup object to the backup storage device.
  • a fourth aspect provides an object storage device, where the device is applied to a distributed object storage system, where the distributed object storage system includes a source storage device and a backup storage device, where the object storage device is located in the backup storage device.
  • the device includes: a communication module, a processing module, and a storage module; the communication module is configured to receive multiple backup objects sent by the source storage device, where each of the backup objects includes an object and the object
  • the processing module is configured to store the multiple backup objects by using the storage module.
  • the source storage device stores a plurality of target objects, where the plurality of backup objects are a subset of the plurality of target objects
  • the communication module further uses Sending a synchronization request message to the source storage device, where the synchronization request message includes a sequence number of the target backup object saved in the backup storage device, and the sequence number of the target backup object is greater than any other one saved by the backup storage device.
  • the serial number of the backup object; the communication module is further configured to receive a target object that is sent by the source storage device and whose sequence number is greater than a sequence number of the target backup object.
  • an embodiment of the present invention provides an object storage device, which can implement the functions performed by the source storage device in the foregoing method design, and the functions can be implemented by hardware or by executing corresponding software through hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the structure of the object storage device includes a processor and a communication interface.
  • the processor is configured to support the object storage device to perform the corresponding functions of the above methods.
  • the communication interface is used to support communication between the object storage device and other object storage devices.
  • the object storage device can also include a memory for coupling with the processor that holds the program instructions and data necessary for the object storage device.
  • an embodiment of the present invention provides an object storage device, which can implement functions performed by a backup storage device in the above method design, and the functions can be implemented by hardware, or can execute corresponding software through hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the structure of the object storage device includes a processor and a communication interface configured to support the object storage device to perform a corresponding function in the above method.
  • the communication interface is used to support communication between the object storage device and other object storage devices.
  • the object storage device can also include a memory for coupling with the processor that holds the program instructions and data necessary for the object storage device.
  • an embodiment of the present invention provides a distributed object storage system, where the system includes the source storage device and the backup storage device.
  • an embodiment of the present invention provides a computer storage medium for storing computer software instructions for use in the source storage device, including a program designed to execute the first aspect.
  • an embodiment of the present invention provides a computer storage medium for storing computer software instructions for use in the foregoing backup storage device, including a program designed to execute the second aspect.
  • embodiments of the present invention provide a computer program product comprising instructions that, when executed by a computer, cause the computer to perform the functions performed by the source storage device in the method design of the first aspect above.
  • an embodiment of the present invention provides a computer program product, including instructions, when the program is executed by a computer, the instruction causes a computer to perform the method design of the second aspect The function performed by the backup storage device.
  • FIG. 1 is a schematic structural diagram of a distributed object storage system
  • FIG. 2 is a schematic diagram of a hardware structure of an object storage device
  • FIG. 3 is a schematic diagram of an application scenario of an object copying method according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of communication of an object copying method according to an embodiment of the present invention
  • 4a is a schematic diagram of a method for generating a target object according to an embodiment of the present invention
  • FIG. 4b is a schematic diagram of another method for generating a target object according to an embodiment of the present invention
  • FIG. 4c is a schematic diagram of another method for generating a target object according to an embodiment of the present invention
  • FIG. 5 is a structural diagram of an object storage device according to an embodiment of the present invention.
  • FIG. 6 is a structural diagram of another object storage device according to an embodiment of the present invention.
  • FIG. 1 is a schematic structural diagram of a distributed object storage system.
  • a distributed object storage system includes a plurality of object storage devices that are interconnected through a network.
  • the client sends the object to any one of the object storage devices in the distributed object storage system, the object storage device stores the object, and backs up the object to one or more other object storage devices.
  • the other object storage device receives the object and saves it as a backup object, thereby ensuring the reliability of the object.
  • the distributed object storage system includes three object storage devices, which are an object storage device 1, an object storage device 2, and an object storage device 3, respectively.
  • the distributed object storage system may include only two object storage devices or a plurality of object storage devices.
  • the number of object storage devices included in the distributed object storage system is not limited in the embodiment of the present invention.
  • the reliability of the object is improved in the multi-copy method in the distributed object storage system.
  • the object 1, the object 2, and the object 3 are not only stored on the object storage device 1, but also stored on the object storage device 2 and the object storage device 3. .
  • the object storage device can be a disk array or a flash array or a storage server.
  • Disk Array A storage device that includes at least one controller and multiple disks.
  • a flash array refers to a storage device including at least one controller and a plurality of solid state devices (SSDs).
  • SSDs solid state devices
  • a solid state drive is a flash memory (English: flash memory) chip as a storage medium, also known as a solid state drive (English: solid state drive, SSD).
  • the object storage device may also be a storage server that includes at least one solid state drive.
  • the object storage device includes a controller 211 and one or more solid state disks 222.
  • the controller 211 includes at least an interface 223, an interface 224, a processor 225, and a memory 226.
  • the interface 223 is configured to communicate with the client.
  • the interface 224 is configured to communicate with the solid state hard disk 222.
  • the processor 225 may be a central processing unit (CPU).
  • the processor 225 is configured to receive an object from the client, process the object, for example, assign a sequence number to the object, combine the object with the sequence number of the object to generate a target object, and send the target object to the backup storage device.
  • the processor 225 can also send the target object to the solid state hard disk 222.
  • the memory 226 is configured to store a program. In addition, the memory 226 is also used to temporarily store write data requests received from the client or data read from the solid state hard disk 222.
  • the controller 221 receives the object sent by the client, temporarily stores the object in the memory 226, and the processor 225 assigns a sequence number to the object, combines the object with the sequence number of the object to generate a target object, and temporarily stores the target object in the memory 226. in. The target object is then read from memory 226 and sent to other object storage devices.
  • the memory 226 includes a random access memory (English: random-access memory, RAM).
  • the memory 226 further includes a non-volatile memory, such as at least one magnetic memory. It can be understood that the memory 226 can be a random memory (RAM), a magnetic disk, a hard disk, a solid state disk (SSD), or a non-volatile memory, and the like, which can store program code. Read the media.
  • Object storage devices can be classified into source storage devices and backup storage devices according to roles.
  • the source storage device is the initiator of the copy; the backup storage device is the executor of the copy. If each object storage device performs writes in the same order, you can guarantee a copy of the object stored by each object storage device Consistency, the above sequence can be understood as a chronological order. For example, the receiving time of the object 1 in the source storage device is earlier than the receiving time of the object 2, and the source storage device stores the object 1 and then the object 2, and similarly, the backup storage. After the device receives the object 1 and the object 2 from the source storage device, it also stores the object 1 and then the object 2.
  • each object may be assigned a serial number, which is used to indicate the order in which the objects are received. In one example, the smaller the serial number, the earlier the object is received, and of course, the embodiment of the present invention does not. Limited to this.
  • the object storage device needs to perform a write operation for the object 1 and a write operation for the object 2. If the sequence number of the object 1 is smaller than the sequence number of the object 2, the source storage device and the backup storage device perform the write operation for the object 1 first, and then The write operation for the object 2 is performed to ensure that the source storage device and the backup storage device store the object copy.
  • the source storage device After receiving the write request message sent by the client, the source storage device not only writes the object locally, but also copies the object carried by the write request message to the backup storage device.
  • An object storage device typically involves two kinds of data when it is stored locally: the object itself and the operation log of the object.
  • the operation log includes an orderly incremented sequence number and operation type information, and is used by the object storage device to perform corresponding operations on the object sequence according to the operation log.
  • the serial number is allocated by the source storage device, and each object storage device uses the serial number to confirm the operation sequence and state of the object.
  • the state refers to whether the object is successfully written. If the object is successfully written, the operation log is saved, so that the serial number exists. If the object is not successfully written, the operation log corresponding to the object is deleted, so that the sequence number does not exist.
  • the processor first stores the object 1 included in the first write request message into the cache, where the cache may be the memory 226 shown in FIG. 2 . Then, the processor generates an operation log 1 for the object 1, and the sequence number generator 1 assigns the sequence number 1 to the object 1, and the processor writes the sequence number 1 into the operation log of the object 1, wherein the sequencer can be understood as the processor. functional module.
  • the source storage device receives the second write request message, similar to the foregoing process, the object 2 included in the second write request message is first stored in the cache.
  • the processor then generates an operation log 2 for the object 2, the value maintained in the sequencer generator is incremented by 1, and the sequence number generator assigns the sequence number 2 to the object 2, and the processor writes the sequence number 2 into the operation log of the object 2.
  • the processor writes the object to the persistent medium, according to the sequence number recorded in the operation log, since the sequence number of the object 1 is smaller than the sequence number of the object 2, the object 1 is first written to the persistent medium, and then the object 2 is In the persistent medium, the persistent medium may be specifically the solid state hard disk 222 shown in FIG. 2.
  • the source storage device sends the operation logs of the object 1 and the object 1 to the backup storage device, and the backup storage device stores the operation log of the object 1 first, and then writes the object 1 to the local disk system, and if the object is successfully written, the backup storage is performed.
  • the operation log of the object 1 is stored in the device.
  • the operation log of the object 1 records the sequence number 1. If the object is not successfully written, the operation log of the object 1 is not saved in the backup storage device, that is, the sequence number 1 is not recorded. .
  • Operation logs and objects need to be stored persistently. Because the operation logs and objects are read and written separately, the number of I/Os is high. In order to improve read and write performance, the operation logs are generally stored on high-performance storage media, for example. NVRAM, while objects are stored on a Hard Disk Drive (HDD).
  • HDD Hard Disk Drive
  • this object copying method it is necessary to separately read and write the object and the operation log of the object, resulting in multiple disk I/O operations, and the performance is poor.
  • high-performance storage such as NVRAM is used.
  • the media stores the operation log, which causes the system cost to be high.
  • the operation log synchronization is first performed in order to perform data synchronization with low performance.
  • An embodiment of the present invention provides an object copying method, which is applied to the distributed object storage system shown in FIG. 1, and combines an object and a serial number to generate a target object, and stores the target object. Since the object and the serial number are combined as a whole, the object and the serial number can be read at one time when the target object is read, and the number of accesses to the disk is reduced.
  • FIG. 3 is a schematic diagram of an application scenario of an object replication method according to an embodiment of the present invention.
  • the method is applied to the source storage device to copy the object to the backup storage device.
  • the object and the object copy stored on each object storage device need to have consistency, and the object has a one-to-one correspondence with the serial number, and the object is identified by the serial number of the object.
  • Consistency of storage objects between storage devices For example, the source storage device records sequence numbers 1 to 10.
  • the backup storage device records sequence numbers 1 to 8, indicating that the source storage device and the device are in standby.
  • the storage device stores objects that are inconsistent.
  • the backup storage device lacks objects corresponding to sequence numbers 9 and 10 compared to the source storage device.
  • the method is applied to a distributed object storage system.
  • the distributed object storage system includes at least a source storage device and a backup storage device, and is based on the object storage shown in FIG. 2 .
  • the specific structure of the device, the method can be performed by the processor 225, the method comprising:
  • Step 401 The source storage device receives a plurality of write request messages sent by the client, where each write request message includes an object.
  • the multiple write request messages may be multiple write request messages sent by one client, or may be write request messages sent by multiple clients.
  • Step 402 The source storage device allocates a sequence number to each object in each write request message, and the sequence number of each object is sequentially incremented.
  • the controller 221 receives the object transmitted by the client, temporarily stores the object in the memory 226, and the processor 225 assigns a serial number to the object, and temporarily stores the serial number in the memory 226.
  • the source storage device allocates a sequence number to each object in each write request message according to the sequence of the received multiple write request messages. For example, the source storage device first receives a write request message for the write request message. The object in the allocation number is 100, and the source storage device then receives a write request message, and assigns a sequence number 101 to the object in the write request message, wherein the sequence number of each object is sequentially incremented, and the incremented value may be but not limited to 1.
  • step 403 the source storage device merges each object with the sequence number of the object to generate a plurality of target objects.
  • the target object can be temporarily saved in the memory 226.
  • the processor 225 can also send the target object to the solid state hard disk 222.
  • the way of merging can be, but is not limited to, the following ways: adding the sequence number of the object to the front of the object header, generating the target object, as shown in FIG. 4a; or adding the sequence number of the object to the object Behind the tail, generate the target object, as shown in Figure 4b; or, divide the object into two parts, object 00 and object 01, add the sequence number of the object between object 00 and object 01, and generate the target object, as shown in Figure 4c. .
  • each object storage device handles an object with a unique incrementing sequence number, which is combined into a target object; each object storage device writes the target object to the persistent medium; in the event of an object storage device failure
  • the checksum can be used to detect whether the object and serial number included in the target object are written to the persistent medium, and the integrity and accuracy of the data are ensured by the checksum.
  • the target object may include only the sequence number of the object and the object, and may also include operation type information, for example, the operation type information is used to identify the write operation.
  • the serial number is first recorded in the operation log, and the object and the serial number are combined to be considered as the object and the contents of the operation log are merged. If the objects on the N object storage devices are the same, they have the same sequence number; otherwise, if the object storage device fails for a period of time, during this time, it will store a small number of objects on it, because the sequence numbers are sequentially incremented, Therefore, the sequence number of the object stored on it is smaller than the sequence number of the object stored on the other object storage device, and the consistency between the plurality of object storage devices can be achieved by the synchronization operation after the failure recovery.
  • Step 404 The source storage device sends multiple target objects to the backup storage device.
  • the source storage device may send multiple target objects to the backup storage device at the same time, and may also send multiple target objects to the backup storage device.
  • Step 405 the backup storage device stores a plurality of target objects.
  • the target object stored by the backup storage device may be referred to as a backup object.
  • the target object can be temporarily saved in the memory 226 of the backup storage device.
  • the processor 225 of the backup storage device can also send the target object to the solid state drive 222 of the backup storage device.
  • the source storage device merges the object with the serial number of the object to obtain the target object, and then sends the target object to the backup storage device. Since the target object received by the backup storage device is a whole, the backup storage device does not. The object and the object's sequence number are written twice The storage medium is written to the storage medium at one time, thereby reducing the number of disk I/O operations.
  • the embodiment of the present invention may further include:
  • step 406 the source storage device stores a plurality of target objects.
  • the target object can be temporarily saved in the memory 226 of the source storage device.
  • the processor 225 of the source storage device can also send the target object to the solid state hard disk 222 of the source storage device.
  • step 406 and step 404 the execution sequence of step 406 and step 404 is not limited, and step 406 and step 404 may be performed at the same time.
  • Step 404 may be performed before step 406, and step 406 may be performed first and then step 404. Only the case where step 404 is performed first and then step 406 is performed is shown in FIG.
  • the embodiment of the present invention may further include:
  • Step 407 The backup storage device sends a synchronization request message to the source storage device, where the synchronization request message includes the sequence number of the target backup object saved in the backup storage device, and the sequence number of the target backup object is greater than the sequence number of any other backup object saved by the backup storage device.
  • the source storage device stores a plurality of target objects
  • the backup storage device stores a plurality of backup objects, wherein the plurality of backup objects are a subset of the plurality of target objects, and the target backup object is a serial number of the plurality of backup objects. The largest backup object.
  • Step 408 The source storage device sends the target object whose sequence number is greater than the sequence number of the target backup object to the backup storage device.
  • the source storage device After receiving the synchronization request message sent by the backup storage device, the source storage device matches the sequence number of each object previously saved according to the sequence number of the target backup object included in the message, and determines that the target object whose sequence number is greater than the sequence number of the target backup object is sent to Back up your storage device.
  • the backup storage device may store the target objects to implement data synchronization with the source storage device.
  • the source storage device receives the first indication sent by the backup storage device
  • the first indication information is used to indicate that the target object is successfully written in the backup storage device
  • the source storage device sends the second indication information to the client
  • the second indication information is used to indicate that the target object is written on both the source storage device and the backup storage device. Successful.
  • the source storage device receives the third indication information sent by the backup storage device, where the third indication information is used to indicate that the target object fails to write in the backup storage device, and the source storage device performs fault tolerance processing according to the third indication information, for example, Mark the backup storage device as abnormal, fault, and enter the fault-tolerant state.
  • the source storage device sends the fourth indication information to the client, where the fourth indication information is used to indicate that the target object is successfully written on the source storage device and fails to be written in the backup storage device.
  • FIG. 5 is a structural diagram of an object storage device according to an embodiment of the present invention.
  • the device is applied to a distributed object storage system, where the distributed object storage system includes a source storage device and a backup storage device, where the device is located in the source storage device.
  • the device includes: a communication module 501 and a processing module 502;
  • the communication module 501 is configured to receive multiple write request messages sent by the client, where each write request message includes an object;
  • the processing module 502 is configured to allocate a sequence number to each object in each write request message, and the sequence numbers of each object are sequentially incremented; each object is combined with the sequence number of the object to generate a plurality of target objects;
  • the communication module 501 is further configured to send multiple target objects to the backup storage device.
  • the device further includes a storage module 503, and the processing module 502 is further configured to store a plurality of target objects through the storage module 503.
  • the backup storage device stores a plurality of backup objects, and the plurality of backup objects are a subset of the plurality of target objects.
  • the communication module 501 is further configured to receive a synchronization request message sent by the backup storage device, where the synchronization request message includes The serial number of the target backup object saved in the backup storage device, the sequence number of the target backup object is greater than the sequence number of any other backup object saved by the backup storage device, and the communication module 501 is further configured to send the target object whose sequence number is greater than the sequence number of the target backup object.
  • the communication module 501, the processing module 502, and the storage module 503 can be implemented by executing the program in the memory 226 by the processor 225 shown in FIG. 2 in actual application.
  • the device The source storage device is used to perform the object copying method provided by the embodiment of the present invention, and the same portions are not described herein.
  • FIG. 6 is a structural diagram of another object storage device according to an embodiment of the present disclosure.
  • the device is applied to a distributed object storage system, where the distributed object storage system includes a source storage device and a backup storage device, where the device is located in the backup storage device.
  • the device includes: a communication module 601, a processing module 602, and a storage module 603;
  • the communication module 601 is configured to receive multiple backup objects sent by the source storage device, where each backup object includes an object and a sequence number of the object;
  • the processing module 602 is configured to store multiple backup objects by using the storage module 603.
  • the source storage device stores a plurality of target objects, and the plurality of backup objects are a subset of the plurality of target objects.
  • the communication module 601 is further configured to send a synchronization request message to the source storage device, where the synchronization request message includes a backup.
  • the serial number of the target backup object saved in the storage device, the sequence number of the target backup object is greater than the sequence number of any other backup object saved by the backup storage device, and the communication module 601 is further configured to receive the sequence number sent by the source storage device that is greater than the sequence number of the target backup object. Target object.
  • the communication module 601, the processing module 602, and the storage module 603 can be implemented by executing the program in the memory 226 by the processor 225 shown in FIG. 2 in actual application.
  • the backup storage device in which the device is located is used to perform the object copying method provided by the embodiment of the present invention, and the same portions are not described herein.
  • the steps of a method or algorithm described in connection with the present disclosure may be implemented in a hardware, or may be implemented by a processor executing software instructions.
  • the software instructions may be composed of corresponding software modules, which may be stored in a random access memory (RAM), a flash memory, a read only memory (ROM), an erasable programmable read only memory ( Erasable Programmable ROM (EPROM), electrically erasable programmable read only memory (EEPROM), registers, hard disk, removable hard disk, compact disk read only (CD-ROM) or any other form of storage medium known in the art.
  • An example A storage medium is coupled to the processor, such that the processor can read information from the storage medium and can write information to the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and the storage medium can be located in an ASIC. Additionally, the ASIC can be located in a core network interface device.
  • the processor and the storage medium may also exist as discrete components in the core network interface device.
  • the functions described herein can be implemented in hardware, software, firmware, or any combination thereof.
  • the functions may be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium.
  • Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another.
  • a storage medium may be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)

Abstract

本发明实施例涉及对象复制方法及对象存储设备，该方法应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，该方法包括：所述源存储设备接收客户端发送的多个写请求消息，其中，每个所述写请求消息包括对象；所述源存储设备为每个写请求消息中的对象分配序号，每个对象的序号依次递增；所述源存储设备将每个对象与所述对象的序号合并，以生成多个目标对象；所述源存储设备将所述多个目标对象发送给所述备份存储设备。本发明实施例能够减少磁盘I/O操作次数。

Description

对象复制方法、装置及对象存储设备 技术领域
本发明实施例涉及存储领域，尤其涉及对象复制方法、装置及对象存储设备。
背景技术
分布式对象存储***(英文：object-based storage system)包含若干个对象存储设备(英文：object-based storage device，OSD)，这些对象存储设备通过网络互联。每个对象存储设备负责将对象(英文：object)写入存储介质或从存储介质读取对象。对象是分布式对象存储***操作的基础单元，包含数据以及与数据相关的属性信息。
在分布式对象存储***中，通常都采用多副本和纠删码(英文：erasure code，EC)的方法进行数据冗余，以应对对象存储设备故障带来的数据丢失或者损坏的风险。即使是单个对象存储设备故障或者多个对象存储设备同时故障，也不影响对象的读写操作，从而提升数据的可靠性。
由于多个对象存储设备中可能存储一个对象的多个对象副本，就需要保证对象与多个对象副本之间的一致性。
通常情况下，多个对象存储设备之间的对象复制是通过控制每个对象存储设备以相同的顺序执行相同的操作，最终以同样的输入保证得到同样的输出，使多个对象副本处于一致的状态。
具体的，每个对象对应一个操作日志，操作日志中包含一个序号，通过操作日志的顺序化，实现了***的强一致性，在实现强一致性的过程中，由于需要单独读写操作日志，导致磁盘输入/输出(Input/Output，I/O)操作 次数较多。例如，对象存储设备将对象写入存储介质之前，先要将该对象对应的操作日志写入存储介质，然后再将该对象写入存储介质，也就是说，对象和该对象对应的操作日志分别写入存储介质，而不是一次性写入存储介质。
发明内容
本申请提供了一种对象复制方法、装置及对象存储设备，能够减少磁盘I/O操作次数。
第一方面，提供了一种对象复制方法，该方法应用于分布式对象存储***，该分布式对象存储***包括源存储设备和备份存储设备。源存储设备接收客户端发送的多个写请求消息，其中，每个写请求消息包括对象；源存储设备为每个写请求消息中的对象分配序号，每个对象的序号依次递增；源存储设备将每个对象与该对象的序号合并，以生成多个目标对象；源存储设备将多个目标对象发送给备份存储设备。
本发明实施例中，源存储设备在进行对象复制时，先将对象与对象的序号合并得到目标对象，然后再将目标对象发送给备份存储设备，由于备份存储设备接收到的目标对象是一个整体，因此备份存储设备不会将对象与对象的序号分两次写入存储介质，而是一次性写入存储介质，因此减少了磁盘I/O操作次数。
在第一方面的一种可能的实施方式中，源存储设备存储多个目标对象。
在第一方面的另一种可能的实施方式中，备份存储设备中保存有多个备份对象，多个备份对象是多个目标对象的子集。源存储设备接收备份存储设备发送的同步请求消息，同步请求消息包括备份存储设备中保存的目标备份对象的序号，目标备份对象的序号大于备份存储设备保存的其他任意一个备份对象的序号；源存储设备将序号大于目标备份对象的序号的目标对象发送给备份存储设备。根据该实施方式，源存储设备与备份存储设备在进行对象同步时，只需要根据备份存储设备发送的目标备份对象的序号，将大于目标 备份对象的序号的目标对象发送给备份存储设备，而无需进行日志同步，因而同步所需的时间短，分布式对象存储***性能好。
第二方面，提供了一种对象复制方法，该方法应用于分布式对象存储***，该分布式对象存储***包括源存储设备和备份存储设备。备份存储设备接收源存储设备发送的多个备份对象，其中，每个备份对象包括对象和该对象的序号；备份存储设备存储多个备份对象。
本发明实施例中，备份存储设备接收源存储设备发送的多个备份对象，将备份对象作为一个整体一次性写入存储介质，因此减少了磁盘I/O操作次数。
在第二方面的一种可能的实施方式中，源存储设备中保存有多个目标对象，多个备份对象是多个目标对象的子集，备份存储设备向源存储设备发送同步请求消息，同步请求消息包括备份存储设备中保存的目标备份对象的序号，目标备份对象的序号大于备份存储设备保存的其他任意一个备份对象的序号；备份存储设备接收源存储设备发送的序号大于目标备份对象的序号的目标对象。根据该实施方式，备份存储设备在与源存储设备进行对象同步时，只需要向源存储设备发送目标备份对象的序号，接收源存储设备发送的大于目标备份对象的序号的目标对象，而无需进行日志同步，因而同步所需的时间短，分布式对象存储***性能好。
第三方面，提供了一种对象存储装置，该装置应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，所述对象存储装置位于所述源存储设备中，所述装置包括：通信模块和处理模块；所述通信模块，用于接收客户端发送的多个写请求消息，其中，每个所述写请求消息包括对象；所述处理模块，用于为每个写请求消息中的对象分配序号，每个对象的序号依次递增；将每个对象与所述对象的序号合并，以生成多个目标对象；所述通信模块，还用于将所述多个目标对象发送给所述备份存储设备。
在第三方面的一种可能的实施方式中，所述装置还包括存储模块；所述处理模块，还用于通过所述存储模块存储所述多个目标对象。
在第三方面的另一种可能的实施方式中，所述备份存储设备中保存有多个备份对象，所述多个备份对象是所述多个目标对象的子集；所述通信模块，还用于接收所述备份存储设备发送的同步请求消息，所述同步请求消息包括所述备份存储设备中保存的目标备份对象的序号，所述目标备份对象的序号大于所述备份存储设备保存的其他任意一个备份对象的序号；所述通信模块，还用于将序号大于所述目标备份对象的序号的目标对象发送给所述备份存储设备。
第四方面，提供了一种对象存储装置，所述装置应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，所述对象存储装置位于所述备份存储设备中，所述装置包括：通信模块、处理模块和存储模块；所述通信模块，用于接收所述源存储设备发送的多个备份对象，其中，每个所述备份对象包括对象和所述对象的序号；所述处理模块，用于通过所述存储模块存储所述多个备份对象。
在第四方面的一种可能的实施方式中，所述源存储设备中保存有多个目标对象，所述多个备份对象是所述多个目标对象的子集；所述通信模块，还用于向所述源存储设备发送同步请求消息，所述同步请求消息包括所述备份存储设备中保存的目标备份对象的序号，所述目标备份对象的序号大于所述备份存储设备保存的其他任意一个备份对象的序号；所述通信模块，还用于接收所述源存储设备发送的序号大于所述目标备份对象的序号的目标对象。
另一方面，本发明实施例提供了一种对象存储设备，该对象存储设备可以实现上述方法设计中源存储设备所执行的功能，所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个上述功能相应的模块。
在一个可能的设计中，该对象存储设备的结构中包括处理器和通信接口， 该处理器被配置为支持该对象存储设备执行上述方法中相应的功能。该通信接口用于支持该对象存储设备与其他对象存储设备之间的通信。该对象存储设备还可以包括存储器，该存储器用于与处理器耦合，其保存该对象存储设备必要的程序指令和数据。
又一方面，本发明实施例提供了一种对象存储设备，该对象存储设备可以实现上述方法设计中备份存储设备所执行的功能，所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个上述功能相应的模块。
在一个可能的设计中，该对象存储设备的结构中包括处理器和通信接口，该处理器被配置为支持该对象存储设备执行上述方法中相应的功能。该通信接口用于支持该对象存储设备与其他对象存储设备之间的通信。该对象存储设备还可以包括存储器，该存储器用于与处理器耦合，其保存该对象存储设备必要的程序指令和数据。
又一方面，本发明实施例提供了一种分布式对象存储***，该***包括上述方面所述的源存储设备和备份存储设备。
再一方面，本发明实施例提供了一种计算机存储介质，用于储存为上述源存储设备所用的计算机软件指令，其包含用于执行上述第一方面所设计的程序。
再一方面，本发明实施例提供了一种计算机存储介质，用于储存为上述备份存储设备所用的计算机软件指令，其包含用于执行上述第二方面所设计的程序。
再一方面，本发明实施例提供了一种计算机程序产品，其包含指令，当所述程序被计算机所执行时，该指令使得计算机执行上述第一方面方法设计中源存储设备所执行的功能。
再一方面，本发明实施例提供了一种计算机程序产品，其包含指令，当所述程序被计算机所执行时，该指令使得计算机执行上述第二方面方法设计 中备份存储设备所执行的功能。
附图说明
图1为分布式对象存储***结构示意图；
图2为对象存储设备的硬件结构示意图；
图3为本发明实施例提供的对象复制方法的应用场景示意图；
[根据细则91更正 15.02.2017]　
图4为本发明实施例提供的一种对象复制方法通信示意图；
图4a为本发明实施例提供的一种目标对象的生成方式示意图；
图4b为本发明实施例提供的另一种目标对象的生成方式示意图;
图4c为本发明实施例提供的又一种目标对象的生成方式示意图
图5为本发明实施例提供的一种对象存储装置结构图；
图6为本发明实施例提供的另一种对象存储装置结构图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行描述。
图1为分布式对象存储***结构示意图。如图1所示，一个分布式对象存储***包括多个对象存储设备，这些对象存储设备通过网络互联。客户端将对象发送给分布式对象存储***中的任意一个对象存储设备，对象存储设备存储对象，并且将对象备份至其他一个或多个对象存储设备中。其他对象存储设备接收对象，将其保存为备份对象，由此来保证对象的可靠性。如图1所示，分布式对象存储***包括3个对象存储设备，这3个对象存储设备分别为对象存储设备1、对象存储设备2和对象存储设备3。在实际应用中，分布式对象存储***可以仅包括2个对象存储设备或更多个对象存储设备，本发明实施例对于分布式对象存储***包括的对象存储设备的数目不做限定。该分布式对象存储***中以多副本的方法提升对象的可靠性，例如，对象1、对象2和对象3不仅存储在对象存储设备1上，还存储在对象存储设备2和对象存储设备3上。
对象存储设备可以是磁盘阵列或者闪存阵列或者存储服务器。磁盘阵列 是指包括至少一个控制器和多个磁盘的存储设备。闪存阵列是指包括至少一个控制器和多个固态硬盘(英文：solid state device，SSD)的存储设备。固态硬盘是以闪存(英文：flash memory)芯片为存储介质的存储器，又名固态驱动器(英文：solid state drive，SSD)。另外，对象存储设备还可以是存储服务器，所述存储服务器包含至少一个固态硬盘。
以闪存阵列为例，如图2所示，对象存储设备包括控制器211和一个或多个固态硬盘222。其中，控制器211至少包括接口223、接口224、处理器225和存储器226。
接口223，用于和客户端通信。接口224，用于和固态硬盘222通信。处理器225可能是一个中央处理器(英文：central processing unit，CPU)。
处理器225，用于接收来自客户端的对象、处理所述对象，例如，为对象分配序号，将对象与对象的序号合并以生成目标对象，将目标对象发送给备份存储设备。处理器225还可以将所述目标对象发送给固态硬盘222。
存储器226，用于存放程序。另外，存储器226还用于临时存储从客户端接收的写数据请求或从固态硬盘222读取的数据。控制器221接收客户端发送的对象，将所述对象暂时保存在存储器226中，由处理器225为对象分配序号，将对象与对象的序号合并以生成目标对象，将目标对象暂时保存在存储器226中。然后从存储器226中读取所述目标对象并发送给其他对象存储设备。存储器226包含随机访问存储器(英文：random-access memory，RAM)。可选地，存储器226还包括非易失性存储器(英文：non-volatile memory)，例如至少一个磁存储器。可以理解的是，存储器226可以为随机存储器(Random-Access Memory，RAM)、磁碟、硬盘、固态硬盘(Solid State Disk，SSD)或者非易失性存储器等各种可以存储程序代码的机器可读介质。
对象存储设备按照角色可以分为源存储设备和备份存储设备。源存储设备是复制的发起者；备份存储设备是复制的执行者。如果每个对象存储设备都按照相同的顺序执行写操作，可以保证每个对象存储设备存储的对象副本 的一致性，上述顺序可以理解为时间先后顺序，例如，源存储设备中对象1的接收时间早于对象2的接收时间，源存储设备先存储对象1然后再存储对象2，同样地，备份存储设备从源存储设备接收到对象1和对象2后，也是先存储对象1然后再存储对象2。
本发明实施例中，可以给每个对象分配一个序号，该序号用于表示该对象接收的先后顺序，在一个示例中，序号越小代表该对象接收的越早，当然本发明实施例也不限定于此。对象存储设备需要执行针对对象1的写操作和针对对象2的写操作，若对象1的序号小于对象2的序号，则源存储设备和备份存储设备均先执行针对对象1的写操作，然后再执行针对对象2的写操作，从而保证源存储设备和备份存储设备存储的对象副本相一致。
当源存储设备接收到客户端发送的写请求消息后，不仅要在本地写入对象，还要将该写请求消息携带的对象复制到备份存储设备。对象存储设备在将对象保存在本地时通常涉及两种数据：对象本身和该对象的操作日志。操作日志中包含一个顺序递增的序号以及操作类型等信息，用于对象存储设备根据操作日志针对对象顺序执行相应的操作。当一个对象存储设备有多个对象需要执行操作时，根据每个对象的操作日志中的序号，先针对序号较小的操作日志对相应的对象执行操作。该序号由源存储设备进行分配，每个对象存储设备用该序号确认对象的操作顺序和状态，上述状态指对象是否写入成功的状态，对象写入成功则保存操作日志，从而存在该序号，对象写入不成功则删除该对象对应的操作日志，从而不存在该序号。
在一个示例中，当源存储设备接收到第一写请求消息后，先由处理器将该第一写请求消息包括的对象1存储到缓存中，上述缓存具体可以为图2所示的存储器226，然后由处理器为对象1生成操作日志1，由序号发生器为对象1分配序号1，由处理器将序号1写入对象1的操作日志中，其中，序号发生器可以理解为处理器的功能模块。当源存储设备接收到第二写请求消息后，与上述处理过程类似，先将该第二写请求消息包括的对象2存储到缓存中， 然后由处理器为对象2生成操作日志2，序号发生器中维护的值加1，由序号发生器为对象2分配序号2，由处理器将序号2写入对象2的操作日志中。当处理器将对象写入持久化介质中时，根据操作日志中记录的序号，由于对象1的序号小于对象2的序号，因此先将对象1写入到持久化介质中，然后再将对象2写入持久化介质中，上述持久化介质具体可以图2所示的固态硬盘222。此外，源存储设备将对象1和对象1的操作日志发送给备份存储设备，备份存储设备先存储对象1的操作日志，然后将对象1写入本地的磁盘***，若对象写入成功则备份存储设备中保存有对象1的操作日志，对象1的操作日志中记录有序号1，若对象写入不成功，则备份存储设备中不保存有对象1的操作日志，也就是说不记录有序号1。操作日志和对象均需要进行持久化存储，由于通常操作日志和对象分别进行读写操作，导致I/O次数较多，为了提高读写性能，操作日志一般存储在高性能的存储介质上，例如NVRAM，而对象则存储在硬盘驱动器(Hard Disk Drive，HDD)上。
因此，按照这种对象复制方法，需要分别读写对象和该对象的操作日志，导致需要进行多次磁盘I/O操作，性能较差，为了提升读写性能，使用类似NVRAM这样的高性能存储介质存储操作日志，导致***成本高居不下。在故障恢复时，首先要进行操作日志同步，才能进行数据同步，性能较低。
本发明实施例提供了一种对象复制方法，应用于图1所示的分布式对象存储***，将对象和序号合并生成目标对象，存储所述目标对象。由于对象和序号合并后作为一个整体存在，因此在读取所述目标对象时可以一次性读取对象和序号，减少了访问磁盘的次数。
图3为本发明实施例提供的对象复制方法的应用场景示意图。该方法应用于源存储设备将对象复制到备份存储设备上，每个对象存储设备上存储的对象与对象副本需要具备一致性，对象与序号具有一一对应的关系，通过对象的序号来标识对象存储设备间存储对象的一致性，例如，源存储设备中记录有序号1至10，备份存储设备中记录有序号1至8，说明源存储设备与备 份存储设备存储的对象不一致，备份存储设备与源存储设备相比缺少了序号9和10对应的对象。
图4为本发明实施例提供的一种对象复制方法通信示意图，该方法应用于分布式对象存储***，分布式对象存储***至少包括源存储设备和备份存储设备，基于图2所示的对象存储设备的具体结构，该方法可以由处理器225执行，该方法包括：
步骤401，源存储设备接收客户端发送的多个写请求消息，其中，每个写请求消息包括对象。
其中，上述多个写请求消息可以是一个客户端发送的多个写请求消息，也可以是多个客户端分别发送的写请求消息。
步骤402，源存储设备为每个写请求消息中的对象分配序号，每个对象的序号依次递增。
基于图2所示的对象存储设备的具体结构。控制器221接收客户端发送的对象，将所述对象暂时保存在存储器226中，由处理器225为对象分配序号，将序号暂时保存在存储器226中。
在一个示例中，源存储设备根据接收到的多个写请求消息的先后顺序为每个写请求消息中的对象分配序号，例如，源存储设备先接收到一个写请求消息，为该写请求消息中的对象分配序号100，源存储设备随后又接收到一个写请求消息，为该写请求消息中的对象分配序号101，其中，每个对象的序号依次递增，递增的数值可以但不限于1。
步骤403，源存储设备将每个对象与该对象的序号合并，以生成多个目标对象。
目标对象可以暂时保存在存储器226中。处理器225还可以将所述目标对象发送给固态硬盘222。
合并的方式可以但不限于以下几种方式：将对象的序号添加到该对象头部的前面，生成目标对象，如图4a所示；或者，将对象的序号添加到该对象 尾部的后面，生成目标对象，如图4b所示；或者，将对象分成对象00和对象01两部分，将对象的序号添加到对象00和对象01之间，生成目标对象，如图4c所示。
在一个示例中，每个对象存储设备处理的对象都有唯一递增的序号，这个序号和对象合并为一个目标对象；每个对象存储设备将目标对象写入持久化介质；在对象存储设备故障时，可以通过校验和(checksum)来检测目标对象包括的对象和序号是否均写入持久化介质，通过校验和来保证数据的完整性和准确性。
本发明实施例中，目标对象中可以仅包括对象和对象的序号，还可以包括操作类型信息，例如，操作类型信息用于标识写操作。
通常地，将序号先记录在操作日志中，将对象与序号合并可以认为是将对象与操作日志的内容合并。如果N个对象存储设备上的对象一致，则拥有相同的序号；反之，如果有对象存储设备故障了一段时间，在这段时间内，它上面会少存储一部分对象，由于序号是顺序递增的，因此它上面存储的对象的序号将小于其它对象存储设备上面存储的对象的序号，通过在故障恢复后的同步操作，多个对象存储设备间可以达到一致性。
步骤404，源存储设备将多个目标对象发送给备份存储设备。
本发明实施例中，源存储设备可以将多个目标对象同时发送给备份存储设备，还可以将多个目标对象分别发送给备份存储设备。
步骤405，备份存储设备存储多个目标对象。
其中，备份存储设备存储的目标对象可以称为备份对象。
目标对象可以暂时保存在备份存储设备的存储器226中。备份存储设备的处理器225还可以将所述目标对象发送给备份存储设备的固态硬盘222。
本申请提供的方案中，源存储设备将对象与对象的序号合并得到目标对象，然后再将目标对象发送给备份存储设备，由于备份存储设备接收到的目标对象是一个整体，因此备份存储设备不会将对象与对象的序号分两次写入 存储介质，而是一次性写入存储介质，从而减少了磁盘I/O操作次数。
本发明实施例除了包括前述步骤401至405还可以包括：
步骤406，源存储设备存储多个目标对象。
目标对象可以暂时保存在源存储设备的存储器226中。源存储设备的处理器225还可以将所述目标对象发送给源存储设备的固态硬盘222。
本发明实施例中，对于步骤406和步骤404的执行顺序不做限定，可以同时执行步骤406和步骤404，也可以先执行步骤404再执行步骤406，还可以先执行步骤406再执行步骤404。图5中仅示出先执行步骤404再执行步骤406的情况。
若备份存储设备发生故障，在其故障期间有一些对象没有接收到，当备份存储设备故障恢复后，向源存储设备发送同步请求消息，请求进行同步。本发明实施例除了包括前述步骤401至406还可以包括：
步骤407，备份存储设备向源存储设备发送同步请求消息，同步请求消息包括备份存储设备中保存的目标备份对象的序号，目标备份对象的序号大于备份存储设备保存的其他任意一个备份对象的序号。
其中，源存储设备中保存有多个目标对象，备份存储设备中保存有多个备份对象，前述多个备份对象是前述多个目标对象的子集，目标备份对象是前述多个备份对象中序号最大的备份对象。
步骤408，源存储设备将序号大于目标备份对象的序号的目标对象发送给备份存储设备。
源存储设备接收到备份存储设备发送的同步请求消息之后，根据消息包括的目标备份对象的序号，与之前保存的各个对象的序号进行匹配，确定将序号大于目标备份对象的序号的目标对象发送给备份存储设备。
备份存储设备接收源存储设备发送的序号大于目标备份对象的序号的目标对象后，可以存储这些目标对象，以实现与源存储设备的数据同步。
此外，在一个示例中，源存储设备接收备份存储设备发送的第一指示信 息，第一指示信息用于指示目标对象在备份存储设备写入成功，源存储设备向客户端发送第二指示信息，第二指示信息用于指示目标对象在源存储设备和备份存储设备均写入成功。
在另一个示例中，源存储设备接收备份存储设备发送的第三指示信息，第三指示信息用于指示目标对象在备份存储设备写入失败，源存储设备根据第三指示信息进行容错处理，例如标记该备份存储设备异常、故障，从而进入容错状态。源存储设备向客户端发送第四指示信息，第四指示信息用于指示目标对象在源存储设备写入成功且在备份存储设备写入失败。
图5为本发明实施例提供的一种对象存储装置结构图，该装置应用于分布式对象存储***，该分布式对象存储***包括源存储设备和备份存储设备，该装置位于源存储设备中，该装置包括：通信模块501和处理模块502；
通信模块501，用于接收客户端发送的多个写请求消息，其中，每个写请求消息包括对象；
处理模块502，用于为每个写请求消息中的对象分配序号，每个对象的序号依次递增；将每个对象与对象的序号合并，以生成多个目标对象；
通信模块501，还用于将多个目标对象发送给备份存储设备。
在一个示例中，该装置还包括存储模块503；处理模块502，还用于通过存储模块503存储多个目标对象。
在一个示例中，备份存储设备中保存有多个备份对象，多个备份对象是多个目标对象的子集；通信模块501，还用于接收备份存储设备发送的同步请求消息，同步请求消息包括备份存储设备中保存的目标备份对象的序号，目标备份对象的序号大于备份存储设备保存的其他任意一个备份对象的序号；通信模块501，还用于将序号大于目标备份对象的序号的目标对象发送给备份存储设备。
需要说明的是，通信模块501、处理模块502和存储模块503在实际应用时可以通过图2所示的处理器225执行存储器226中的程序来实现。该装置 位于的源存储设备用于执行本发明实施例提供的对象复制方法，相同的部分在此不做赘述。
图6为本发明实施例提供的另一种对象存储装置结构图，该装置应用于分布式对象存储***，该分布式对象存储***包括源存储设备和备份存储设备，该装置位于备份存储设备中，该装置包括：通信模块601、处理模块602和存储模块603；
通信模块601，用于接收源存储设备发送的多个备份对象，其中，每个备份对象包括对象和对象的序号；
处理模块602，用于通过存储模块603存储多个备份对象。
在一个示例中，源存储设备中保存有多个目标对象，多个备份对象是多个目标对象的子集；通信模块601，还用于向源存储设备发送同步请求消息，同步请求消息包括备份存储设备中保存的目标备份对象的序号，目标备份对象的序号大于备份存储设备保存的其他任意一个备份对象的序号；通信模块601，还用于接收源存储设备发送的序号大于目标备份对象的序号的目标对象。
需要说明的是，通信模块601、处理模块602和存储模块603在实际应用时可以通过图2所示的处理器225执行存储器226中的程序来实现。该装置位于的备份存储设备用于执行本发明实施例提供的对象复制方法，相同的部分在此不做赘述。
结合本发明公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例 性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于核心网接口设备中。当然，处理器和存储介质也可以作为分立组件存在于核心网接口设备中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本发明所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
以上所述的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本发明的保护范围之内。

Claims (12)

  1. 一种对象复制方法，其特征在于，所述方法应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，所述方法包括：
    所述源存储设备接收客户端发送的多个写请求消息，其中，每个所述写请求消息包括对象；
    所述源存储设备为每个写请求消息中的对象分配序号，每个对象的序号依次递增；
    所述源存储设备将每个对象与所述对象的序号合并，以生成多个目标对象；
    所述源存储设备将所述多个目标对象发送给所述备份存储设备。
  2. 如权利要求1所述的方法，其特征在于，所述方法还包括：
    所述源存储设备存储所述多个目标对象。
  3. 如权利要求1或2所述的方法，其特征在于，所述备份存储设备中保存有多个备份对象，所述多个备份对象是所述多个目标对象的子集，所述方法还包括：
    所述源存储设备接收所述备份存储设备发送的同步请求消息，所述同步请求消息包括所述备份存储设备中保存的目标备份对象的序号，所述目标备份对象的序号大于所述备份存储设备保存的其他任意一个备份对象的序号；
    所述源存储设备将序号大于所述目标备份对象的序号的目标对象发送给所述备份存储设备。
  4. 一种对象复制方法，其特征在于，所述方法应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，所述方法包括：
    所述备份存储设备接收所述源存储设备发送的多个备份对象，其中，每个所述备份对象包括对象和所述对象的序号；
    所述备份存储设备存储所述多个备份对象。
  5. 如权利要求4所述的方法，其特征在于，所述源存储设备中保存有多个目标对象，所述多个备份对象是所述多个目标对象的子集，所述方法还包括：
    所述备份存储设备向所述源存储设备发送同步请求消息，所述同步请求消息包括所述备份存储设备中保存的目标备份对象的序号，所述目标备份对象的序号大于所述备份存储设备保存的其他任意一个备份对象的序号；
    所述备份存储设备接收所述源存储设备发送的序号大于所述目标备份对象的序号的目标对象。
  6. 一种对象存储装置，其特征在于，所述装置应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，所述对象存储装置位于所述源存储设备中，所述装置包括：通信模块和处理模块；
    所述通信模块，用于接收客户端发送的多个写请求消息，其中，每个所述写请求消息包括对象；
    所述处理模块，用于为每个写请求消息中的对象分配序号，每个对象的序号依次递增；将每个对象与所述对象的序号合并，以生成多个目标对象；
    所述通信模块，还用于将所述多个目标对象发送给所述备份存储设备。
  7. 如权利要求6所述的装置，其特征在于，所述装置还包括存储模块；
    所述处理模块，还用于通过所述存储模块存储所述多个目标对象。
  8. 如权利要求6或7所述的装置，其特征在于，所述备份存储设备中保存有多个备份对象，所述多个备份对象是所述多个目标对象的子集；
    所述通信模块，还用于接收所述备份存储设备发送的同步请求消息，所述同步请求消息包括所述备份存储设备中保存的目标备份对象的序号，所述目标备份对象的序号大于所述备份存储设备保存的其他任意一个备份对象的序号；
    所述通信模块，还用于将序号大于所述目标备份对象的序号的目标对象 发送给所述备份存储设备。
  9. 一种对象存储装置，其特征在于，所述装置应用于分布式对象存储***，所述分布式对象存储***包括源存储设备和备份存储设备，所述对象存储装置位于所述备份存储设备中，所述装置包括：通信模块、处理模块和存储模块；
    所述通信模块，用于接收所述源存储设备发送的多个备份对象，其中，每个所述备份对象包括对象和所述对象的序号；
    所述处理模块，用于通过所述存储模块存储所述多个备份对象。
  10. 如权利要求9所述的装置，其特征在于，所述源存储设备中保存有多个目标对象，所述多个备份对象是所述多个目标对象的子集；
    所述通信模块，还用于向所述源存储设备发送同步请求消息，所述同步请求消息包括所述备份存储设备中保存的目标备份对象的序号，所述目标备份对象的序号大于所述备份存储设备保存的其他任意一个备份对象的序号；
    所述通信模块，还用于接收所述源存储设备发送的序号大于所述目标备份对象的序号的目标对象。
  11. 一种对象存储设备，其特征在于，所述设备包括存储器和处理器，所述存储器用于存储程序指令，所述处理器用于根据所述存储器中存储的程序指令执行权利要求1至3中任一项所述的方法。
  12. 一种对象存储设备，其特征在于，所述设备包括存储器和处理器，所述存储器用于存储程序指令，所述处理器用于根据所述存储器中存储的程序指令执行权利要求4或5所述的方法。
PCT/CN2016/110314 2016-12-16 2016-12-16 对象复制方法、装置及对象存储设备 WO2018107460A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2016/110314 WO2018107460A1 (zh) 2016-12-16 2016-12-16 对象复制方法、装置及对象存储设备
CN201680003361.9A CN108475254A (zh) 2016-12-16 2016-12-16 对象复制方法、装置及对象存储设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/110314 WO2018107460A1 (zh) 2016-12-16 2016-12-16 对象复制方法、装置及对象存储设备

Publications (1)

Publication Number Publication Date
WO2018107460A1 true WO2018107460A1 (zh) 2018-06-21

Family

ID=62557820

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/110314 WO2018107460A1 (zh) 2016-12-16 2016-12-16 对象复制方法、装置及对象存储设备

Country Status (2)

Country Link
CN (1) CN108475254A (zh)
WO (1) WO2018107460A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013003684A1 (en) * 2011-06-30 2013-01-03 Amazon Technologies, Inc. Methods and apparatus for remotely updating executing processes
CN105630808A (zh) * 2014-10-31 2016-06-01 北京奇虎科技有限公司 基于分布式文件***的文件读取、写入方法及节点服务器
CN105659204A (zh) * 2013-08-13 2016-06-08 经度事业闪存公司 用于进行带有注释的原子写入操作的方法和装置
CN105912628A (zh) * 2016-04-07 2016-08-31 北京奇虎科技有限公司 主从数据库的同步方法及装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6269080B1 (en) * 1999-04-13 2001-07-31 Glenayre Electronics, Inc. Method of multicast file distribution and synchronization
US7698428B2 (en) * 2003-12-15 2010-04-13 International Business Machines Corporation Apparatus, system, and method for grid based data storage
CN103581262B (zh) * 2012-08-06 2017-12-29 腾讯科技（深圳）有限公司 一种主备数据同步方法、装置和***
CN103369051B (zh) * 2013-07-22 2016-04-27 中安消技术有限公司 一种数据服务器集群***及数据同步方法
CN103649901A (zh) * 2013-07-26 2014-03-19 华为技术有限公司 数据发送方法、数据接收方法和存储设备
CN103716384A (zh) * 2013-12-17 2014-04-09 创新科存储技术（深圳）有限公司 跨数据中心实现云存储数据同步的方法和装置
CN104954101B (zh) * 2015-07-24 2018-06-01 湖北文理学院 基于同步ack的多终端数据同步方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013003684A1 (en) * 2011-06-30 2013-01-03 Amazon Technologies, Inc. Methods and apparatus for remotely updating executing processes
CN105659204A (zh) * 2013-08-13 2016-06-08 经度事业闪存公司 用于进行带有注释的原子写入操作的方法和装置
CN105630808A (zh) * 2014-10-31 2016-06-01 北京奇虎科技有限公司 基于分布式文件***的文件读取、写入方法及节点服务器
CN105912628A (zh) * 2016-04-07 2016-08-31 北京奇虎科技有限公司 主从数据库的同步方法及装置

Also Published As

Publication number Publication date
CN108475254A (zh) 2018-08-31

Similar Documents

Publication Publication Date Title
US9135119B1 (en) System and method for data management
US7761732B2 (en) Data protection in storage systems
US20110099148A1 (en) Verification Of Remote Copies Of Data
EP2879040B1 (en) Data storage method, data storage apparatus, and storage device
US11157177B2 (en) Hiccup-less failback and journal recovery in an active-active storage system
US9262344B2 (en) Local locking in a bi-directional synchronous mirroring environment
US8463746B2 (en) Method and system for replicating data
US9507671B2 (en) Write cache protection in a purpose built backup appliance
WO2018098972A1 (zh) 一种日志恢复方法、存储装置和存储节点
US20190317872A1 (en) Database cluster architecture based on dual port solid state disk
WO2018120844A1 (zh) 一种差异数据备份方法和差异数据备份装置
WO2016117022A1 (ja) ログの管理方法及び計算機システム
WO2019080370A1 (zh) 一种数据读写方法、装置和存储服务器
WO2018076633A1 (zh) 一种远程数据复制方法、存储设备及存储***
US10664193B2 (en) Storage system for improved efficiency of parity generation and minimized processor load
US10740189B2 (en) Distributed storage system
US11513729B1 (en) Distributed write buffer for storage systems
US10656867B2 (en) Computer system, data management method, and data management program
WO2018107460A1 (zh) 对象复制方法、装置及对象存储设备
WO2018055686A1 (ja) 情報処理システム
US11074003B2 (en) Storage system and restoration method
US11256716B2 (en) Verifying mirroring of source data units to target data units
US10860221B1 (en) Page write to non-volatile data storage with failure recovery
US20160371009A1 (en) Storage system and memory control method
US10528529B1 (en) Multi-core journal system and method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16923983

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16923983

Country of ref document: EP

Kind code of ref document: A1