CN111522883A

CN111522883A - Backup method, device and equipment of object data and storage medium

Info

Publication number: CN111522883A
Application number: CN202010349216.4A
Authority: CN
Inventors: 丁光凯; 夏伟强; 王定琋
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2020-04-28
Filing date: 2020-04-28
Publication date: 2020-08-11
Anticipated expiration: 2040-04-28
Also published as: CN111522883B

Abstract

The application discloses a backup method, a backup device, backup equipment and a storage medium of object data, and belongs to the technical field of data storage. The method comprises the following steps: reading an asynchronous backup task in an asynchronous backup task table, wherein the asynchronous backup task comprises a first hash parameter and first hash ring information, the asynchronous backup task refers to a task for backing up first object data in the first storage node, and the first hash ring indicated by the first hash ring information comprises a plurality of storage nodes in the cloud storage system. And determining a second storage node according to the first hash parameter and the first hash ring information, wherein the second storage node is a storage node used for backing up the first object data, and backing up the first object data to the second storage node. In the method and the device, the second storage node for backup can be dynamically determined, and the condition that only one copy of object data can be stored can be avoided, so that the storage stability is ensured.

Description

Backup method, device and equipment of object data and storage medium

Technical Field

The present application relates to the field of data storage technologies, and in particular, to a method, an apparatus, a device, and a storage medium for backing up object data.

Background

In a cloud storage system, in order to improve storage reliability, object data is generally backed up in different storage nodes when being written in, so that when one or more storage nodes in the cloud storage system are in failure, read-write service of the object data is not affected.

In the related art, a backup mode is generally adopted, in which storage nodes in a cloud storage system are grouped in a fixed mode, and multi-copy storage is performed in a group. For example, if the cloud storage system includes a storage node a, a storage node B, a storage node C, and a storage node D, the storage node a and the storage node B may be grouped together, and the storage node C and the storage node D may be grouped together when grouped together.

However, in the above method, since the group backup is performed in a fixed manner, if the number of storage nodes in the cloud storage system is odd, some object data may be stored only in one copy, so that the storage stability is poor.

Disclosure of Invention

The application provides a backup method, a backup device, backup equipment and a backup storage medium of object data, which can solve the problem that some object data in the related art can only be stored in one copy, so that the storage stability is poor. The technical scheme is as follows:

in one aspect, a method for backing up object data is provided, and is applied to a first storage node, where the first storage node is any storage node in a cloud storage system, and the method includes:

reading an asynchronous backup task in an asynchronous backup task table, wherein the asynchronous backup task comprises a first hash parameter and first hash ring information, the asynchronous backup task is a task for backing up first object data in a first storage node, and the first hash ring indicated by the first hash ring information comprises a plurality of storage nodes in the cloud storage system;

determining a second storage node according to the first hash parameter and the first hash ring information, wherein the second storage node is a storage node used for backing up the first object data;

and backing up the first object data to the second storage node.

In a possible implementation manner of the present application, the determining a second storage node according to the first hash parameter and the first hash ring information includes:

determining whether other storage nodes exist in the first hash ring online based on the first hash ring information, wherein the other storage nodes are storage nodes except the first storage node;

if other storage nodes exist in the first hash ring online, determining the number of the storage nodes in the first hash ring according to the first hash ring information;

performing hash operation according to the first hash parameter and the number of the storage nodes to obtain a hash value;

determining a storage node corresponding to the hash value from the first hash ring;

and if the storage node corresponding to the hash value is the first storage node and the next storage node of the first storage node is online in the first hash ring, determining the next storage node of the first storage node as the second storage node.

In a possible implementation manner of the present application, after determining the storage node corresponding to the hash value from the first hash ring, the method further includes:

if the storage node corresponding to the hash value is the last storage node of the first storage node and the last storage node of the first storage node is online, determining the last storage node of the first storage node as the second storage node; alternatively, the first and second electrodes may be,

if the storage node corresponding to the hash value is the last storage node of the first storage node and the last storage node of the first storage node is not online, if the next storage node of the first storage node is online, determining the next storage node of the first storage node as the second storage node.

if the storage node corresponding to the hash value is not the first storage node and is not the last storage node of the first storage node, determining the storage node corresponding to the hash value as the second storage node if the storage node corresponding to the hash value is online.

In a possible implementation manner of the present application, after determining whether there are other storage nodes online in the first hash ring based on the first hash ring information, the method further includes:

if no other storage node is online in the first hash ring, acquiring the latest hash ring information;

and determining the second storage node from the hash ring corresponding to the latest hash ring information based on the latest hash ring information and the first hash parameter.

In one possible implementation manner of the present application, the method further includes:

receiving a write request of the first object data, wherein the write request carries the first hash parameter and the first hash ring information;

determining a master storage node based on the first hash parameter and the first hash ring information;

storing the first object data to the primary storage node;

in the event that the first object data is successfully stored to the primary storage node, determining a next storage node of the primary storage node from the first hash ring;

and if the next storage node of the main storage node is not on line, recording the asynchronous backup task in the asynchronous backup task table.

In one possible implementation manner of the present application, after determining a next storage node of the primary storage nodes from the first hash ring, the method further includes:

if the next storage node of the main storage nodes is online, writing the first object data to the next storage node of the main storage nodes;

recording the asynchronous backup task in the asynchronous backup task table in case of failure to write the first object data to a storage node next to the primary storage node.

reading a copy supplement task, wherein the copy supplement task is generated under the condition that the storage node offline event is determined to exist, and comprises node information and an offline time point of an offline storage node;

index information in a local object index information table is inspected;

if the offline storage node is determined to be a storage node used for storing second object data corresponding to target index information according to the node information, the offline time point and the index information in the object index information table, recording the asynchronous backup task in the asynchronous backup task table based on the target index information, wherein the target index information is the index information in the object index information table.

In a possible implementation manner of the present application, the index information includes a storage time point of corresponding object data and corresponding hash ring information, and the method further includes:

if the index information of which the corresponding storage time point is positioned before the offline time point exists in the object index information table, determining the index information of which the corresponding storage time point is positioned before the offline time point as the target index information;

and if the offline storage node is determined to be the last storage node of the first storage node in a second hash ring corresponding to the target index information according to the node information and the hash ring information corresponding to the target index information, or the offline storage node is determined to be the next storage node of the first storage node, determining that the offline storage node is a storage node for storing second object data corresponding to the target index information.

receiving a reading request, wherein the reading request carries index information and the first hash ring information;

if the first object data corresponding to the index information locally exists, returning the first object data;

and if the first object data corresponding to the index information does not exist locally, reading the first object data according to the first hash ring information.

In a possible implementation manner of this application, the read request carries a first hash parameter, the first hash ring information further includes version information of the first hash ring, and the reading the first object data according to the first hash ring information includes:

determining a next storage node of the first storage node in the first hash ring according to the first hash ring information;

if the first object data is not read from the next storage node of the first storage node, determining all hash rings with the versions lower than that of the first hash ring according to the version information of the first hash ring;

selecting hash rings from all the hash rings for multiple times according to the sequence of the versions from high to low;

each time one hash ring is selected, determining a storage node from the currently selected hash ring according to the currently selected hash ring and the first hash parameter, and reading the first object data from the determined storage node; until the first object data is read.

In another aspect, an apparatus for backing up object data is configured at a first storage node, where the first storage node is any storage node in a cloud storage system, and the apparatus includes:

the reading module is used for reading an asynchronous backup task in an asynchronous backup task table, wherein the asynchronous backup task comprises a first hash parameter and first hash ring information, the asynchronous backup task is a task for backing up first object data in a first storage node, and the first hash ring indicated by the first hash ring information comprises a plurality of storage nodes in the cloud storage system;

a determining module, configured to determine a second storage node according to the first hash parameter and the first hash ring information, where the second storage node is a storage node used for backing up the first object data;

and the backup module is used for backing up the first object data to the second storage node.

In one possible implementation manner of the present application, the determining module is configured to:

storing the first object data to the primary storage node;

index information in a local object index information table is inspected;

In one possible implementation manner of the present application, the reading module is configured to:

In a possible implementation manner of the present application, the read request carries a first hash parameter, the first hash ring information further includes version information of the first hash ring, and the read module is configured to:

In another aspect, a computer device is provided, where the computer device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus, the memory is used to store a computer program, and the processor is used to execute the program stored in the memory to implement the steps of the above-mentioned object data backup method.

In another aspect, a computer-readable storage medium is provided, in which a computer program is stored, which, when being executed by a processor, implements the steps of the above-mentioned backup method for object data.

In another aspect, a computer program product containing instructions is provided, which when run on a computer, causes the computer to perform the steps of the above-described method for backing up object data.

The technical scheme provided by the application can at least bring the following beneficial effects:

the first storage node reads an asynchronous backup task in an asynchronous backup task table, the asynchronous backup task comprises a first hash parameter and first hash ring information, the asynchronous backup task refers to a task for backing up first object data in the first storage node, and the first hash ring indicated by the first hash ring information comprises a plurality of storage nodes in the cloud storage system. And determining a second storage node according to the first hash parameter and the first hash ring information, wherein the second storage node is a storage node used for backing up the first object data, and backing up the first object data to the second storage node. In the method and the device, the second storage node for backup can be dynamically determined, and the condition that only one copy of object data can be stored can be avoided, so that the storage stability is ensured.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic diagram of a hash ring provided in an embodiment of the present application;

fig. 2 is a schematic architecture diagram of a cloud storage system according to an embodiment of the present application;

fig. 3 is a flowchart of a method for backing up object data according to an embodiment of the present application;

FIG. 4 is a schematic diagram of another hash ring provided in embodiments of the present application;

FIG. 5 is a flowchart of another method for backing up object data according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an apparatus for backing up object data according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a storage node according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Before explaining the backup method of object data provided in the embodiment of the present application in detail, relevant terms and implementation environments provided in the embodiment of the present application are introduced.

First, related terms related to the embodiments of the present application will be briefly described.

An object file: the method is a basic storage unit in the cloud storage system, each object file corresponds to a unique key value (object-key), and corresponding object data can be accessed through the key value.

And (3) copy supplementation: in the cloud storage system, one or more copies of each object data exist in different storage nodes, and if a certain storage node is abnormal, the object data on the storage node can be copied to other normal storage nodes, so that the object data are always backed up.

HASH ring (HASH ring): the method refers to a ring formed by arranging all online storage nodes in a cloud storage system according to rules. When reading and writing the object data, the client can calculate a hash value according to the hash parameter corresponding to the object data and the number of storage nodes in the hash ring, can locate the storage node for storing the object data according to the hash value, and then directly connect the storage node to read and write the object data, the storage nodes in the hash ring have a certain sequence, and the mutual backup relationship of the copies can be determined based on the sequence. For example, as shown in fig. 1, the storage node A, B, C, D, E forms a hash ring, and the copy relationships are AB, BC, CD, DE, and EA, i.e. the object data in a can be backed up to B, and so on, the object data in E can be backed up to a.

Next, a brief description will be given of an implementation environment related to the embodiments of the present application.

The object data backup method provided by the embodiment of the application is applied to a cloud storage system, the cloud storage system can be a cloud storage high-performance access cluster, also can be referred to as an HAS cluster for short, and the cloud storage system is a centerless and distributed storage architecture. Referring to fig. 2, fig. 2 is a schematic diagram illustrating an architecture of a cloud storage system according to an exemplary embodiment. The cloud storage system includes a plurality of storage nodes 210, a client 220, and an EC (Erasure Code) storage device 230. The client 220 may establish a communication connection with each storage node 210, and each storage node 210 establishes a communication connection with the EC storage device 230.

In the centerless cloud storage system, each storage node 210 stores index information of respective object data. The management module may be responsible for maintaining state information of all storage nodes, generating a hash ring, updating a hash ring version, and providing functions such as query of hash ring information for a client. The storage module may provide services such as reading, writing, deleting, and querying of object data for a client, is responsible for operations of reading and writing object data to a Disk, and records index information of the object data, and may further perform services such as copy replication and copy supplementation.

In addition, part of the storage nodes 210 in the cloud storage system may also have no management module, and for such storage nodes without configuration of a management module, the storage nodes may be connected to a storage node equipped with a management module, for example, the storage nodes may be connected to the management module, so as to synchronize data such as state information and hash ring information of each storage node from the management module.

The client 220 may be equipped with an SDK (Software development kit) for performing interactions such as reading, writing, querying and the like with the storage node 210 through the SDK. The client 220 may also synchronize the hash ring information, the storage node information and the state information of the storage node, etc. from the management module of the storage node through the SDK, wherein the storage node information may include, but is not limited to, a storage node address, a listening port. As an example, the management module in each storage node may map its own address to a specified virtual address, and thus the client may synchronize data such as hash ring information, storage node information and state information of the storage node from the specified virtual address. Wherein, the appointed virtual address can be set according to actual requirements.

In some embodiments, the client 220 may be a standalone device or may be configured in a terminal. As an example, the terminal may be any electronic product that can interact with a user through one or more manners such as a keyboard, a touch pad, a touch screen, a remote controller, a voice interaction device, or a handwriting device, for example, a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a pocket PC (pocket PC), a tablet Computer, a smart car, a smart television, a smart speaker, and the like.

The EC storage device refers to a device that can recover lost data by using erasure codes. In the cloud storage system, in order to improve the performance of writing object data by a user, the object data is firstly cached in the SSD, and is asynchronously transferred to the EC storage device when a certain condition is met.

In the embodiment of the application, in order to ensure the security of the object data cached in the SSD and the performance of object data extraction, the object data may be stored in the SSD of different storage nodes in a double copy manner. Specific implementations thereof can be seen in the following examples.

After describing related terms and implementation environments related to the embodiments of the present application, a detailed explanation will be provided for a backup method of object data according to the embodiments of the present application with reference to the accompanying drawings.

Referring to fig. 3, fig. 3 is a flowchart of a method for backing up object data according to an embodiment of the present application, where the method is applied in the implementation environment shown in fig. 2 and executed by a first storage node, where the first storage node is any storage node in the cloud storage system, and the method may include the following implementation steps.

Step 301: reading an asynchronous backup task in an asynchronous backup task table, wherein the asynchronous backup task comprises a first hash parameter and first hash ring information, the asynchronous backup task refers to a task for backing up first object data in the first storage node, and the first hash ring indicated by the first hash ring information comprises a plurality of storage nodes in the cloud storage system.

The first hash parameter may include a resource pool identifier and a key value of the first object data, the resource pool identifier may be pre-allocated by the system, and the key value of the first object data may be set by a user in a customized manner. For example, the first hash parameter may include a bucket and an object, where the bucket is a resource pool identifier, and the object is a key value of the first object data, and in this embodiment, the first hash parameter uniquely corresponds to the first object data.

As an example, the asynchronous backup task may include index information of the first object data to be backed up, in addition to the first hash parameter and the first hash ring information.

The asynchronous backup task table may include one or more asynchronous backup tasks, and the first storage node reads the asynchronous backup task from the asynchronous backup task table and implements the asynchronous backup task in the manner provided in this embodiment. After the current asynchronous backup task is completed, the first storage node may delete the completed asynchronous replication task, continue to read the next asynchronous backup task from the asynchronous backup task table, and implement the next asynchronous backup task in the manner provided by the present application, and so on until all asynchronous backup tasks in the asynchronous backup task table are completed.

Here, the generation process of the asynchronous replication task will be described first. As an example, the generation of the asynchronous replication task may include several possible scenarios:

a first possible implementation: during the writing process of the first object data, the asynchronous replication task is generated under the condition that a certain condition is met.

In the writing process of the first object data, the first storage node receives a writing request of the first object data, and the writing request carries the first hash parameter and the first hash ring information. Determining a primary storage node based on the first hash parameter and the first hash ring information, storing the first object data to the primary storage node, and in case the first object data is successfully stored to the primary storage node, determining a next storage node of the primary storage node from the first hash ring. If the next storage node of the primary storage node is not online, the asynchronous backup task is recorded in the asynchronous backup task table.

The first hash ring information may include version information of the first hash ring and a storage node list corresponding to the first hash ring, where the storage node list includes node information of storage nodes on the first hash ring. Illustratively, the version information of the first hash ring is version-1, and the storage node list corresponding to the first hash ring includes: serial-id1, serial-id2, and serial-id3 …. For example, the first hash ring information is version-6: serial-A, serial-C, serial-D, serial-E.

It is understood that the write request may also carry the first object data, and further, the write request may also carry object data description information such as a length and a data type of the first object data, which is not limited in this embodiment of the present application.

After receiving a write request sent by a client, the first storage node may parse the write request to obtain the first hash parameter and the first hash ring information. Then, the first storage node may determine the number of storage nodes in the first hash ring according to the first hash ring information, and then perform a hash calculation based on the first hash parameter and the number of storage nodes to obtain the hash value, for example, the hash value may be determined by the following formula (1):

Hash_index＝HASH(bucket+objectkey)％hash(1)

wherein the Hash _ index indicates the Hash value, the bucket indicates the resource pool identifier, the object indicates the key value of the first object data, the Hash indicates the number of storage nodes, and the% indicates the remainder operation.

The hash value is a subscript position of the storage node list corresponding to the first hash ring, so that the corresponding storage node can be determined from the first hash ring according to the hash value to obtain the main storage node. It should be noted that, in the embodiment of the present application, the storage node determined from the first hash ring based on the hash value is referred to as a primary storage node.

The first storage node stores the first object data into the main storage node, and in case of successful storage, the first storage node determines a next storage node of the main storage node from the first hash ring, that is, determines a storage node for backing up the first object data. If the next storage node of the main storage node in the first hash ring is not online, it can be considered that the backup cannot be performed currently, and in this case, the first storage node generates an asynchronous backup task and stores the asynchronous backup task into the asynchronous backup task table.

As an example, if the next storage node of the primary storage node is online, the first object data is written to the next storage node of the primary storage node, and in case of failure to write the first object data to the next storage node of the primary storage node, the asynchronous backup task is recorded in the asynchronous backup task table.

That is, in one possible implementation, the next storage node of the primary storage node in the first hash ring is online, in which case the first object data may be written to the next storage node of the primary storage node, i.e., the first object data may be backed up in the next storage node of the primary storage node. However, in some cases, if the backup fails, it indicates that the first object data needs to be backed up again, so an asynchronous backup task may be generated and recorded in the asynchronous backup task table at this time.

It should be noted that, after determining the primary storage node, the first storage node may determine whether the first storage node is the primary storage node, and if the first storage node is the primary storage node, store the first object data in the local SSD. If the first storage node is not a main storage node, the first object data can be sent to the main storage node for storage, after the main storage node is successfully written, a write success message can be returned, after the first storage node receives the write success message, a next storage node of the main storage node can be determined, the write request is sent to the next storage node of the determined main storage node, and the next storage node of the main storage node stores the first object data into the SSD of the next storage node, so that the first object data is backed up.

In this embodiment, the storage node for backing up the first object data is referred to as a standby storage node, that is, in the above process, the storage node next to the main storage node is the standby storage node.

Further, the first storage node may return a storage result of the first object data to the client, and may return a write success message if the first object data is successfully written to the primary storage node even if the writing to the backup storage node fails. Of course, if the first storage node fails to write the first object data to the primary storage node, a write failure message may be returned to the client.

For example, referring to FIG. 4, the client, storage node 1 and storage node 2 synchronize versions of hash ring information from the management module. The client determines a corresponding storage node 1 according to the first hash parameter and the first hash ring information, writes first object data into the storage node 1, stores the first object data in the first storage node 1, and determines a storage node 2 for backing up the first object data from the first hash ring. The first storage node sends the first object data to the storage node 2, the storage node 2 backs up the first object data and returns a storage result, if the storage result is a write failure message, the storage node 2 indicates that the backup of the first object data by the storage node 2 fails, and at this time, the storage node 1 records the asynchronous backup task.

A second possible implementation: and if the main storage node or the standby storage node for storing the first object data is offline, generating the asynchronous replication task.

In some embodiments, some abnormalities may exist in the storage nodes in the cloud storage system, such as a server crash, a system crash, and the like, which cause the storage nodes to be offline and unable to provide object read-write services to the outside. If the main storage node or the standby storage node for storing the first object data is offline, only one storage node in the cloud storage system is caused to store the first object data, that is, the cloud storage system does not have a backup of the first object data.

In an implementation, the first storage node may read a replica supplementary task, the replica supplementary task being generated in a case where it is determined that the storage node offline event exists, the replica supplementary task including node information of the offline storage node and an offline time point, and polling index information in a local object index information table. If the offline storage node is determined to be a storage node used for storing second object data corresponding to target index information according to the node information, the offline time point and the index information in the object index information table, recording the asynchronous backup task in the asynchronous backup task table based on the target index information, wherein the target index information is the index information in the object index information table.

As described above, some storage nodes in the cloud storage system are provided with management modules, and the management modules can patrol the states of other storage nodes connected to the management modules. If a storage node is offline, the management module will generate a new hash ring, for example, referring to fig. 5, the storage node list corresponding to hash ring version 1 is: a- > B- > C- > D- > E; if the storage node B is offline, the hash ring version 2 is generated, and the corresponding storage node list is: a- > C- > D- > E, and the new object data writing is calculated and positioned according to the queue.

As an example, the management module detects that a storage node is offline, and may generate a storage node offline event, where the storage node offline event includes node information of the offline storage node and an offline time point. Further, in order to avoid misjudgment of the storage node being temporarily offline due to occasional power failure, the management node may also regenerate the storage node offline event after detecting that the storage node is offline for more than a certain time threshold.

The time threshold may be set according to actual requirements, for example, the time threshold may be 2 hours, which is not limited in this embodiment of the application.

After generating the storage node offline event, the management module may synchronize the storage node offline event to other online storage nodes.

For a first storage node, if the management module is configured in the first storage node, the first storage node may generate the storage node offline event when it is detected that the storage node is offline (or offline exceeds a certain time threshold), or, if the management module is not configured in the first storage node, the first storage node receives the storage node offline event sent by the management module.

After the first storage node determines that the storage node offline event exists, the first storage node may record the copy supplement task into the copy supplement task table. In this way, the first storage node may periodically patrol the replica supplementary task table, read the replica supplementary task, and patrol the index information in the local object index information table, where the object index information table may store the index information of all the object data, that is, the number of the index information in the object index information table may be multiple. The first storage node queries whether the offline storage node is a storage node for storing second object data corresponding to target index information according to node information of the offline storage node, an offline time point and index information in an object index information table, if so, it indicates that only one storage node in the current cloud storage system stores the second object data corresponding to the target index information, in this case, based on the target index information, the asynchronous backup task is generated and added to the asynchronous backup task table to facilitate subsequent backup, for example, if the management module detects that the storage node B is offline for more than 2 hours, a copy supplement task is triggered, and at this time, a copy mutual backup relationship should be changed: [ AC ], [ CD ], [ DE ], [ EA ].

As an example, the index information includes a storage time point of the corresponding object data and corresponding hash ring information, in this case, if there is index information whose corresponding storage time point is before the offline time point in the object index information table, the index information whose corresponding storage time point is before the offline time point is determined as the target index information. And if the offline storage node is determined to be the last storage node of the first storage node in the second hash ring corresponding to the target index information according to the node information and the hash ring information corresponding to the target index information, or the offline storage node is determined to be the next storage node of the first storage node, determining that the offline storage node is a storage node for storing second object data corresponding to the target index information.

It is understood that, if there is index information whose corresponding storage time point is before the offline time point in the object index information table, it indicates that there is object data stored before the offline time point in the first storage node, in this case, the object data may be stored in the offline storage node, and for further confirmation, the first storage node determines that there is index information whose corresponding storage time point is before the offline time point in the object index information table as target index information.

Then, a corresponding second hash ring may be determined according to hash ring information corresponding to the target index information, and then it is determined whether the offline storage node is a previous storage node of the first storage node in the second hash ring, if the offline storage node is a previous storage node of the first storage node, it is stated that the offline storage node is a main storage node for storing the second object data, it is understood that the offline storage node may be determined as a storage node for storing the second object data corresponding to the target index information. In addition, if the offline storage node is not the last storage node of the first storage node, but the offline storage node is the next storage node of the first storage node, which means that the offline storage node is a standby storage node for storing the second object data, it is understood that at this time, it may be determined that the offline storage node is a storage node for storing the second object data corresponding to the target index information.

Further, after the first storage node retrieves all index information in the object index information table, the first storage node may delete the record in the duplicate supplementary task table.

In this manner, after recording the asynchronous replication task in the asynchronous replication task table, the first storage node may periodically poll the asynchronous replication task table, i.e., read the asynchronous replication task, in order to perform a replication operation, as described below.

Step 302: and determining a second storage node according to the first hash parameter and the first hash ring information, wherein the second storage node is a storage node used for backing up the first object data.

That is, after the asynchronous backup task of the first object data is read by the first storage node, the storage node for backing up the first object data may be determined, that is, the second storage node is determined, so that the first object data may be backed up subsequently, thereby completing the asynchronous backup task.

As an example, determining a specific implementation of the second storage node according to the first hash parameter and the first hash ring information may include the following:

3021: and determining whether other storage nodes exist in the first hash ring online or not based on the first hash ring information, wherein the other storage nodes are storage nodes except the first storage node.

The first hash ring information may include a storage node list corresponding to the first hash ring, and therefore, according to the first hash ring information, other storage nodes in the first hash ring may be determined, and then, the states of the other storage nodes may be determined, so as to determine whether the other storage nodes are online currently.

As an example, if there are other storage nodes online in the first hash ring, it is stated that the second storage node can be determined from the first hash ring, in which case 3022 and 3025 are performed as follows. If no other storage node is online in the first hash ring, it indicates that the second storage node cannot be determined from the first hash ring, in which case the following step 3026 may be performed.

3022: if other storage nodes exist in the first hash ring online, determining the number of the storage nodes in the first hash ring according to the first hash ring information, performing hash operation according to the first hash parameter and the number of the storage nodes to obtain a hash value, and determining the storage node corresponding to the hash value from the first hash ring.

In implementation, the first storage node may count the number of storage nodes included in the storage node list corresponding to the first hash ring, to obtain the number of storage nodes in the first hash ring. Then, a hash operation may be performed according to the first hash parameter and the number of storage nodes, for example, the hash operation may be performed according to the above formula (1), so as to obtain a hash value, as described above, the hash value indicates the subscript position of the storage node in the first hash ring, so that the corresponding storage node can be determined from the first hash ring according to the hash value.

3023: and if the storage node corresponding to the hash value is the first storage node and the next storage node of the first storage node is online in the first hash ring, determining the next storage node of the first storage node as the second storage node.

That is, if the storage node corresponding to the hash value is the first storage node, it indicates that the first storage node is a main storage node originally used for storing the first object data, and at this time, if the next storage node of the first storage node in the first hash ring is online, the next storage node of the first storage node may be determined as the second storage node.

3024: and if the storage node corresponding to the hash value is the last storage node of the first storage node and the last storage node of the first storage node is online, determining the last storage node of the first storage node as the second storage node.

If the storage node corresponding to the hash value is the last storage node of the first storage node, it indicates that the last storage node of the first storage node is the primary storage node originally used for storing the first object data, so that it can be inferred that the first storage node is the standby storage node originally used for storing the first object data.

3025: if the storage node corresponding to the hash value is the last storage node of the first storage node and the last storage node of the first storage node is not online, if the next storage node of the first storage node is online, determining the next storage node of the first storage node as the second storage node.

That is, if the first storage node is a spare storage node originally used for storing the first object data, a primary storage node originally used for storing the first object data is determined, if the primary storage node is not online, it may be determined whether a next storage node of the first storage node is online, and if online, a next storage node of the first storage node may be determined as a second storage node.

It should be noted that, if the next storage node of the first storage node is not online, it may be continuously determined whether the next storage node of the first storage node is online, and if the next storage node of the first storage node is online, the next storage node of the first storage node is determined as the second storage node. If the next storage node of the first storage node is not online, the inquiry is continued in the mode until the online storage node is inquired from the first hash ring, and the inquired online storage node is determined as the second storage node.

As an example, if the storage node corresponding to the hash value is not the first storage node and is not the last storage node of the first storage node, the storage node corresponding to the hash value is determined as the second storage node if the storage node corresponding to the hash value is online.

That is, if the first storage node is not a primary storage node originally for storing the first object data, or is not a backup storage node originally for storing the first object data, it may be determined whether the primary storage node originally for storing the first object data is online, and if so, the online primary storage node is determined as the second storage node. If the main storage node is not online, determining whether the next storage node of the storage node corresponding to the hash value is online from the first hash ring, if the next storage node of the storage node corresponding to the hash value is online, determining the next storage node of the storage node corresponding to the hash value as a second storage node, if the next storage node of the storage node corresponding to the hash value is offline, continuously querying whether the next storage node of the storage node corresponding to the hash value is online, and so on.

3026: and if no other storage node is online in the first hash ring, acquiring the latest hash ring information, and determining the second storage node from the hash ring corresponding to the latest hash ring information based on the latest hash ring information and the first hash parameter.

It is understood that if no other storage node exists in the first hash ring, it means that the second storage node for backing up the first object data cannot be determined from the first hash ring, in this case, the latest hash ring information may be obtained, and then the second storage node for backing up the first object data may be determined from the hash ring corresponding to the latest hash ring information based on the latest hash ring information and the first hash parameter.

For a specific implementation of determining the second storage node from the hash ring corresponding to the latest hash ring information based on the latest hash ring information and the first hash parameter, reference may be made to determining the second storage node from the first hash ring based on the first hash ring information and the first hash parameter.

Step 303: and backing up the first object data to the second storage node.

After the first storage node determines the second storage node, the first object data is sent to the second storage node and stored by the second storage node, so that the first object data is backed up in the second storage node.

For example, as shown in fig. 5, in one possible implementation, the asynchronous backup task is generated in a case where the storage node B is offline, in which case, according to the above implementation, the storage node C may be determined from the first hash ring as the second storage node, and then the first object data may be backed up into the storage node C.

In addition, it should be noted that, when a certain storage node in the cloud storage system is offline, the management module may generate a new hash ring, the new hash ring may exclude the offline storage node, and subsequent read-write may use the new hash ring to locate the storage node, so as to ensure reliability of the system. For example, the new hash ring may not be used to successfully locate the storage node where the object data is located, so the system may maintain the hash rings of the respective versions, and if the storage node located by the client through the hash ring of the latest version has no object data, destage reading may be performed.

As an example, the first storage node receives a read request, where the read request carries index information and the first hash ring information, returns the first object data if the first object data corresponding to the index information locally exists, and reads the first object data according to the first hash ring information if the first object data corresponding to the index information does not locally exist.

That is, after receiving the read request of the first object data, the first storage node may query whether the first object data exists locally based on the index information of the first object data, and if the first object data exists locally, the first storage node may directly feed back the first object data to the client. Of course, if the index information does not exist locally, the first object data may be further read according to the first hash ring information.

As an example, the read request carries a first hash parameter, and the first hash ring information further includes version information of the first hash ring, in this case, according to the first hash ring information, a specific implementation of reading the first object data may include: and determining the next storage node of the first storage node in the first hash ring according to the first hash ring information. And if the first object data is not read from the next storage node of the first storage node, determining all hash rings with the versions lower than that of the first hash ring according to the version information of the first hash ring. According to the sequence of the versions from high to low, hash rings are selected from all the hash rings for multiple times, each hash ring is selected, a storage node is determined from the currently selected hash ring according to the currently selected hash ring and the first hash parameter, and the first object data is read from the determined storage node; until the first object data is read.

That is, the first storage node may determine, according to the first hash ring information, a storage node next to the first storage node in the first hash ring, and in general, the storage node next to the first storage node is a standby storage node, so that the first storage node may read the first object data from the storage node next to the first storage node, for example, may send a read request to the storage node next to the first storage node, if the storage node next to the first storage node receives the read request, it may query whether the first object data exists locally, if so, send the first object data to the first storage node, and after receiving the first object data, the first storage node feeds back the first object data to the client.

Further, if the first object data does not exist in a storage node next to the first storage node, the storage node next to the first storage node may feed back a read failure message to the first storage node. After receiving the read failure message, the first storage node may select a hash ring whose version is one version lower than that of the first hash ring according to the version information of the first hash ring, then read the first object data from the selected hash ring in the manner described above, and if the first object data is read, return the first object data to the client.

And if the first object data is not read, continuously selecting a hash ring with two versions lower than the version of the first hash ring, then continuously reading the first object data from the selected hash ring in the manner, if the first object data is read, returning the first object data to the client, and if the first object data is not read, continuously selecting the hash ring of the next version in the manner, and reading the first object data until the first object data is read. Of course, if the first object data is not read after all versions of the hash ring are traversed, it is indicated that the first object data is failed to be read, in which case, the first storage node returns a read failure message to the client.

In this embodiment of the application, a first storage node reads an asynchronous backup task in an asynchronous backup task table, where the asynchronous backup task includes a first hash parameter and first hash ring information, the asynchronous backup task refers to a task for backing up first object data in the first storage node, and a first hash ring indicated by the first hash ring information includes a plurality of storage nodes in the cloud storage system. And determining a second storage node according to the first hash parameter and the first hash ring information, wherein the second storage node is a storage node used for backing up the first object data, and backing up the first object data to the second storage node. In the method and the device, the second storage node for backup can be dynamically determined, and the condition that only one copy of object data can be stored can be avoided, so that the storage stability is ensured.

Fig. 6 is a schematic structural diagram of a device for backing up object data according to an embodiment of the present application, where the device for backing up object data may be implemented as part or all of a first storage node by software, hardware, or a combination of the two. Referring to fig. 6, the apparatus includes:

a reading module 610, configured to read an asynchronous backup task in an asynchronous backup task table, where the asynchronous backup task includes a first hash parameter and first hash ring information, the asynchronous backup task is a task that backs up first object data in the first storage node, and the first hash ring indicated by the first hash ring information includes multiple storage nodes in the cloud storage system;

a determining module 620, configured to determine a second storage node according to the first hash parameter and the first hash ring information, where the second storage node is a storage node used for backing up the first object data;

a backup module 630, configured to backup the first object data to the second storage node.

In one possible implementation manner of the present application, the determining module 620 is configured to:

storing the first object data to the primary storage node;

index information in a local object index information table is inspected;

In one possible implementation manner of the present application, the reading module 610 is configured to:

In a possible implementation manner of this application, the read request carries a first hash parameter, the first hash ring information further includes version information of the first hash ring, and the read module 610 is configured to:

It should be noted that: in the backup device for object data provided in the above embodiment, when backing up object data, only the division of each functional module is illustrated, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the object data backup device provided in the above embodiments and the object data backup method embodiment belong to the same concept, and specific implementation processes thereof are described in the method embodiment and are not described herein again.

Fig. 7 is a schematic structural diagram of a storage node according to an embodiment of the present application. The storage node 700 includes a Central Processing Unit (CPU)701, a system memory 704 including a Random Access Memory (RAM)702 and a Read Only Memory (ROM)703, and a system bus 705 connecting the system memory 704 and the central processing unit 701. The storage node 700 also includes a basic input/output system (I/O system) 706, which facilitates transfer of information between devices within the computer, and a mass storage device 707 for storing an operating system 713, application programs 714, and other program modules 715.

The basic input/output system 706 includes a display 708 for displaying information and an input device 709, such as a mouse, keyboard, etc., for a user to input information. Wherein the display 708 and the input device 709 are connected to the central processing unit 701 through an input output controller 710 connected to the system bus 705. The basic input/output system 706 may also include an input/output controller 710 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 710 may also provide output to a display screen, a printer, or other type of output device.

The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer-readable media provide non-volatile storage for the storage node 700. That is, the mass storage device 707 may include a computer-readable medium (not shown), such as a hard disk or CD-ROM drive.

Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 704 and mass storage device 707 described above may be collectively referred to as memory.

According to various embodiments of the present application, the storage node 700 may also operate as a remote computer connected to a network through a network such as the internet. That is, the storage node 700 may be connected to the network 712 through the network interface unit 711 connected to the system bus 705, or may be connected to another type of network or a remote computer system (not shown) using the network interface unit 711.

The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.

In some embodiments, a computer-readable storage medium is also provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the backup method for object data in the above embodiments. For example, the computer readable storage medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

It is noted that the computer-readable storage medium referred to herein may be a non-volatile storage medium, in other words, a non-transitory storage medium.

It should be understood that all or part of the steps for implementing the above embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions may be stored in the computer-readable storage medium described above.

That is, in some embodiments, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the above-described method for backing up object data.

The above-mentioned embodiments are provided not to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for backing up object data is applied to a first storage node, wherein the first storage node is any storage node in a cloud storage system, and the method comprises the following steps:

and backing up the first object data to the second storage node.

2. The method of claim 1, wherein determining a second storage node based on the first hash parameter and the first hash ring information comprises:

3. The method of claim 2, wherein after determining the storage node corresponding to the hash value from the first hash ring, further comprising:

4. The method of claim 2, wherein after determining the storage node corresponding to the hash value from the first hash ring, further comprising:

5. The method of claim 2, wherein after determining whether other storage nodes are online in the first hash ring based on the first hash ring information, further comprising:

6. The method of claim 1, wherein the method further comprises:

storing the first object data to the primary storage node;

7. The method of claim 6, wherein after determining the next storage node of the primary storage nodes from the first hash ring, further comprising:

8. The method of claim 1, wherein the method further comprises:

index information in a local object index information table is inspected;

9. The method of claim 8, wherein the index information includes a storage time point of the corresponding object data and corresponding hash ring information, the method further comprising:

10. The method of claim 1, wherein the method further comprises:

11. The method of claim 10, wherein the read request carries a first hash parameter, wherein the first hash ring information further includes version information of the first hash ring, and wherein reading the first object data according to the first hash ring information comprises:

12. An apparatus for backing up object data, configured to a first storage node, where the first storage node is any storage node in a cloud storage system, the apparatus comprising:

13. A computer device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, the memory stores a computer program, and the processor executes the program stored in the memory to implement the steps of the method according to any one of claims 1-11.

14. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 11.