CN103384550A - Data storage method and device - Google Patents

Data storage method and device Download PDF

Info

Publication number
CN103384550A
CN103384550A CN2012800036568A CN201280003656A CN103384550A CN 103384550 A CN103384550 A CN 103384550A CN 2012800036568 A CN2012800036568 A CN 2012800036568A CN 201280003656 A CN201280003656 A CN 201280003656A CN 103384550 A CN103384550 A CN 103384550A
Authority
CN
China
Prior art keywords
child resource
section
resource pond
data
copy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2012800036568A
Other languages
Chinese (zh)
Other versions
CN103384550B (en
Inventor
田晓波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN103384550A publication Critical patent/CN103384550A/en
Application granted granted Critical
Publication of CN103384550B publication Critical patent/CN103384550B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2094Redundant storage or storage space

Abstract

The embodiment of the invention discloses a data storage method and device, belonging to the technical field of electronic information, which can store data in multiple resource pools to reduce the probability of completely damaging the data and improve the security of data. The data storage method is used for the storage system, the resource pools in the storage system are logically divided into at least 2 sub resource pools, each sub resource pool comprises adjacent N storage apparatus, N being larger than and equal to 2. The data storage method comprises the following steps: dividing the data to be stored into R slices, distributing the slices into the divided sub resource pools, wherein the number of the desirably generated copies of the data to be stored is small than N, R being larger than and equal to 2; generating a copy slice of each slice distributed in each sub resource pool and storing the copy slice in the sub resource pool corresponding to the slice, wherein the copy slice is corresponded to the same slice of the data to be stored.

Description

The method of storage data and device
Technical field
The present invention relates to electronic information technical field, relate in particular to a kind of method and device of storage data.
Background technology
The distributed storage technology is a kind of data storage technology commonly used, under the distributed storage scene, hard disks all in system can be carried out unified management as a memory resource pool, all whole memory resource pool be carried out data interaction as a storage area when in system, data being read and write and storing.For example:
Distributed memory system can be mapped to a virtual volume on whole hard disks in memory resource pool, and the spatial division that can roll up is the data block of several 1M sizes, and a piece in the whole hard disks in the corresponding resource pool of each data block is deposited logical partition unit Partition.When Partition has a plurality of copy, the interior total data of Partition can be saved many parts, such as: under two-pack scene originally, two Partition in the whole hard disks in the corresponding resource pool of each data block, and the content of these 2 Partition is identical.At basis allocation algorithm commonly used, identical two Partition are as P1 and P1 ', tend to be dispensed on two different hard disks as: P1 is at hard disk 1, P1 ' is at hard disk 2, Partition can be distributed in whole resource pool fifty-fifty, can make the read or write speed maximization of hard disk in resource pool due to the mode of this distribution Partition, thereby be adopted widely.
When in distributed memory system during any one hard disk corruptions, can carry out data reconstruction, for example: hard disk 1 damages, because the P1 on hard disk 1, P4 ', P5, P8 ' have corresponding copy on hard disk 2,3,4, can be by corresponding copy data of repairing hard disk 1 all on hard disk 2,3,4.
Any one hard disk corruptions in existing distributed memory system, can carry out data reconstruction, in the process of carrying out data reconstruction, if other hard disks fault again in memory resource pool, the data of whole virtual volume will be damaged, for example: as shown in Figure 1, when hard disk 1 damages and begin to carry out the data reparation, if hard disk 2 is damaging, the copy P1 ' that is stored in the correspondence that is used for reparation hard disk 1 on hard disk 2 will damage, thereby can't repair data, cause the corrupted data of whole virtual volume, reduced the security of data.
Summary of the invention
Embodiments of the invention provide a kind of method and device of storage data, all memory devices in storage system can be divided at least two resource pools, and store the data in the resource pool that marks off, because the data in each resource pool are independently, only have 2 memory devices in same resource pool to damage, data just can be damaged fully, thereby have reduced the probability that data are damaged fully, the security that has improved data.
For achieving the above object, embodiments of the invention adopt following technical scheme:
First aspect, embodiments of the invention provide a kind of method of storage data, be used for storage system, described storage system comprises the resource pool for the storage data, described resource pool is at least 2 sub-resource pools by logical partitioning, each child resource pond comprises an adjacent N memory device, N 〉=2, and described method comprises:
The data of needs storages are divided into R section, R described section are distributed in the described child resource pond of division, the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
To being distributed to each the described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section, the described copy section described section identical with data is mutually corresponding.
In conjunction with first aspect, in the possible implementation of the first of first aspect, the described child resource pond that described copy is stored under corresponding described section comprises: described copy section is stored on memory device in child resource pond under corresponding described section, and the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
In conjunction with first aspect, and the possible implementation of the first of first aspect, in the possible implementation of the second of first aspect, also comprise: the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different;
Described to being distributed to each the described section ghost section in each child resource pond, comprise: the copy number of sections that generates according to corresponding each the described section expectation of the rank of child resource pond set under the child resource pond will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
In conjunction with first aspect, and the possible implementation of the first of first aspect, in the third possible implementation of first aspect, in the child resource pond under described copy section being stored in corresponding described section after, also comprise:
For any described child resource pond, during the quantity of the memory device in reducing described child resource pond, described section in the memory device that reduces is assigned on other memory devices in described child resource pond the number of copies that the remaining memory device quantity in described child resource pond generates greater than described expectation.
In conjunction with first aspect, and the possible implementation of the first of first aspect, in the 4th kind of possible implementation of first aspect, in the child resource pond under described copy section being stored in corresponding described section after, also comprise:
For any described child resource pond, during the quantity of the memory device in increasing described child resource pond, the copy section of the described section in this child resource pond is re-assigned on memory device in this child resource pond.
In conjunction with the possible implementation of the second of first aspect, in the 5th kind of possible implementation, describedly R section be distributed in advance the child resource pond of dividing comprise:
According to the significance level of data in R section, in R the child resource pond that is distributed to different stage of cutting into slices.
Second aspect, embodiments of the invention provide a kind of device of storage data, be used for storage system, described storage system comprises the resource pool for the storage data, described resource pool is at least 2 sub-resource pools by logical partitioning, each child resource pond comprises an adjacent N memory device, N 〉=2, and described method comprises:
Distribution module, the data that are used for needing to store are divided into R section, R described section are distributed in the described child resource pond of division, and the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
The copy generation module is used for being distributed to each described section ghost section in each child resource pond;
Storage module is used for described copy section is stored in child resource pond under corresponding described section, and the described copy section described section identical with data is mutually corresponding.
In conjunction with second aspect, in the possible implementation of the first of second aspect, also comprise: described storage module, also be used for described copy section is stored into the memory device in the child resource pond under corresponding described section, and the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
In conjunction with second aspect, and the possible implementation of the first of second aspect, in the possible implementation of the second of second aspect, also comprise:
Divide module, be used for the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different;
Described copy generation module also is used for the copy number of sections that corresponding each the described section expectation of rank according to child resource pond set under the child resource pond generates, and will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
In conjunction with second aspect, and the possible implementation of the first of second aspect, in the third possible implementation of second aspect, also comprise:
The negative regulator module is used for for any described child resource pond, reduces the quantity of the memory device in described child resource pond;
Data allocations module, the described section that is used for the memory device that will reduce are assigned to other memory devices in described child resource pond, the number of copies that the remaining memory device quantity in described child resource pond generates greater than described expectation.
In conjunction with second aspect, and the possible implementation of the first of second aspect, in the 4th kind of possible implementation of second aspect, also comprise:
Positive adjustment module is used for for any described child resource pond, increases the quantity of the memory device in described child resource pond;
Described data allocations module, when also being used for the quantity when the memory device that increases described child resource pond, the copy of the described section in this child resource pond is cut into slices to be re-assigned on memory device in this child resource pond.
In conjunction with the possible implementation of the second of second aspect, in the 5th kind of possible implementation, also comprise:
Described distribution module also is used for the significance level according to R section data, in R the child resource pond that is distributed to different stage of cutting into slices.
The third aspect, embodiments of the invention provide a kind of storage system of storage data, described storage system comprises the resource pool for the storage data, described resource pool is at least 2 sub-resource pools by logical partitioning, each child resource pond comprises an adjacent N memory device, N 〉=2, described storage system comprises: processor, communication interface, bus, characterized by further comprising memory device, described processor, described communication interface and all memory devices, complete mutual communication by described bus, wherein:
Described processor, the data that are used for needing to store are divided into R section, by described communication interface, R described section are distributed in the described child resource pond of division, and the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
Described processor also is used for being distributed to each described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section, and the described copy section described section identical with data is mutually corresponding.
In conjunction with the third aspect, in the possible implementation of the first of the third aspect, comprising:
Described processor, also be used for described copy section is stored into the memory device in the child resource pond under corresponding described section, and the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
In conjunction with the third aspect, and the possible implementation of the first of the third aspect, in the possible implementation of the second of the third aspect, described processor, also be used for the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different;
Described processor also is used for the copy number of sections that corresponding each the described section expectation of rank according to child resource pond set under the child resource pond generates, and will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
in conjunction with the third aspect, and the possible implementation of the first of the third aspect, in the third possible implementation of the third aspect, described processor, also be used for behind the child resource pond that described copy section is stored under corresponding described section, for any described child resource pond, during the quantity of the memory device in reducing described child resource pond, by described communication interface, the described section in the memory device that reduces is assigned on other memory devices in described child resource pond, the number of copies that remaining memory device quantity in described child resource pond generates greater than described expectation.
In conjunction with the third aspect, and the possible implementation of the first of the third aspect, in the 4th kind of possible implementation of the third aspect, described processor, also be used for behind the child resource pond that described copy section is stored under corresponding described section, for any described child resource pond, during the quantity of the memory device in increasing described child resource pond, by described communication interface, the copy section of the described section in this child resource pond is re-assigned on memory device in this child resource pond.
In conjunction with the possible implementation of the second of the third aspect, in the 5th kind of possible implementation, described processor also is used for the significance level according to R section data, in R the child resource pond that is distributed to different stage of cutting into slices.
Fourth aspect, embodiments of the invention provide a kind of computer program of storage data, be used for storage system, described storage system comprise for the storage data resource pool, it is characterized in that, described resource pool is at least 2 sub-resource pools by logical partitioning, each child resource pond comprises an adjacent N memory device, N 〉=2, described computer program comprises the computer-readable recording medium of having stored program code, the instruction that described program code comprises is used for:
The data of needs storages are divided into R section, R described section are distributed in the described child resource pond of division, the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
To being distributed to each the described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section, the described copy section described section identical with data is mutually corresponding.
The method of the repair data that the embodiment of the present invention provides, device, storage system and computer program, all memory devices in storage system can be divided at least two sub-resource pools, and store the data in the child resource pond that marks off, because the data in each child resource pond are independently, only have 2 memory devices in same child resource pond to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device will cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of child resources ponds, and store the data in a plurality of child resources pond that marks off, only have when 2 memory devices in same child resource pond damage, data just can be damaged fully, the probability that damages due to 2 memory devices in same child resource pond, less than in prior art in the situation that only have resource pool to damage the probability of 2 memory devices, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Description of drawings
In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, the below will do to introduce simply to the accompanying drawing of required use in embodiment, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
The flow chart of the method for a kind of storage data that Fig. 1 provides for the embodiment of the present invention;
The schematic diagram of a kind of instantiation of the method for a kind of storage data that Fig. 1 a provides for the embodiment of the present invention;
The schematic diagram of the another kind of instantiation of the method for a kind of storage data that Fig. 1 b provides for the embodiment of the present invention;
The flow chart of the method for the another kind of storage data that Fig. 2 provides for the embodiment of the present invention;
A kind of flow chart of the method for another storage data that Fig. 3 a provides for the embodiment of the present invention;
The schematic diagram of a kind of instantiation of the method for another storage data that Fig. 3 a1 provides for the embodiment of the present invention;
The another kind of flow chart of the method for another storage data that Fig. 3 b provides for the embodiment of the present invention;
The schematic diagram of the another kind of instantiation of the method for another storage data that Fig. 3 b1 provides for the embodiment of the present invention;
A kind of flow chart of the method for another storage data that Fig. 4 a provides for the embodiment of the present invention;
The another kind of flow chart of the method for another storage data that Fig. 4 b provides for the embodiment of the present invention;
The structural representation of the device of a kind of storage data that Fig. 5 provides for the embodiment of the present invention;
The structural representation of the device of the another kind of storage data that Fig. 6 provides for the embodiment of the present invention;
The network architecture schematic diagram of the storage system of a kind of storage data that Fig. 7 provides for the embodiment of the present invention.
The specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills belong to the scope of protection of the invention not making all other embodiment that obtain under the creative work prerequisite.
On the one hand, the embodiment of the present invention provides a kind of method of storage data, as shown in Figure 1, comprising:
Need to prove, the embodiment of the present invention can be used for storage system, and described storage system comprises the resource pool for the storage data, it is characterized in that, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises an adjacent N memory device, N 〉=2.And the embodiment of the present invention can be carried out by the equipment that possesses transfer of data, processing, memory function, such as: the equipment such as management server, mobile workstation that are used for management hard disk array in distributed memory system.
101, the data of needs storages are divided into R section, the individual described section of R is distributed in the described child resource pond of division.
Wherein, need copy amount that the data expectation of storage generates less than N, the copy amount that the data expectation that namely needs to store generates is less than the quantity of the memory device in the child resource pond, and R 〉=2.For example: the data slicer distribution can be broken up the technological means in the distributed storage framework, such as: the section that is assigned in child resource pond 1 is P1, P2, P3, the copy amount that expectation generates is 2, management server can be P1, P2, P3 according to the section that is assigned in child resource pond 1, generate copy section P1 ', P2 ', the P3 ' of the first authentic copy, and the section of the copy of triplicate P1 ' ', P2 ' ', P3 ' '.And the technological means of breaing up by distribution is stored in P1, P2, P3, P1 ', P2 ', P3 ', P1 ' ', P2 ' ', P3 ' ' on memory device in child resource pond 1.
In the present embodiment, the data that management server can will need to store by technological means commonly used are divided into a plurality of data slicers, for example: the data that management server can will need by the data slicer method in RAID technology well-known to those skilled in the art to store are divided at least 2 data sections, and the data slicer that marks off is distributed on the child resource pond.And, in the present embodiment, the specific embodiment of distributed data section can have multiple on the child resource pond, for example: the mode that management server can be broken up by the distribution in the RAID technology is distributed in data slicer on the child resource pond that marks off, and also the data slicer draw can be distributed on the child resource pond that marks off.
Further, in the present embodiment, management server can be divided into the memory device in system a plurality of child resources ponds on logic level, and each child resource pond is comprised of a plurality of memory devices, for example:
Always have 24 hard disks in distributed memory system, management server can pass through MDC(Metadata Controller, metadata controller) be 3 sub-resource pools with these 24 hard disk partition, each child resource pond has comprised 8 hard disks, thereby can form the framework in many child resources pond as shown in Figure 1a.
in actual applications, carry out LBA(Logical Block Addressing commonly used in the process of addressing in stored data, LBA) can be mapped in different child resources ponds, each child resource pond and piece are deposited logical partition unit Partition and can be distributed by the cluster management node of MDC, such as: in each child resource pond, n Partition arranged, on each hard disk in each child resource pond, a plurality of Partition can be arranged, the PartitionID(piece that can record in MDC in each child resource pond is deposited logical partition unit ID) and child resource pond ID, and by each Partition in the form sign DISK to Image array of " [child resource pond ID.PartitionID] " this character information, thereby make each Partition that unique identification information be arranged, other equipment in system can adopt Partition in technological means access child resource commonly used pond according to the identification information of each Partition.
102, to being distributed to each the described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section.
In the present embodiment, the data that need to store are expected to generate at least two identical copies, and the described data of storage that need expect that the number of copies of generation is less than N.And copy section is mutually corresponding with the identical section of the data that need storage, is stored in a copy in sub-resource pool and cuts into slices and cut into slices identical with other copies that is stored in this child resource pond.That is, the section of the copy in sub-resource pool is cut into slices different with the copy in other child resources ponds.For example: as shown in Fig. 1 b, copy section in the first authentic copy can be managed server and be divided into P1-P8, copy section in triplicate can be divided into P1 '-P8 ', and the first authentic copy is identical with the content of triplicate, be that P1 is identical with P1 ', P2 is identical with P2 ', until P8 is identical with P8 '.
In child resource pond 1, hard disk 1 has carried P1, P2 ', P5, P6 ', hard disk 2 has carried P1 ', P2, P5 ', P6, the content that is the first authentic copy in child resource pond 1 and the triplicate content in child resource pond 1 is identical, in like manner, the content of the first authentic copy in child resource pond 2 and the triplicate content in child resource pond 2 is identical.
Further, such as: P1 and P1 ' are in child resource pond 1, and namely the Partition identical with P1 only is present in child resource pond 1, in like manner, identical Partition makes the data in sub-resource pool be independent of in terms of content other child resources ponds in same child resource pond.When hard disk corruptions, such as: hard disk 1 damages, only need according to P1 ', P2, P5 ', P6 in the hard disk 2 in the child resource pond 1 that coexists, can repair P1, P2 ', P5, P6 ' in hard disk 1, make when the hard disk corruptions in a sub-resource pool, management server only needs can repair data in impaired hard disk according to the data in other hard disks in this child resource pond, thereby the data in making the child resource pond are independent of other child resources ponds in terms of content.
need to prove, in embodiments of the present invention, because the copy in sub-resource pool section is cut into slices different with the copy in other child resources ponds, make data in the child resource pond be independent of in terms of content other child resources ponds, and the copy that the data that store as required generate is stored on a plurality of child resources ponds, the condition damaged fully of whole copy data is: have 2 memory devices to damage simultaneously in a sub-resource pool, for example: under the scene originally of the two-pack as shown in Fig. 1 b, the condition that whole copy data damages fully is: hard disk 1 and hard disk 2 in child resource pond 1 damage simultaneously, or hard disk 3 and hard disk 4 in child resource pond 2 damage simultaneously.If occur only damaging a hard disk in child resource pond 1, also only damage a hard disk in child resource pond 2 simultaneously, copy data still can be repaired, and when namely only occurring the situation that 2 hard disks in a sub-resource pool damage simultaneously, copy data just can damage.Because the damage probability of hard disk is identical basically, if the damage probability of hard disk is A, if adopt the scheme of prior art, such as: the scheme of prior art be hard disk 1,2,3,4 in Fig. 1 b all in a resource pool, the probability (i.e. 2 probabilities that hard disk damages simultaneously) that damages of copy data is A 2, or to can be understood as the probability that distributed memory system copy data in the time window of data reconstruction damages be A; And the scheme of the embodiment of the present invention, hard disk 1 in Fig. 1 b, 2 is in child resource pond 1, hard disk 3,4 is in child resource pond 4, and because the data in child resource pond 1 are independent of child resource pond 2 in terms of content, the probability that damages of copy data (must be namely same child resource pond in 2 probabilities that hard disk damages simultaneously) is 0.5A 2, or to can be understood as the probability that distributed memory system copy data in the time window of data reconstruction damages be 0.5A; Hence one can see that, in the scheme of the embodiment of the present invention, the probability that damages of copy data is lower more at most in the child resource pond that marks off, such as: mark off 100 sub-resource pools, the probability that copy data damages in the time window of data reconstruction is 0.01A, if a hard disk is just in time damaged in each child resource pond, the scheme that provides of the embodiment of the present invention can be tackled maximum 100 this extreme cases of hard disk corruptions, and because the data in each child resource pond are separate, can also repair data.
Further can give an example, in actual applications, the fault rate of memory device such as hard disk is generally 4%, then considers that the performance parameter of the memory device that some are commonly used is on the impact of hard disk failure rate, according to the scheme of the embodiment of the present invention, can obtain data as shown in Table 1:
Total hard disk number: 180 dishes The corrupted data rate Single hard disk failure rate
Existing scheme: 1 sub-resource pool (every pond 180 hard disks) 0.997066418 4%
The present invention program: 90 sub-resource pools (every pond 2 hard disks) 0.999983563 4%
Other data storage scheme commonly used of this area 0.996077775 4%
Table one
This shows, the embodiment of the present invention is with respect to prior art, the child resource pond comprises an adjacent N memory device, and the probability that adjacent memory device damages simultaneously is very little, therefore the memory device that damages the not probability in same child resource pond is very large, and the data in a sub-resource pool are damaged, thereby just can recover to increase by the data on other memory devices in this child resource pond the fault-tolerance of distributed memory system, reduce the probability of corrupted data, thereby improved the security of data.
the method of the repair data that the embodiment of the present invention provides, all memory devices can be divided at least two sub-resource pools, and store the data in the child resource pond that marks off, because the data in each child resource pond are independently, for example: as shown in Fig. 1 b, in child resource pond 1, the data slicer of storage is P1 ', P2, P5 ', P6, P1, P2 ', P5, P6 ', identical without any the data slicer of storing in the section of data and child resource pond 2, whether the corrupted data in child resource pond 2 can not have influence on the data in child resource pond 1, and do not need to extract data in child resource pond 1 and participate in repairing data in child resource pond 2, therefore the data in each child resource pond are independently.Because the data in each child resource pond are independent, corrupted data in sub-resource pool whether, the process such as data reparation can not have influence on the data in other child resource pond, only have 2 memory devices in same child resource pond to damage, data just can be damaged fully.in prior art, the memory device that stores mutual recoverable data may damage simultaneously, cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of child resources ponds, each child resource pond comprises N adjacent memory device, and store the data in a plurality of child resources pond that marks off, only have when the memory device at the mutual recoverable data place in same child resource pond all damages, data just can be damaged fully, thereby the very little embodiment of the present invention that makes of probability of all damaging due to the adjacent memory device in same child resource pond can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Optionally, the embodiment of the present invention also provides a kind of method of storage data, as shown in Figure 2, can also comprise:
201, the data of needs storages are divided into R section, the individual described section of R is distributed in the described child resource of the memory device that comprises specified type pond.
Wherein, a sub-resource pool can comprise at least 2 adjacent memory devices, and memory device is used for the storage data.
And the child resource pond that obtains comprises the memory device of specified type, and the type of memory device includes but not limited to: hard disk, solid-state hard disk SSD.
202, to being distributed to each the described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section.
In the present embodiment, management server can be determined the type of the memory device that will use, and will need the section of the data that store to be stored in the child resource pond that comprises the type memory device, again to being distributed to each the described section ghost section in each child resource pond, and the section of described copy is stored in child resource pond under corresponding described section, for example:
Mapping relations between the type of the data as shown in Table 2 that can prestore in management server and the type of memory device, and the type of the definite memory device that will use of the type of the data that store as required, obtain again the child resource pond that comprises the type memory device, will need at last the data storing that stores in comprising the child resource pond of the type memory device.
Data type Storage device type The child resource pond at memory device place
Picture Mechanical hard disk Child resource pond 1
Word Solid state hard disc Child resource pond 2
Video Mechanical hard disk Child resource pond 3
Table two
Again for example:
Mapping relations between the size of the data as shown in Table 3 that can prestore in management server and the type of memory device, and the type of the definite memory device that will use of the size of the data that store as required, obtain again the child resource pond that comprises the type memory device, will need at last the data storing that stores in comprising the child resource pond of the type memory device.
Size of data Storage device type The child resource pond at memory device place
100M Solid state hard disc Child resource pond 1
1G Mechanical hard disk Child resource pond 2
1T Mechanical hard disk Child resource pond 3
Table three
In the present embodiment namely, the type of the definite memory device that will use of management server is specifically implemented means can be had multiple, in actual applications, those skilled in the art also can be on the basis of the embodiment of the present invention determine the type of the memory device that will use in conjunction with the prior art means.Because the security of dissimilar memory device is also different, for example: the probability of the damage of at present a lot of solid state hard discs is less than mechanical hard disk.Thereby management server can be selected according to the significance level of data the memory device of respective type, thereby the security that has further improved significant data.
Further alternative, the embodiment of the present invention provides again a kind of method of storage data, wherein as shown in Fig. 3 a, comprising:
301, the data of needs storages are divided into R section, the individual described section of R is distributed in the described child resource pond of division.
302, to being distributed to each the described section ghost section in each child resource pond, and described copy section is stored on memory device in child resource pond under corresponding described section.
Wherein, the copy section is mutually corresponding with the identical section of the data that need storage.The memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.For example:
As shown in Fig. 1 b, the section of the copy of the first authentic copy is P1-P8, and the copy section of triplicate is P1 '-P8 ', and P1 is identical with P1 ', and P2 is identical with P2 ', until P8 is identical with P8 '.In child resource pond 1, hard disk 1 has carried P1, P2 ', P5, P6 ', hard disk 2 has carried P1 ', P2, P5 ', P6, the content that is the first authentic copy in child resource pond 1 and the triplicate content in child resource pond 1 is identical, in like manner, the content of the first authentic copy in child resource pond 2 and the triplicate content in child resource pond 2 is identical.
Further, P1 and P1 ' are in child resource pond 1, and namely the Partition identical with P1 only is present in child resource pond 1, in like manner, identical Partition makes the data in sub-resource pool be independent of in terms of content other child resources ponds in same child resource pond.And the hard disk of each copy section storage is different with the hard disk that corresponding section is stored, such as a: copy in child resource pond 1 cut into slices with child resource pond 2 in any section of any copy not identical.And the copy section that data are identical is stored on different memory devices, such as: P1 and P1 ' are on different hard disks.
303a for any described child resource pond, reduces the quantity of the memory device in described child resource pond.
In the present embodiment, management server can be realized by technological means commonly used the hot plug/unplug of the memory device in the child resource pond, namely can increase or reduce the quantity of the memory device in the child resource pond in the process that the child resource pond is being moved.For example: when a certain hard disk failure in the group resource pool also need to carry out data reconstruction by technological means commonly used, management server can be rebuild recovery with the data on faulty hard disk in the book resource pool, as shown in Fig. 3 a1.After hard disk 1 fault in group resource pool x, the Partition on hard disk 1 can be according to realizing that rule by prior setting has all moved in other 3 hard disks in the child resource pond.Thereby realize reducing the quantity of the memory device in the child resource pond in the process that the child resource pond is being moved.
304a is assigned to the described section in the memory device that reduces on other memory devices in described child resource pond.
Arranged side by side, as shown in Fig. 3 b, can also comprise:
303b for any described child resource pond, increases the quantity of the memory device in described child resource pond.
For example: as shown in Fig. 3 b1.After having increased hard disk 4 in management server antithetical phrase resource pool x, Partition on hard disk 1-3 can pass through the prior rule mean allocation of setting in this 4 fast hard disk of hard disk 1-4 according to realization, thereby the partial content on hard disk 1-3 has been moved in hard disk 4.Thereby realize increasing the quantity of the memory device in the child resource pond in the process that the child resource pond is being moved.
304b is re-assigned to the copy section of the described section in this child resource pond on memory device in this child resource pond.
In the present embodiment, management server can the memory device in having increased the child resource pond after, the section in the child resource pond is redistributed, such as: as shown in Fig. 3 b1, with all data slicers on hard disk 1-3, re-starting on hard disk 1-4 distributes breaks up.
The embodiment of the present invention can also realize the hot plug/unplug of the memory device in the child resource pond; make each child resource pond when the abnormal running situations such as memory device damages, memory device is shut down maintenance occurring; the data that can guarantee to be stored in the child resource pond are not lost; make distributed memory system in the situation that when operation can guarantee that data stabilization get rid of obsolete memory device at any time, thereby further improved the security of data.
The method of the repair data that the embodiment of the present invention provides, all memory devices can be divided at least two sub-resource pools, and store the data in the child resource pond that marks off, because the data in each child resource pond are independently, comprise that take a sub-resource pool two memory devices are as example, only have 2 memory devices in same child resource pond to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device just may cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of child resources ponds, and store the data in a plurality of child resources pond that marks off, only have when 2 memory devices in same child resource pond damage, data just can be damaged fully, the probabilities that damage due to 2 in same child resource pond adjacent memory devices, less than storing the probabilities that can mutually carry out the memory device that data recover in the situation that only have resource pool to damage 2 in prior art, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Further alternative again, the embodiment of the present invention also provides a kind of method of storage data, can also comprise: the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different; Described to being distributed to each the described section ghost section in each child resource pond, comprise: the copy number of sections that generates according to corresponding each the described section expectation of the rank of child resource pond set under the child resource pond will be distributed to each described section in the child resource pond and generate the copy section of respective amount.In specific implementation, can as shown in Fig. 4 a, comprise:
401a is divided into the pond set of different stage child resource again with the child resource pond of described division.
Wherein, the pond set of each child resource comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different.For example:
Determine Q one-level child resource pond, or the individual secondary child resource of Q ' pond.Wherein, the copy amount that the section expectation of one-level child resource pond storage generates is M, and the copy amount of the section expectation generation of secondary child resource pond storage is O, and M>O 〉=2, Q 〉=2, Q ' 〉=2.
In the present embodiment, management server can be set by technological means commonly used the copy amount of the section expectation generation that can store in the child resource pond, for example: copy 1,2,3 content are identical, the copy amount that in the x of child resource pond, the section expectation generates is 2, and the copy amount that in child resource pond y, the section expectation generates is 3.In child resource pond x, storing from the section of the copy in copy 1 P1, P2, P3, after cut into slices from the copy in copy 2 P1 ', P2 ', P3 ', just can not store again from the section of the copy in copy 3 P1 ' ', P2 ' ', P3 ' ', if will store simultaneously from copy 1,2,3 all or part of content, need to use child resource pond y.
And, can add for the character that limits copy amount at the identification information that is used for expression child resource pond attribute, such as: the quantity of the copy of the data expectation that 010 expression child resource pond permission is stored is that the quantity of the copy of the stored data expectation of 2,011 expression child resource pond permissions is 3.
402a, the copy number of sections that generates according to corresponding each the described section expectation of the rank of child resource pond set under the child resource pond will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
In the present embodiment, the grade mode in the child resource pond that management server is determined to use can be by multiple, for example: the type of the data that management server can store as required, determine the grade in child resource pond, such as: as shown in Table 4, management server is the type of deposit data and the mapping relations between the grade of child resource pond in advance, and the type of the data that store as required and the mapping relations shown in table four determine to store the grade that needs the required child resource pond of the data that store.
Figure BDA00003181918200201
Table four
due in actual applications, the significance level of different pieces of information is also different, the embodiment of the present invention can be taked to the data of different significance levels the storage mode of different brackets, for important comparatively speaking data can be by possessing more copy storage mode to guarantee the security of data, for unessential data comparatively speaking can be by possessing less copy storage mode with the utilization rate of memory device in the increase system, thereby improve the operational efficiency of distributed memory system, make management server more memory device can be used for storing the data of outbalance, thereby further improved the security of significant data.
Arranged side by side, as shown in Fig. 4 b, also comprise:
401b is divided into the pond set of different stage child resource again with the child resource pond of described division.
Wherein, the pond set of each child resource comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different.
402b is divided into R section with the data of needs storages, and the significance level of data in cutting into slices according to R is in R the child resource pond that is distributed to different stage of cutting into slices.
Wherein, R 〉=2.For example:
Need to store the data of system volume of virtual machine and the data of user volume in the child resource pond in using the process of virtual machine, because the security of the data of system volume is more important, management server can be corresponding according to the data of system volume section generate 3 copies and be stored in one-level child resource pond, and section corresponding according to the data of user volume generates 2 copies and is stored in secondary child resource pond, thereby realize generating according to the different piece of the data that store with a needs the copy of varying number, and be stored in the child resource pond of appropriate level.
due in actual applications, the significance level of the different piece of same piece of data is also different, the embodiment of the present invention can be taked for the data of the significance level of the different piece of data the storage mode of different brackets, for storage mode that in data, part and parcel can be by possessing more copy comparatively speaking to guarantee security, for relatively with unessential part can be by possessing less copy storage mode with the utilization rate of memory device in the increase system, thereby improve the operational efficiency of distributed memory system, make management server more memory device can be used for the part of storage data outbalance, thereby further improved the security of pith in data.
The method of the repair data that the embodiment of the present invention provides, all memory devices can be divided at least two sub-resource pools, and store the data in the child resource pond that marks off, because the data in each child resource pond are independently, only have 2 memory devices in same child resource pond to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device will cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of child resources ponds, and store the data in a plurality of child resources pond that marks off, only have when 2 memory devices in same child resource pond damage, data just can be damaged fully, the probability that damages due to 2 memory devices in same child resource pond is very little, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
On the other hand, the embodiment of the present invention provides a kind of device of storage data, be used for storage system, described storage system comprises the resource pool for the storage data, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises an adjacent N memory device, N 〉=2, as shown in Figure 5, comprising:
Distribution module 51, the data that are used for needing to store are divided into R section, the individual described section of R are distributed in the described child resource pond of division.
Wherein, the data that needs are stored are expected the copy amount of generation less than N, R 〉=2.
Copy generation module 52 is used for being distributed to each described section ghost section in each child resource pond.
Storage module 53 is used for described copy section is stored in child resource pond under corresponding described section, and the described copy section described section identical with data is mutually corresponding.
The device of the repair data that the embodiment of the present invention provides, all memory devices can be divided at least two sub-resource pools, and store the data in the child resource pond that marks off, because the data in each child resource pond are independently, only have 2 memory devices in same child resource pond to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device will cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of child resources ponds, and store the data in a plurality of child resources pond that marks off, only have when 2 memory devices in same child resource pond damage, data just can be damaged fully, the probability that damages due to 2 memory devices in same child resource pond, less than in prior art in the situation that only have resource pool to damage the probability of 2 memory devices, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Optionally, as shown in Figure 6, the device of the storage data that the embodiment of the present invention provides, can comprise:
Distribution module 62, the data that are used for needing to store are divided into R section, the individual described section of R are distributed in the described child resource pond of division.
Wherein, the data that needs are stored are expected the copy amount of generation less than N, R 〉=2.
Copy generation module 63 is used for being distributed to each described section ghost section in each child resource pond.
Storage module 64 is used for described copy section is stored in child resource pond under corresponding described section, and the described copy section described section identical with data is mutually corresponding.
Further, described storage module 64 also is used for described copy section is stored into the memory device in the child resource pond under corresponding described section.
And the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
Further alternative, the device of the storage data that the embodiment of the present invention provides can also comprise:
Divide module 61, be used for the child resource pond of described division is divided into the pond set of different stage child resource again.
Wherein, the pond set of each child resource comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different.
Described copy generation module 63 also is used for the copy number of sections that corresponding each the described section expectation of rank according to child resource pond set under the child resource pond generates, and will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
due in actual applications, the significance level of different pieces of information is also different, the device that the embodiment of the present invention provides can be taked to the data of different significance levels the storage mode of different brackets, for important comparatively speaking data can be by possessing more copy storage mode to guarantee the security of data, for unessential data comparatively speaking can be by possessing less copy storage mode with the utilization rate of memory device in the increase system, thereby improve the operational efficiency of distributed memory system, make management server more memory device can be used for storing the data of outbalance, thereby further improved the security of significant data.
Further alternative again, the device of the storage data that the embodiment of the present invention provides can also comprise:
Negative regulator module 65 is used for for any described child resource pond, reduces the quantity of the memory device in described child resource pond.
Positive adjustment module 66 is used for for any described child resource pond, increases the quantity of the memory device in described child resource pond.
Data allocations module 67, the described section that is used for the memory device that will reduce are assigned to other memory devices in described child resource pond.
Wherein, the remaining memory device quantity in the child resource pond is greater than the number of copies of described expectation generation.
Described data allocations module 67, when also being used for the quantity when the memory device that increases described child resource pond, the copy of the described section in this child resource pond is cut into slices to be re-assigned on memory device in this child resource pond.
Further optional, described distribution module 62 also is used for the significance level according to R section data, in R the child resource pond that is distributed to different stage of cutting into slices.
The embodiment of the present invention can also realize the hot plug/unplug of the memory device in the child resource pond; when making memory device in each child resource pond the abnormal running situation such as maintenance occur damaging, shut down; the data that can guarantee to be stored in the child resource pond are not lost; make distributed memory system in the situation that when operation can guarantee that data stabilization get rid of obsolete memory device at any time, thereby further improved the security of data.The device of the repair data that the embodiment of the present invention provides, all memory devices can be divided at least two sub-resource pools, and store the data in the child resource pond that marks off, because the data in each child resource pond are independently, only have 2 memory devices in same child resource pond to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device will cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of child resources ponds, and store the data in a plurality of child resources pond that marks off, only have when 2 memory devices in same child resource pond damage, data just can be damaged fully, the probability that damages due to 2 memory devices in same child resource pond, less than in prior art in the situation that only have resource pool to damage the probability of 2 memory devices, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Again on the one hand, the embodiment of the present invention provides a kind of storage system of storage data, as shown in Figure 7, described storage system comprises the storage area 74 for the storage data, described storage area 74 is at least 2 sub-resource pools by logical partitioning, each child resource pond comprises an adjacent N memory device, N 〉=2, described storage system comprises: processor 71, communication interface 72, bus 73, described processor 71, described communication interface 72 and described storage area 74 in all memory devices, complete mutual communication by described bus 73, wherein:
Described processor 71, the data that are used for needing to store are divided into R section, by described communication interface 72, the individual described section of R are distributed in the described child resource pond of division.
Wherein, the data that needs are stored are expected the copy amount of generation less than N, R 〉=2.
Described processor 71 also is used for being distributed to each described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section.
Wherein, the copy section is mutually corresponding with the identical section of the data that need storage.
Further, described processor 71 also is used for described copy section is stored into the memory device in the child resource pond under corresponding described section.
Wherein, the memory device of each described copy section storage is different from the memory device of corresponding section storage, and the copy section that data are identical is stored on different memory devices.
Optionally, described processor 71 also is used for the child resource pond of described division is divided into the pond set of different stage child resource again.
Wherein, the pond set of each child resource comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each section generates is different.
Described processor 71 also is used for the copy number of sections that corresponding each the described section expectation of rank according to child resource pond set under the child resource pond generates, and will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
Further alternative, described processor 71, also be used for behind the child resource pond that described copy section is stored under corresponding described section, for any described child resource pond, during the quantity of the memory device in reducing described child resource pond, by described communication interface 72, the described section in the memory device that reduces is assigned on other memory devices in described child resource pond.
Wherein, the remaining memory device quantity in the child resource pond is greater than the number of copies of described expectation generation.
Arranged side by side is optional, described processor 71, also be used for behind the child resource pond that described copy section is stored under corresponding described section, for any described child resource pond, during the quantity of the memory device in increasing described child resource pond, by described communication interface 72, the copy section of the described section in this child resource pond is re-assigned on memory device in this child resource pond.
Further optional, described processor 71 also is used for the significance level according to R section data, in R the child resource pond that is distributed to different stage of cutting into slices.
The storage system of the repair data that the embodiment of the present invention provides, all memory devices in storage system can be divided at least two resource pools, and store the data in the resource pool that marks off, because the data in each resource pool are independently, only have 2 memory devices in same resource pool to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device will cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of resource pools, and store the data in a plurality of resource pools that mark off, only have when 2 memory devices in same resource pool damage, data just can be damaged fully, probability due to 2 memory devices damages in same resource pool, less than in prior art in the situation that only have resource pool to damage the probability of 2 memory devices, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Again on the one hand, the embodiment of the present invention provides a kind of computer program of repair data, be used for storage system, described storage system comprises the resource pool for the storage data, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises an adjacent N memory device, N 〉=2, described computer program comprises the computer-readable recording medium of having stored program code, and the instruction that described program code comprises is used for:
The data of needs storages are divided into R section, R described section are distributed in the described child resource pond of division, the copy amount that the described data that need storage are expected generation is less than N, R 〉=2.
To being distributed to each the described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section, the described copy section described section identical with data is mutually corresponding.
The computer program of the repair data that the embodiment of the present invention provides, all memory devices in storage system can be divided at least two resource pools, and store the data in the resource pool that marks off, because the data in each resource pool are independently, only have 2 memory devices in same resource pool to damage, data just can be damaged fully.in prior art in the situation that only have a resource pool, damage 2 situations that memory device will cause data to damage fully, the embodiment of the present invention is owing to can marking off a plurality of resource pools, and store the data in a plurality of resource pools that mark off, only have when 2 memory devices in same resource pool damage, data just can be damaged fully, probability due to 2 memory devices damages in same resource pool, less than in prior art in the situation that only have resource pool to damage the probability of 2 memory devices, thereby make the embodiment of the present invention can reduce with respect to prior art the probability that data are damaged fully, thereby improved the security of data.
Each embodiment in this specification all adopts the mode of going forward one by one to describe, and between each embodiment, identical similar part is mutually referring to getting final product, and each embodiment stresses is difference with other embodiment.Especially, for apparatus embodiments, substantially similar in appearance to embodiment of the method, so describe fairly simplely, relevant part gets final product referring to the part explanation of embodiment of the method due to it.
One of ordinary skill in the art will appreciate that all or part of flow process that realizes in above-described embodiment method, to come the relevant hardware of instruction to complete by computer program, described program can be stored in computer read/write memory medium, this program can comprise the flow process as the embodiment of above-mentioned each side method when carrying out.Wherein, described storage medium can be magnetic disc, CD, read-only store-memory body (Read-Only Memory, ROM) or random store-memory body (Random Access Memory, RAM) etc.
The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited to this, anyly is familiar with those skilled in the art in the technical scope that the present invention discloses; the variation that can expect easily or replacement are within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of claim.

Claims (19)

1. the method for a storage data, be used for storage system, and described storage system comprises the resource pool for the storage data, it is characterized in that, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises an adjacent N memory device, N 〉=2, described method comprises:
The data of needs storages are divided into R section, R described section are distributed in the described child resource pond of division, the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
To being distributed to each the described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section, the described copy section described section identical with data is mutually corresponding.
2. the method for storage data according to claim 1, it is characterized in that, the described child resource pond that described copy is stored under corresponding described section comprises: described copy section is stored on memory device in child resource pond under corresponding described section, and the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
3. the method for storage data according to claim 1 and 2 is characterized in that:
Also comprise: the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different;
Described to being distributed to each the described section ghost section in each child resource pond, comprise: the copy number of sections that generates according to corresponding each the described section expectation of the rank of child resource pond set under the child resource pond will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
4. the method for storage data according to claim 1 and 2, is characterized in that, in the child resource pond under the section of described copy being stored in corresponding described section after, also comprise:
For any described child resource pond, during the quantity of the memory device in reducing described child resource pond, described section in the memory device that reduces is assigned on other memory devices in described child resource pond the number of copies that the remaining memory device quantity in described child resource pond generates greater than described expectation.
5. the method for storage data according to claim 1 and 2, is characterized in that, in the child resource pond under the section of described copy being stored in corresponding described section after, also comprise:
For any described child resource pond, during the quantity of the memory device in increasing described child resource pond, the copy section of the described section in this child resource pond is re-assigned on memory device in this child resource pond.
6. the method for storage data according to claim 3, is characterized in that, describedly R section is distributed in advance the child resource pond of dividing comprises:
According to the significance level of data in R section, in R the child resource pond that is distributed to different stage of cutting into slices.
7. the device of a storage data, be used for storage system, and described storage system comprises the resource pool for the storage data, it is characterized in that, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises an adjacent N memory device, N 〉=2, described method comprises:
Distribution module, the data that are used for needing to store are divided into R section, R described section are distributed in the described child resource pond of division, and the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
The copy generation module is used for being distributed to each described section ghost section in each child resource pond;
Storage module is used for described copy section is stored in child resource pond under corresponding described section, and the described copy section described section identical with data is mutually corresponding.
8. the device of storage data according to claim 7, it is characterized in that, also comprise: described storage module, also be used for described copy section is stored into the memory device in the child resource pond under corresponding described section, and the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
9. the device of according to claim 7 or 8 described storage datas, is characterized in that, also comprises:
Divide module, be used for the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different;
Described copy generation module also is used for the copy number of sections that corresponding each the described section expectation of rank according to child resource pond set under the child resource pond generates, and will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
10. the device of according to claim 7 or 8 described storage datas, is characterized in that, also comprises:
The negative regulator module is used for for any described child resource pond, reduces the quantity of the memory device in described child resource pond;
Data allocations module, the described section that is used for the memory device that will reduce are assigned to other memory devices in described child resource pond, the number of copies that the remaining memory device quantity in described child resource pond generates greater than described expectation.
11. the device of storage data according to claim 10 is characterized in that, also comprises:
Positive adjustment module is used for for any described child resource pond, increases the quantity of the memory device in described child resource pond;
Described data allocations module, when also being used for the quantity when the memory device that increases described child resource pond, the copy of the described section in this child resource pond is cut into slices to be re-assigned on memory device in this child resource pond.
12. the device of storage data according to claim 9 is characterized in that, also comprises:
Described distribution module also is used for the significance level according to R section data, in R the child resource pond that is distributed to different stage of cutting into slices.
13. the storage system of a storage data, described storage system comprises the resource pool for the storage data, it is characterized in that, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises an adjacent N memory device, N 〉=2, described storage system comprises: processor, communication interface, bus characterized by further comprising memory device, described processor, described communication interface and all memory devices, complete mutual communication by described bus, wherein:
Described processor, the data that are used for needing to store are divided into R section, by described communication interface, R described section are distributed in the described child resource pond of division, and the copy amount that the described data that need storage are expected generation is less than N, R 〉=2;
Described processor also is used for being distributed to each described section ghost section in each child resource pond, and described copy section is stored in child resource pond under corresponding described section, and the described copy section described section identical with data is mutually corresponding.
14. the storage system of storage data according to claim 13 is characterized in that, comprising:
Described processor, also be used for described copy section is stored into the memory device in the child resource pond under corresponding described section, and the memory device of each described copy section storage is different from the memory device of corresponding described section storage, and the copy section that data are identical is stored on different memory devices.
15. the storage system of according to claim 13 or 14 described storage datas, it is characterized in that, described processor, also be used for the child resource pond of described division is divided into the pond set of different stage child resource again, the set of each child resource pond comprises at least one child resource pond, and the copy number of sections that in the child resource pond set of different stage, the expectation of each described section generates is different;
Described processor also is used for the copy number of sections that corresponding each the described section expectation of rank according to child resource pond set under the child resource pond generates, and will be distributed to each described section in the child resource pond and generate the copy section of respective amount.
16. the storage system of according to claim 13 or 14 described storage datas, it is characterized in that, described processor, also be used for behind the child resource pond that described copy section is stored under corresponding described section, for any described child resource pond, during the quantity of the memory device in reducing described child resource pond, by described communication interface, the described section in the memory device that reduces is assigned on other memory devices in described child resource pond the number of copies that the remaining memory device quantity in described child resource pond generates greater than described expectation.
17. the storage system of according to claim 13 or 14 described storage datas, it is characterized in that, described processor, also be used for behind the child resource pond that described copy section is stored under corresponding described section, for any described child resource pond, during the quantity of the memory device in increasing described child resource pond, by described communication interface, the copy section of the described section in this child resource pond is re-assigned on memory device in this child resource pond.
18. the storage system of storage data according to claim 15 is characterized in that, described processor also is used for the significance level according to R section data, in R the child resource pond that is distributed to different stage of cutting into slices.
19. the computer program of a repair data, be used for storage system, described storage system comprises the resource pool for the storage data, it is characterized in that, described resource pool is at least 2 sub-resource pools by logical partitioning, and each child resource pond comprises N adjacent memory device, N 〉=2, described computer program comprises the computer-readable recording medium of having stored program code, and the instruction that described program code comprises is used for carrying out described method as arbitrary in claim 1-6.
CN201280003656.8A 2012-12-28 2012-12-28 The method of storage data and device Active CN103384550B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2012/087922 WO2014101144A1 (en) 2012-12-28 2012-12-28 Data storage method and device

Publications (2)

Publication Number Publication Date
CN103384550A true CN103384550A (en) 2013-11-06
CN103384550B CN103384550B (en) 2016-05-25

Family

ID=49492132

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201280003656.8A Active CN103384550B (en) 2012-12-28 2012-12-28 The method of storage data and device

Country Status (2)

Country Link
CN (1) CN103384550B (en)
WO (1) WO2014101144A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106416405A (en) * 2014-06-27 2017-02-15 夏普株式会社 Resource pool access for device to device communications
CN106792692A (en) * 2016-12-27 2017-05-31 兴唐通信科技有限公司 A kind of physics dicing method based on SDN technologies
WO2017113280A1 (en) * 2015-12-31 2017-07-06 华为技术有限公司 Distributed storage system and metadata managing method
CN107925587A (en) * 2015-08-21 2018-04-17 华为技术有限公司 Method and apparatus for network section
WO2018176998A1 (en) * 2017-03-29 2018-10-04 华为技术有限公司 Data storage method and device
CN110275669A (en) * 2018-03-15 2019-09-24 杭州海康威视数字技术股份有限公司 Date storage method, device and electronic equipment
CN110532271A (en) * 2019-08-30 2019-12-03 北京浪潮数据技术有限公司 Distributed storage method, control system and the device realized based on DHT algorithm
CN112181913A (en) * 2020-10-10 2021-01-05 上海威固信息技术股份有限公司 Computer storage system and storage method based on block chain
US11032858B2 (en) 2017-03-28 2021-06-08 Huawei Technologies Co., Ltd. Network access method, terminal, access network, and core network

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998038568A1 (en) * 1997-02-27 1998-09-03 International Business Machines Corporation Transformational raid for hierarchical storage management system
CN101459597A (en) * 2007-12-14 2009-06-17 华为技术有限公司 Method and system for using logic resource
US20100040058A1 (en) * 2008-08-15 2010-02-18 Vmware, Inc. Systems and Methods of Configuring a Resource Pool as a Network End Point
CN102073560A (en) * 2011-01-17 2011-05-25 北京深思洛克软件技术股份有限公司 Data backup method and device
CN102123167A (en) * 2010-12-31 2011-07-13 成都市华为赛门铁克科技有限公司 Distributed file system, and data storage processing method and data storage processing device thereof
CN102843403A (en) * 2011-06-23 2012-12-26 盛大计算机(上海)有限公司 File processing method based on distributed file system, system, and client

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998038568A1 (en) * 1997-02-27 1998-09-03 International Business Machines Corporation Transformational raid for hierarchical storage management system
CN101459597A (en) * 2007-12-14 2009-06-17 华为技术有限公司 Method and system for using logic resource
US20100040058A1 (en) * 2008-08-15 2010-02-18 Vmware, Inc. Systems and Methods of Configuring a Resource Pool as a Network End Point
CN102123167A (en) * 2010-12-31 2011-07-13 成都市华为赛门铁克科技有限公司 Distributed file system, and data storage processing method and data storage processing device thereof
CN102073560A (en) * 2011-01-17 2011-05-25 北京深思洛克软件技术股份有限公司 Data backup method and device
CN102843403A (en) * 2011-06-23 2012-12-26 盛大计算机(上海)有限公司 File processing method based on distributed file system, system, and client

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106416405A (en) * 2014-06-27 2017-02-15 夏普株式会社 Resource pool access for device to device communications
CN107925587A (en) * 2015-08-21 2018-04-17 华为技术有限公司 Method and apparatus for network section
US10644955B2 (en) 2015-08-21 2020-05-05 Huawei Technologies Co., Ltd. Method and apparatus for network slicing
CN107925587B (en) * 2015-08-21 2020-04-03 华为技术有限公司 Method and apparatus for network slicing
WO2017113280A1 (en) * 2015-12-31 2017-07-06 华为技术有限公司 Distributed storage system and metadata managing method
CN106792692A (en) * 2016-12-27 2017-05-31 兴唐通信科技有限公司 A kind of physics dicing method based on SDN technologies
CN106792692B (en) * 2016-12-27 2019-11-05 兴唐通信科技有限公司 A kind of physics dicing method based on SDN technology
US11032858B2 (en) 2017-03-28 2021-06-08 Huawei Technologies Co., Ltd. Network access method, terminal, access network, and core network
US20200028911A1 (en) * 2017-03-29 2020-01-23 Huawei Technologies Co., Ltd. Data Storage Method and Apparatus
CN108667867A (en) * 2017-03-29 2018-10-16 华为技术有限公司 Date storage method and device
WO2018176998A1 (en) * 2017-03-29 2018-10-04 华为技术有限公司 Data storage method and device
US11575748B2 (en) 2017-03-29 2023-02-07 Huawei Technologies Co., Ltd. Data storage method and apparatus for combining different data distribution policies
US10972542B2 (en) 2017-03-29 2021-04-06 Huawei Technologies Co., Ltd. Data storage method and apparatus
CN110275669A (en) * 2018-03-15 2019-09-24 杭州海康威视数字技术股份有限公司 Date storage method, device and electronic equipment
CN110275669B (en) * 2018-03-15 2022-11-25 杭州海康威视数字技术股份有限公司 Data storage method and device and electronic equipment
CN110532271A (en) * 2019-08-30 2019-12-03 北京浪潮数据技术有限公司 Distributed storage method, control system and the device realized based on DHT algorithm
CN112181913B (en) * 2020-10-10 2021-05-07 上海威固信息技术股份有限公司 Block chain-based computer storage method and system
CN112181913A (en) * 2020-10-10 2021-01-05 上海威固信息技术股份有限公司 Computer storage system and storage method based on block chain

Also Published As

Publication number Publication date
CN103384550B (en) 2016-05-25
WO2014101144A1 (en) 2014-07-03

Similar Documents

Publication Publication Date Title
CN103384550A (en) Data storage method and device
CN102387179B (en) Distributed file system and node, storage method and storage controlling method
US10353787B2 (en) Data stripping, allocation and reconstruction
CN101131675B (en) Apparatus and method for preventing write starvation in a partitioned cache of a storage controller
CN108052655B (en) Data writing and reading method
CN103051691B (en) Partition allocation method, device and distributed memory system
US20120317124A1 (en) Database partitioning by virtual partitions
CN104503708B (en) The method and device of data hash storage
CN102004769B (en) File management method, equipment and memory system
CN104094254A (en) System and method for unbalanced raid management
CN103064765A (en) Method and device for data recovery and cluster storage system
US9436394B2 (en) RAID random distribution scheme
US11144396B1 (en) Raid reliability with a provisional spare disk
US8020032B2 (en) Method for providing deferred maintenance on storage subsystems
CN105404565B (en) A kind of dual-active data guard method and device
DE112020003290T5 (en) DATA PLACEMENT IN A WRITE CACHE ARCHITECTURE SUPPORTING READ HEAT DATA BREAKDOWN
CN106201354A (en) Date storage method and system
CN102226892A (en) Disk fault tolerance processing method and device thereof
CN109725823A (en) Method and apparatus for managing mixing storage disk array
CN108121497B (en) Storage method and storage system
CN102200935A (en) Method for storing data in stream media system
CN108733307B (en) Storage management method, apparatus and computer readable medium
CN102629223A (en) Method and device for data recovery
CN104391760A (en) Data backup method, server and system
US11860746B2 (en) Resilient data storage system with efficient space management

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant