CN105117303B

CN105117303B - A kind of method and apparatus that data are restored

Info

Publication number: CN105117303B
Application number: CN201510489175.8A
Authority: CN
Inventors: 张巍; 张烽; 雷晓松; 项锐
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2012-11-23
Filing date: 2012-11-23
Publication date: 2018-11-30
Anticipated expiration: 2032-11-23
Also published as: CN105117303A

Abstract

The embodiment of the invention provides a kind of modes for restoring data, and currently processed data acquisition system is divided into n data block, and the encoding block of m redundancy of data acquisition system is calculated, the method also includes：When there are when bad block, analyze the damage data of bad block in n data block the location of in bad block in the n data block；According to the position analyzed, each of n data block and m encoding block piece are divided into h fritter according to identical division rule, in all fritters that n data block and m encoding block include, it will be closed as a subset the identical fritter in present position in each data block and encoding block；For one, there are the subclass of bad fritter, restore in the subclass impaired data in bad fritter using the data of the unspoiled fritter in the belonging subclass there are bad fritter respectively.Correspondingly, the embodiment of the invention also provides the equipment for restoring data.The reliability of data storage can be improved in the embodiment of the present invention.

Description

A kind of method and apparatus that data are restored

Technical field

The present invention relates to the method and apparatus that data processing field more particularly to a kind of data are restored.

Background technique

With the development of network technology, data high-speed to be treated increases, and provides for mass data and has high reliability And the storage of high scalability is a huge challenge for modern enterprise.For the reliabilty and availability for guaranteeing data, lead to Frequently with the method for storing multiple data copies, when some copy since the failure of hard disk, server, network or software causes When copy is unusable, service is provided by other copies.A problem is thus brought, storage space utilization is extremely low.With 3 The data instance of copy, storage space utilization only have 30% or so.

Presently mainly a kind of erasure codes (Erasure Code, EC) technology solves the problems, such as that storage space utilization is low, Data acquisition system is divided into n data block by the technology, and each data block stores a part of data of the data acquisition system, then passes through Pure XOR operation and algebra domain operation calculate the encoding block of m redundancy of the data acquisition system, then by this n data block and m A encoding block is stored in different places respectively.The bad block of the corrupted data present in the above-mentioned n data block and m encoding block When, relevant operation is carried out using unspoiled data block or encoding block and restores bad block, wherein n and m is the integer greater than 0.

But in above-mentioned technology, when the quantity of bad block is greater than above-mentioned m, it can not just restore bad block, so as to cause data storage Reliability it is low.

Summary of the invention

The embodiment of the invention provides a kind of method and apparatus for restoring data, and the reliability of data storage can be improved.

First aspect present invention provides a kind of method for restoring data, and currently processed data acquisition system is divided into n data Block, and the encoding block of m redundancy of the data acquisition system is calculated, the n and m are the integer greater than 0, and the method may be used also To include：

When there are when bad block, analyze the damage data of each bad block in the n data block in institute in the n data block State location in each bad block；

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical Division rule be divided into h fritter, will be every in all fritters that the n data block and the m encoding block include The identical fritter in present position is closed as a subset in a data block and encoding block, and the presence that each subclass includes damages number According to the quantity of bad fritter be less than or equal to the m, the h is integer greater than 1；

For one, there are the subclass of bad fritter, respectively using not damaging in the belonging subclass there are bad fritter Impaired data in bad fritter in the data recovery subclass of bad fritter.

With reference to first aspect, in the first possible implementation, respectively using the belonging son there are bad fritter Impaired data in bad fritter in the data recovery subclass of unspoiled fritter in set, including：

It obtains there are in the subclass of bad fritter, the data of fritter and belonging institute in the belonging m encoding block State the data of unspoiled fritter in n data block；

Use unspoiled fritter in the data and the n data block of fritter in the m encoding block of the acquisition Data restore impaired data in the bad fritter in the subclass.

It is with reference to first aspect, in the second possible implementation, described to be directed to one there are the subclass of bad fritter, Restore bad small in the subclass using the data of the unspoiled fritter in the belonging subclass there are bad fritter respectively Impaired data in block, including：

There are the quantity of bad fritter in the subclass of bad fritter for analysis；

It obtains there are in the subclass of bad fritter, quantity phase in the belonging m encoding block and with bad fritter The data of same fritter and the data of the unspoiled fritter in the belonging n data block；

With reference to first aspect or the first possible implementation or second of possible implementation, the third may Implementation in, the position analyzed according to, by each of the n data block and described m encoding block Block is divided into h fritter according to identical division rule：

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical Division rule h fritter is divided into pre-set size, the pre-set size is n data block storage The size of the common multiple of the minimum unit of the minimum unit of data and the m encoding block storing data.

With reference to first aspect or the first possible implementation or second of possible implementation, in the 4th kind of possibility Implementation in, the position analyzed according to, by each of the n data block and described m encoding block Block is divided into h fritter according to identical division rule：

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical Division rule h fritter is divided into pre-set size, the pre-set size as unit of a sector and Size including at least one sector.

Second aspect of the present invention provides a kind of equipment for restoring data, and currently processed data acquisition system is divided into n data Block, and the encoding block of m redundancy of the data acquisition system is calculated, the n and m are the integer greater than 0, the equipment packet It includes：First analytical unit, division unit and recovery unit, wherein：

First analytical unit, for when there are when bad block, analyze in the n data block in the n data block The damage data of each bad block are the location of in each bad block；

The division unit, the position for being analyzed according to the analytical unit, by the n data block and the m Each of a encoding block piece is divided into h fritter according to identical division rule, in the n data block and the m volume In all fritters that code block includes, will in each data block and encoding block the identical fritter in present position as a subset It closes, the quantity of the bad fritter for the presence damage data that each subclass includes is less than or equal to the m, and the h is whole greater than 1 Number；

The recovery unit, for being directed to one, there are the subclass of bad fritter, are existed respectively using belonging this bad small Impaired data in bad fritter in the data recovery subclass of unspoiled fritter in the subclass of block.

In conjunction with second aspect, in the first possible implementation, the recovery unit includes：

First acquisition unit, for obtaining in the subclass there are bad fritter, fritter in the belonging m encoding block The data of unspoiled fritter in data and the belonging n data block；

First restores subelement, the data of fritter and the n number in the m encoding block for using the acquisition Restore impaired data in the bad fritter in the subclass according to the data of fritter unspoiled in block.

In conjunction with second aspect, in the second possible implementation, the recovery unit includes：

Second analytical unit, for analyzing the quantity of bad fritter in the subclass there are bad fritter；

Second acquisition unit, for obtaining in the subclass there are bad fritter, it is in the belonging m encoding block and The data of the data of fritter identical with the quantity of bad fritter and the unspoiled fritter in the belonging n data block；

Second restores subelement, the data of fritter and the n number in the m encoding block for using the acquisition Restore impaired data in the bad fritter in the subclass according to the data of fritter unspoiled in block.

It is possible at the third in conjunction with second aspect or the first possible implementation or second of possible implementation Implementation in, the position that the division unit is also used to analyze according to, by the n data block and the m Each of encoding block piece is divided into h fritter according to identical division rule with pre-set size, described to preset Size be the n data block storing data minimum unit and the m encoding block storing data minimum unit public affairs The size of multiple.

In conjunction with second aspect or the first possible implementation or second of possible implementation, in the 4th kind of possibility Implementation in, the position that the division unit is also used to analyze according to, by the n data block and the m Each of encoding block piece is divided into h fritter according to identical division rule with pre-set size, described to preset Size as unit of a sector and the size including at least one sector.

Third aspect present invention provides a kind of equipment for restoring data, and currently processed data acquisition system is divided into n data Block, and the encoding block of m redundancy of the data acquisition system is calculated, the n and m are the integer greater than 0, which is characterized in that The equipment includes processor, communication interface, memory and bus：

Wherein processor, communication interface, memory complete mutual communication by bus；

The communication interface, the storage for the data from the data and the m encoding block for storing the n data block Equipment receives the data of the n data block and the data of the m encoding block；

Batch processing code is stored in the memory, and the processor is for calling the journey stored in the memory Sequence code, for performing the following operations：

In conjunction with the third aspect, in the first possible implementation, what the processor executed uses belonging respectively The data of unspoiled fritter in the subclass there are bad fritter restore in the subclass impaired number in bad fritter According to operation include：

In conjunction with the third aspect, in the second possible implementation, what the processor executed uses belonging respectively The data of unspoiled fritter in the subclass there are bad fritter restore in the subclass impaired number in bad fritter According to operation include：

It is possible at the third in conjunction with the third aspect or the first possible implementation or second of possible implementation Implementation in, the n data block and described m are compiled in the position analyzed according to that the processor executes Each of code block piece includes according to the operation that identical division rule is divided into h fritter：

In conjunction with the third aspect or the first possible implementation or second of possible implementation, in the 4th kind of possibility Implementation in, the n data block and described m are compiled in the position analyzed according to that the processor executes Each of code block piece includes according to the operation that identical division rule is divided into h fritter：

Fourth aspect present invention provides computer storage medium described in a kind of computer storage medium and is stored with program, described Program includes the following steps when executing：The step as described in method for providing a kind of recovery data such as first aspect present invention.

In above-mentioned technical proposal, since according to the location of damage data, the n data block and described m being compiled Each of code block piece is divided into h fritter according to identical division rule, by the n data block and the m encoding block The identical fritter in present position is closed as a subset in each data block and encoding block in all fritters for including, so that The quantity of the bad fritter for the presence damage data that each subclass includes is less than or equal to the m, and the h is the integer greater than 1； The data institute that the data of damage can be thus divided in less than or equal to h subclass, and be damaged in each subclass The quantity of fritter be less than or equal to the m, then restore every using the unspoiled fritter in each subclass respectively Impaired data in bad fritter in a subclass can at most restore m × h bad fritters, compared to present in this way The reliability of data storage can be improved in the block that can only restore m damage in technology.

Detailed description of the invention

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art To obtain other drawings based on these drawings.

Fig. 1 is a kind of optional application scenario diagram provided in an embodiment of the present invention；

Fig. 2 is a kind of flow diagram of method for restoring data provided in an embodiment of the present invention；

Fig. 3 is a kind of optional data structure schematic diagram provided in an embodiment of the present invention；

Fig. 4 is the flow diagram of another method for restoring data provided in an embodiment of the present invention；

Fig. 5 is the flow diagram of another method for restoring data provided in an embodiment of the present invention；

Fig. 6 is the structural schematic diagram for the equipment that a kind of data provided in an embodiment of the present invention are restored；

Fig. 7 is the structural schematic diagram for the equipment that another data provided in an embodiment of the present invention are restored；

Fig. 8 is the structural schematic diagram for the equipment that a kind of data provided in an embodiment of the present invention are restored；

Fig. 9 is the structural schematic diagram for the equipment that another data provided in an embodiment of the present invention are restored；

Figure 10 is the structural schematic diagram for the equipment that a kind of data provided in an embodiment of the present invention are restored；

Figure 11 is the structural schematic diagram for the equipment that another data provided in an embodiment of the present invention are restored.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.

Fig. 1 is a kind of optional application scenario diagram provided in an embodiment of the present invention, wherein primary storage node is receiving number When according to stream A, data flow A is divided into n data block storage to secondary memory node, secondary memory node 1 shown in FIG. 1 is deposited to secondary Storage node 9 is for storing n data block and m encoding block, and wherein n, which can be equal to 6, m, can be equal to 9, wherein above-mentioned n data Block forms a data acquisition system, and m redundancy of the data acquisition system can be calculated by pure XOR operation or algebra domain operation Encoding block.When the business of the currently performed program of primary storage node shown in FIG. 1 or processing is needed using above-mentioned data acquisition system When, it is necessary to above-mentioned n data block is got to the secondary memory node for storing above-mentioned n data block, that is, gets above-mentioned data Set, to complete currently performed program or currently processed business.Above-mentioned primary storage node can be any control in network Device or storage equipment.

Fig. 2 is a kind of flow diagram of method for restoring data provided in an embodiment of the present invention, currently processed data Set is divided into n data block, and the encoding block of m redundancy of the data acquisition system is calculated, and the n and m are greater than 0 Integer, as shown in Fig. 2, this method further includes：

101, when there are the damage data for when bad block, analyzing each bad block in the n data block in the n data block The location of in each bad block.

Optionally, above-mentioned currently processed data acquisition system, which can be, realizes that equipment of the invention currently needs data to be used Set.For example, the currently performed program of equipment of the invention or the current business of processing need data acquisition system to be used.

102, the position analyzed according to, by each of the n data block and described m encoding block piece according to Identical division rule is divided into h fritter, will in all fritters that the n data block and the m encoding block include The identical fritter in present position is closed as a subset in each data block and encoding block, the presence damage that each subclass includes The quantity of the bad fritter of bad data is less than or equal to the m, and the h is the integer greater than 1.

It should be noted that above-mentioned position includes the initial position and final position of fritter, i.e., in data block and encoding block The identical fritter in middle present position, the size of these fritters is also identical.

Optionally, realize equipment of the invention (such as：Primary storage node device shown in FIG. 1) know above-mentioned n data The storage address information of each of block and m encoding block piece such as obtains in advance and stores above-mentioned n data block and m encoding block Each of piece the storage address information that reports of memory node, or executed by last time and obtained during program or processing business When taking above-mentioned n data block and m encoding block, the storage of each of the above-mentioned n data block of acquisition and m encoding block piece Location information etc..Division in step 102 can be the storage address of each of above-mentioned n data block and m encoding block piece It is divided into h fritter.

Wherein, above-mentioned division rule may include：For each data block and encoding block, the size of the fritter marked off, Each fritter is equal part or of different sizes, and the determination isotactic of each fritter initial position is then.

Optionally, above-mentioned division rule can be the position gone out based on above-mentioned analysis and generate, for example, above-mentioned analysis obtains Position of the damage data of bad block in the bad block is top half, division rule above-mentioned in this way in n data block It will be according to two-part division rule above and below being divided into each piece；For another example, above-mentioned analysis obtains bad block in n data block Damaging position of the data in the bad block is left-hand component, and division rule above-mentioned in this way can will be according to will be in each piece stroke It is divided into the two-part division rule in left and right.

Optionally, step 102 is divided into h fritter for each piece, is such as divided into each piece with middle position 2 small Block, then each piece of 2 fritters for including just are top half and lower half portion in the location of data block or encoding block；And In all fritters that the n data block and the m encoding block are included by step 102 in each data block and encoding block institute Locate the identical fritter in position to close as a subset, also just obtains h subclass, be such as divided into 2 for each piece with middle position A fritter just obtains 2 subclass, is a subset in the fritter that the location of data block or encoding block are top half It closes, is another subclass in the fritter that the location of data block or encoding block are lower half portion.To step 102 N data block and the m encoding block are divided into h subclass, each subclass includes n+m fritter, and each subset Fritter in conjunction is identical in the present position of data block or encoding block.

103, for one, there are the subclass of bad fritter, respectively using in the belonging subclass there are bad fritter Impaired data in bad fritter in the data recovery subclass of unspoiled fritter.

Optionally, the encoding block of the above-mentioned m redundancy that the data are calculated can be mentioned using background technique The encoding block of m redundancy of the data is calculated in EC technology by pure XOR operation and algebra domain operation.

Optionally, n data block and the m encoding block are as shown in Figure 3, wherein n 6, m 3, Fig. 3 is labeled with black Frame indicates the location of the data of damage, it is seen that the quantity of bad block is greater than 3, then using being that impossible restore in the prior art The data of damage.The present invention can analyze the location of data of damage by step 101, and step 102 is according to step The position that rapid 101 analysis obtains, can be divided into two fritters for each of this 6 data blocks and 3 encoding blocks piece, specifically As shown in phantom in Figure 3, two fritters up and down are divided by each piece, step 102 compiles the n data block and described m The identical fritter in present position is closed as a subset in each data block and encoding block in all fritters that code block includes, this Two subclass are just obtained, a subset is closed comprising the fritter above n+m, another subclass includes small below n+m Block, the quantity that can be obtained by the bad fritter that the subclass of fritter above includes in this way is 2, and the subclass of fritter includes below The quantity of bad fritter be 2, can will be damaged in bad fritter that the subclass of fritter above includes respectively finally by step 103 Impaired data are restored in bad data recovery and the bad fritter for including by the subclass of following fritter.Just by above-mentioned technology The damage data that can restore the block of 4 damages, and cannot restore in the prior art, so that improves data storage can By property.

Certainly above-mentioned h is can to change with the case where corrupted data in the present invention, for example, the data of the damage in Fig. 3 When more, step 102 can be by 3,4 or 5 fritters of each piece of division in Fig. 3, for a data block or an encoding block For, the size of the fritter divided can be identical or not identical, and user can be big according to shared by damage data It is small actually to determine；It can restore more to damage data to realize.Certainly, step 102 is divided into h fritter for each piece When, each fritter can be to be divided with pre-set size, i.e., the position analyzed according in step 102 It sets, each of the n data block and described m encoding block piece are divided into h fritter according to identical division rule can To include：

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical Division rule h fritter is divided into pre-set size.

Optionally, the pre-set size can for the n data block storing data minimum unit and the m The size of the common multiple of the minimum unit of a encoding block storing data.Certainly, in the n data block and the m encoding block Each of piece h fritter is divided into the size of the common multiple, it is necessary to be meet it is above-mentioned by the n data block and the m In all fritters that a encoding block includes in each data block and encoding block the identical fritter in present position as a subset It closes, the quantity of the bad fritter for the presence damage data that each subclass includes is less than or equal to the condition of the m.Such as above-mentioned common multiple Several sizes is with the minimum unit of the minimum unit of the n data block storing data and the m encoding block storing data Least common multiple is unit, and the size including least common multiple described at least one.Wherein, above-mentioned minimum unit, which can be, deposits Store up a data (such as：One binary number 0 or 1), and guarantee that required storage can be used normally in the function of the data Space.In the present invention data recovery procedure can be using algebraic operation (such as：Pure XOR operation), then each data All have the function of completely to encode or verify etc., as long as the present invention does not dismantle a data.Such as data block stores number According to minimum unit be 4 bytes, the minimum unit of encoding block storing data is 3 bytes, then above-mentioned least common multiple is just For 12 bytes, such step 102 is just as unit of 12 bytes, the size of divided fritter, then the fritter finally divided Size can be 24 byte-sizeds, be also possible to 12 byte-sizeds to realize the unspoiled data of each fritter storage It is complete, i.e., the data that step 102 will not damage data block in partition process or encoding block is stored, to guarantee encoding block Storage coded data (such as：Check code) coding or the functions such as verification will not fail.

Optionally, the pre-set size is as unit of a sector and the size including at least one sector, That is the size of a fritter is N number of sector, N is more than or equal to 1 positive integer, for example, the size of a fritter includes two Or the size of three sectors.And any one data stored in the n data block and the m encoding block will not all split and deposit Storage is in different sectors, i.e., the data that step 102 will not damage data block in partition process or encoding block is stored, to protect Demonstrate,prove encoding block storage coded data (such as：Check code) coding or the functions such as verification will not fail.

In above-mentioned technical proposal, since according to the location of damage data, the n data block and described m being compiled Each of code block piece is divided into h fritter according to identical division rule, in the n data block and the m encoding block In all fritters for including, it will be closed as a subset the identical fritter in present position in each data block and encoding block, with So that the quantity of the bad fritter for the presence damage data that each subclass includes is less than or equal to the m, the h is greater than 1 Integer；The data of damage can be thus divided in less than or equal to h subclass, and the number damaged in each subclass Quantity according to the fritter at place is less than or equal to the m, then extensive using the unspoiled fritter in each subclass respectively Impaired data in bad fritter in multiple each subclass can at most restore m × h bad fritters in this way, and h is The reliability of data storage can be improved compared to the existing block that can only restore m damage in the art in integer greater than 1.

Fig. 4 is a kind of flow diagram of method for restoring data provided in an embodiment of the present invention, currently processed data Set is divided into n data block, and the encoding block of m redundancy of the data acquisition system is calculated, and the n and m are greater than 0 Integer, as shown in figure 4, this method further includes：

201, when there are the damage data for when bad block, analyzing each bad block in the n data block in the n data block The location of in each bad block.

The almost impossible data damaged in practical applications are entire blocks, as the data of the data block of 1M, damage may Some sector is concentrated on, as shown in figure 3, the data of damage only account for the very small part of entire data block, can thus be led to It crosses step 201 and analyzes the location of data of damage.

202, the position analyzed according to, by each of the n data block and described m encoding block piece according to Identical division rule is divided into h fritter, will in all fritters that the n data block and the m encoding block include The identical fritter in present position is closed as a subset in each data block and encoding block, the presence damage that each subclass includes The quantity of the bad fritter of bad data is less than or equal to the m, and the h is the integer greater than 1.

Optionally, as shown in figure 3, the data of damage have at 4, it is located at 4 blocks, wherein it is located at the top half of block at 2, At another 2 be located at block lower half portion, such step 202 can one by each piece in Fig. 3 be divided into up and down two fritters.

203, it obtains there are in the subclass of bad fritter, data of fritter and affiliated in the belonging m encoding block The data of unspoiled fritter in the n data block；

Optionally, above-mentioned steps 203, which can be, realizes equipment (such as primary storage node shown in FIG. 1) of the invention, from Store the number that fritter in the above-mentioned belonging m encoding block is got in above-mentioned n data block and the equipment of m encoding block According to and the belonging n data block in unspoiled fritter data.

204, for one, there are the subclass of bad fritter, use the number of fritter in the m encoding block of the acquisition Restore impaired data in the bad fritter in the subclass according to the data with unspoiled fritter in the n data block.

Optionally, as shown in figure 3, each fritter is divided into two fritters up and down, two subclass, such step are formed 203 can be the network transmission that data block or encoding block are carried out as unit of fritter each in subclass, on only having in Fig. 3 When the fritter in face has the data of damage, the fritter transmitted in step 203 in this way is just in subclass belonging to bad block Fritter；Network transmission is carried out as unit of entire data block or encoding block compared to the prior art, it is possible to reduce network transmission, with Save Internet resources.

Optionally, step 203 may include：

It is sent for the location information for not damaging fritter in the subclass of bad fritter there are the subclass of bad fritter, will be present To storage it is described do not damage small block data storage equipment (such as：Secondary memory node shown in FIG. 1), it is sent out to the storage equipment Request is sent to return to the data that unspoiled fritter is included in the subclass there are bad fritter；

Receive the data that the storage equipment returns；

The storage equipment of equipment of the invention (such as primary storage node shown in FIG. 1) and other storing datas is achieved (such as：Secondary memory node shown in FIG. 1) between the fritter that can only transmit in subclass where bad fritter included Data information, to reduce network transmission.

As an alternative embodiment, step 204 may include：

Use unspoiled fritter in the data and the n data block of fritter in the m encoding block of the acquisition Data impaired data in the bad fritter in the subclass are restored using EC algorithm.

Optionally, restore bad fritter in each subclass for example, by using pure XOR operation and algebra domain operation.

As an alternative embodiment, when discovery is there are in the subclass of bad fritter, institute belonging to a certain subclass When stating the fritter in m encoding block in the presence of the bad block for damaging data, and there are m volumes in the subclass of bad fritter in the subclass The quantity of bad fritter in code block plus the bad fritter in n data block quantity greater than m when, so that it may again the subclass into The above-mentioned division of row, can be divided into multiple fritters for each fritter of the subclass of the bad fritter, with meet it is above-mentioned include deposit It is less than or equal to the m in the quantity of the bad fritter of damage data.Restored again using the technical characteristic of step 204.Such as： There are include subclass 1, subclass 2 and subclass 3 in the subclass of bad fritter, wherein the belonging m in subclass 3 When fritter in encoding block has the bad block of damage data, and there is the bad block of damage data in the fritter in the m encoding block Quantity be h, the quantity of the bad fritter in subclass 3 in affiliated n data block is t, when h+t is greater than m, so that it may again right Subclass 3 carries out above-mentioned division, and each fritter of the subclass of the bad fritter is divided into multiple fritters, above-mentioned includes to meet Presence damage data bad fritter quantity be less than or equal to the m.Restored again using the technical characteristic of step 204.

As an alternative embodiment, present invention could apply to RS algorithm, (Reed Solomon, the algorithm belong to Alien word has no the accurately Chinese meaning, is common algorithm in algorithm field, and unified RS algorithm, improved RS (Cauchy Reed Solomon, CRS) algorithm, disk array (Redundant Arrays of Inexpensive Disks, RAID) 5 technologies, in RAID6 technology.

In upper technical solution, on the basis of above example, realizes and carry out network transmission as unit of fritter, in this way Internet resources can be saved.At the same time it can also improve the reliability of data storage.

Fig. 5 is a kind of flow diagram of method for restoring data provided in an embodiment of the present invention, currently processed data Set is divided into n data block, and the encoding block of m redundancy of the data acquisition system is calculated, and the n and m are greater than 0 Integer, as shown in figure 5, this method further includes：

301, when there are the damage data for when bad block, analyzing each bad block in the n data block in the n data block The location of in each bad block.

302, the position analyzed according to, by each of the n data block and described m encoding block piece according to Identical division rule is divided into h fritter, will in all fritters that the n data block and the m encoding block include The identical fritter in present position is closed as a subset in each data block and encoding block, the presence damage that each subclass includes The quantity of the bad fritter of bad data is less than or equal to the m, and the h is the integer greater than 1.

303, there are the quantity of bad fritter in the subclass of bad fritter for analysis；

304, it obtains there are in the subclass of bad fritter, number in the belonging m encoding block and with bad fritter Measure the data of identical fritter and the data of the unspoiled fritter in the belonging n data block；

305, for one, there are the subclass of bad fritter, use the number of fritter in the m encoding block of the acquisition Restore impaired data in the bad fritter in the subclass according to the data with unspoiled fritter in the n data block.

It may be implemented only to obtain the identical fritter of in the belonging m encoding block and with bad fritter quantity in this way Data, so as to save network overhead.

In upper technical solution, on the basis of above example, realize only obtain it is in the belonging m encoding block and And the data of fritter identical with the quantity of bad fritter, so as to save network overhead at the same time it can also improve data storage Reliability.

The following is an embodiment of the apparatus of the present invention, and apparatus of the present invention embodiment is for executing embodiment of the present invention method one to three The method of realization, for ease of description, only parts related to embodiments of the present invention are shown, and particular technique details does not disclose , please refer to the embodiment of the present invention one, embodiment two and embodiment three.

Fig. 6 is the structural schematic diagram for the equipment that a kind of data provided in an embodiment of the present invention are restored, currently processed data Set is divided into n data block, and the encoding block of m redundancy of the data acquisition system is calculated, and the n and m are greater than 0 Integer.As shown in fig. 6, the equipment includes：First analytical unit 41, division unit 42 and recovery unit 43, wherein：

First analytical unit 41, for when every in the n data block there are analyzing when bad block in the n data block The damage data of a bad block are the location of in each bad block；

Division unit 42, the position for being analyzed according to analytical unit 41, by the n data block and the m volume Each of code block piece is divided into h fritter according to identical division rule, in the n data block and the m encoding block In all fritters for including, it will be closed as a subset the identical fritter in present position in each data block and encoding block, often The quantity of the bad fritter for the presence damage data that a subclass includes is less than or equal to the m, and the h is the integer greater than 1；

Recovery unit 43, for there are the subclass of bad fritter for one, respectively using it is belonging this there are bad fritters Subclass in the data of unspoiled fritter restore in the subclass impaired data in bad fritter.

As an alternative embodiment, division unit 42 can be also used for the position analyzed according to, by institute It states each of n data block and the m encoding block piece and h is divided into pre-set size according to identical division rule A fritter.

Optionally, the pre-set size is the minimum unit and the m volume of the n data block storing data The size of the common multiple of the minimum unit of code block storing data.

Optionally, the pre-set size is as unit of a sector and the size including at least one sector.

In this way division unit 42 by can the above division mode the unspoiled data of each fritter storage may be implemented It is complete, i.e., the data that division unit 42 will not damage data block in partition process or encoding block is stored, to guarantee to compile Code block storage coded data (such as：Check code) coding or the functions such as verification will not fail.

In above-mentioned technical proposal, since according to the location of damage data, the n data block and described m being compiled Each of code block piece is divided into h fritter according to identical division rule, in the n data block and the m encoding block In all fritters for including, it will be closed as a subset the identical fritter in present position in each data block and encoding block, with So that the quantity of the bad fritter for the presence damage data that each subclass includes is less than or equal to the m, the h is greater than 1 Integer；The data of damage can be thus divided in less than or equal to h subclass, and the number damaged in each subclass Quantity according to the fritter at place is less than or equal to the m, then extensive using the unspoiled fritter in each subclass respectively Impaired data in bad fritter in multiple each subclass can at most restore m × h bad fritters in this way, compare The reliability of data storage can be improved in the block that m damage can only now be restored in the art

Fig. 7 is the structural schematic diagram of another equipment for restoring data provided in an embodiment of the present invention, currently processed number It is divided into n data block according to set, and the encoding block of m redundancy of the data acquisition system is calculated, the n and m are greater than 0 Integer, as shown in fig. 7, comprises：First analytical unit 51, division unit 52 and recovery unit 53, recovery unit 53 include the One acquiring unit 531 and first restores subelement 532, wherein：

First analytical unit 51, for when every in the n data block there are analyzing when bad block in the n data block The damage data of a bad block are the location of in each bad block；

Division unit 521, the position for analyzing according to will be in the n data block and the m encoding block Each of piece be divided into h fritter according to identical division rule, include in the n data block and the m encoding block In all fritters, it will be closed as a subset the identical fritter in present position in each data block and encoding block, each subset The quantity of the bad fritter for the presence damage data that conjunction includes is less than or equal to the m, and the h is the integer greater than 1.

First acquisition unit 531 is small in the belonging m encoding block for obtaining in the subclass there are bad fritter The data of unspoiled fritter in the data of block and the belonging n data block.

First restores subelement 532, and for being directed to one, there are the subclass of bad fritter, uses the m of the acquisition The evil idea that the data of unspoiled fritter are restored in the subclass in the data of fritter and the n data block in a encoding block is small Impaired data in block.

Optionally, as shown in figure 3, each fritter is divided into two fritters up and down, two subclass are formed, such first obtains Taking unit 531 can be the network transmission that data block or encoding block are carried out as unit of fritter each in subclass, when in Fig. 3 When fritter above only has the data of damage, the fritter transmitted in first acquisition unit 531 in this way is just bad block institute Fritter in the subclass of category；Network transmission is carried out as unit of entire data block or encoding block compared to the prior art, it can be with Network transmission is reduced, to save Internet resources.

As an alternative embodiment, the first recovery subelement 532 can be also used for for one, there are bad fritters Subclass, use data unspoiled in the data and the n data block in the m encoding block of the acquisition use EC algorithm restores impaired data in the bad fritter in the subclass.

Optionally, restore to damage in each subclass for example, by using pure XOR operation and algebra domain operation small Block.

As an alternative embodiment, as described in Figure 8, first acquisition unit 531 can also include：

Transmission unit 5311, for will be present in the subclass of bad fritter and do not damage for there are the subclass of bad fritter The location information of fritter be sent to storage it is described do not damage small block data storage equipment (such as：Shown in FIG. 1 storage section Point), request, which is sent, to the storage equipment returns to the number that unspoiled fritter is included in the subclass there are bad fritter According to；

Receiving unit 5312, the data returned for receiving the storage equipment；

Be achieved equipment of the invention (such as primary storage node shown in FIG. 1) and above-mentioned storage equipment (such as：Fig. 1 Shown in time memory node) between can only transmit the data information that the fritter in subclass where bad fritter is included, To reduce network transmission.

As an alternative embodiment, as shown in figure 9, recovery unit 53 can also include：

Second analytical unit 533, for analyzing the quantity of bad fritter in the subclass there are bad fritter；

Second acquisition unit 534, for obtaining in the subclass there are bad fritter, in the belonging m encoding block And the data of fritter identical with the quantity of bad fritter and the unspoiled fritter in the belonging n data block Data；

Second restores subelement 535, and for being directed to one, there are the subclass of bad fritter, uses the m of the acquisition The evil idea that the data of unspoiled fritter are restored in the subclass in the data of fritter and the n data block in a encoding block is small Impaired data in block.

As an alternative embodiment, present invention could apply to RS algorithm, (Reed Solomon, the algorithm belong to Alien word has no the accurately Chinese meaning, is common algorithm in algorithm field, and unified RS algorithm), CRS Algorithm, In RAID5 technology, RAID6 technology.

Figure 10 is the structural schematic diagram for the equipment that another data provided in an embodiment of the present invention are restored, currently processed number It is divided into n data block according to set, and the encoding block of m redundancy of the data acquisition system is calculated, the n and m are greater than 0 Integer, as shown in Figure 10, the equipment includes processor 61, communication interface 62, memory 63 and bus 64：

Wherein processor 61, communication interface 62, memory 63 complete mutual communication by bus 64；

Communication interface 62, the storage for the data from the data and the m encoding block for storing the n data block are set The data of the standby data for receiving the n data block and the m encoding block；

Batch processing code is stored in memory 63, and processor 61 is for calling the program generation stored in memory 63 Code, for performing the following operations：

Optionally, the encoding block of the above-mentioned m redundancy that the data are calculated is specially that background technique is used to mention The encoding block of m redundancy of the data is calculated in EC technology by pure XOR operation and algebra domain operation.

As an alternative embodiment, the position analyzed according to that processor 61 executes, by the n number The step of being divided into h fritter according to identical division rule according to each of block and the m encoding block piece may include：

In this way processor 61 can with and realize the storage of each fritter unspoiled data be it is complete, i.e., processor 61 exists Data block will not be damaged in partition process or data that encoding block is stored, to guarantee the coded data (example of encoding block storage Such as：Check code) coding or the functions such as verification will not fail.

In above-mentioned technical proposal, since according to the location of damage data, the n data block and described m being compiled Each of code block piece is divided into h fritter according to identical division rule, in the n data block and the m encoding block In all fritters for including, it will be closed as a subset the identical fritter in present position in each data block and encoding block, with So that the quantity of the bad fritter for the presence damage data that each subclass includes is less than or equal to the m, the h is greater than 1 Integer；The data of damage can be thus divided in less than or equal to h subclass, and the number damaged in each subclass Quantity according to the fritter at place is less than or equal to the m, then extensive using the unspoiled fritter in each subclass respectively Impaired data in bad fritter in multiple each subclass can at most restore m × h bad fritters in this way, compare The reliability of data storage can be improved in the block that m damage can only now be restored in the art.

Figure 11 is the structural schematic diagram for the equipment that another data provided in an embodiment of the present invention are restored, currently processed number It is divided into n data block according to set, and the encoding block of m redundancy of the data acquisition system is calculated, the n and m are greater than 0 Integer, as shown in figure 11, the equipment includes processor 71, communication interface 72, memory 73 and bus 74：

Wherein processor 71, communication interface 72, memory 73 complete mutual communication by bus 74；

Communication interface 72, the storage for the data from the data and the m encoding block for storing the n data block are set The data of the standby data for receiving the n data block and the m encoding block；

Batch processing code is stored in memory 73, and processor 71 is for calling the program generation stored in memory 73 Code, for performing the following operations：

For one there are the subclass of bad fritter, using fritter in the m encoding block of the acquisition data and The data of unspoiled fritter restore impaired data in the bad fritter in the subclass in the n data block.

It should be noted that communication interface 72 can be used for receiving other equipment (such as：Secondary memory node described in Fig. 1) Be sent to realize the present embodiment equipment (such as：Primary storage node described in Fig. 1) information, and by the information received transmit To processor 71, such as it is used to obtain in the subclass there are bad fritter, the data of fritter in the belonging m encoding block, with And in the belonging n data block unspoiled fritter data.

As an alternative embodiment, processor 71, which executes, receives the bad fritter institute that the storage equipment is sent The subclass in unspoiled fritter included data information the step of may include：

Receive the data that the storage equipment returns.

Wherein, processing 71 can be, and the position for not damaging fritter in the subclass of bad fritter will be present by communication interface 72 Information be sent to storage it is described do not damage small block data storage equipment (such as：Secondary memory node shown in FIG. 1), it is deposited to described Storage equipment sends request and returns to the data that unspoiled fritter is included in the subclass there are bad fritter；Processor 71 can To be the data for receiving the storage equipment by communication interface 72 and returning.

As an alternative embodiment, small in the m encoding block of acquisition described in the use that processor 71 executes The data of unspoiled fritter restore impaired in the bad fritter in the subclass in the data of block and the n data block The step of data may include：

As an alternative embodiment, processor 71 may be used also for calling the program code stored in memory 73 With for performing the following operations：

Communication interface 72 can be also used for receiving the n number from the storage equipment for the data for storing the n data block It is received in the belonging m encoding block according to the data of block, and from the storage equipment for the data for storing the m encoding block And the data of fritter identical with the quantity of bad fritter.

As an alternative embodiment, present invention could apply to RS algorithm, (Reed Solomon, the algorithm belong to Alien word has no the accurately Chinese meaning, is common algorithm in algorithm field, and unified RS algorithm, CRS Algorithm, RAID5 In technology, RAID6 technology.

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.

In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or The mutual coupling, direct-coupling or communication connection discussed can be by some communication interfaces, between device or unit Coupling or communication connection are connect, can be electrical property, mechanical or other forms.

The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.

It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit It is that each unit physically exists alone, can also be integrated in one unit with two or more units.

It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention. And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.

The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. a kind of method that data are restored, currently processed data acquisition system is divided into n data block, and the data are calculated The encoding block of m redundancy of set, the n and m are the integer greater than 0, which is characterized in that the method includes：

When the total number in the n data block and the m encoding block including bad block is greater than m, each bad block is analyzed Data are damaged the location of in correspondence bad block；

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical stroke Divider is then divided into h fritter, will be in every number in all fritters that the n data block and the m encoding block include It is closed according to the identical fritter in present position in block and each encoding block as a subset, the number for the bad fritter that each subclass includes Amount is less than or equal to the m, and the bad fritter refers to that the fritter comprising damage data, the h are the integer greater than 1；

For one, there are the subclass of bad fritter, respectively using unspoiled in the belonging subclass there are bad fritter Impaired data in bad fritter in the data recovery subclass of fritter.

2. the method according to claim 1, wherein described respectively using the belonging subset there are bad fritter Impaired data in bad fritter in the data recovery subclass of unspoiled fritter in conjunction, including：

It obtains in the subclass there are bad fritter, the data of unspoiled fritter in the belonging m encoding block, and The data of unspoiled fritter in the belonging n data block；

Using the data of unspoiled fritter in the data and the n data block of unspoiled fritter in the m encoding block, That restores that the bad fritter of in the subclass there are bad fritter the and belonging n data block included is impaired Data.

3. the method according to claim 1, wherein respectively using in the belonging subclass there are bad fritter The data of unspoiled fritter restore in the subclass impaired data in bad fritter, including：

Analyze the quantity of bad fritter in the subclass there are bad fritter；

It obtains in the subclass there are bad fritter, quantity phase in the belonging m encoding block and with bad fritter The data of same unspoiled fritter and the data of the unspoiled fritter in the belonging n data block；

It is unspoiled in data and the n data block using unspoiled fritter in the m encoding block of the acquisition The data of fritter restore impaired data in the bad fritter in the subclass there are bad fritter.

4. the method as described in claim 1, which is characterized in that the position analyzed according to, by the n data Each of block and the m encoding block piece are divided into h fritter according to identical division rule and include：

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical stroke Divider is then divided into h fritter with pre-set size, and the pre-set size is the n data block storing data Minimum unit and the m encoding block storing data minimum unit common multiple size.

5. the method as described in claim 1, which is characterized in that the position analyzed according to, by the n data Each of block and the m encoding block piece are divided into h fritter according to identical division rule and include：

According to the position analyzed, by each of the n data block and described m encoding block piece according to identical stroke Divider is then divided into h fritter with pre-set size, the pre-set size as unit of a sector and including The size of at least one sector.

6. a kind of method that data are restored, currently processed data acquisition system is divided into n data block, and the data are calculated The encoding block of m redundancy of set, the n and m are the integer greater than 0, which is characterized in that the method includes：

When there is bad block in the n data block and the m encoding block, the damage data of each bad block are analyzed right It should be the location of in bad block；

For one, there are the subclass of bad fritter, obtain in the subclass there are bad fritter the and belonging n The data of unspoiled fritter in a data block, and obtain in the subclass there are bad fritter and belonging institute The data for stating the unspoiled fritter in m encoding block, restore that described there are bad using the data of the unspoiled fritter of acquisition The impaired data that the bad fritter of the belonging n data block includes in the subclass of fritter.

7. method as claimed in claim 6, which is characterized in that the position analyzed according to, by the n data Each of block and the m encoding block piece are divided into h fritter according to identical division rule and include：

8. method as claimed in claim 6, which is characterized in that the position analyzed according to, by the n data Each of block and the m encoding block piece are divided into h fritter according to identical division rule and include：

9. such as method as claimed in claim 6 to 8, which is characterized in that described when in the n data block and the m volume When there is bad block in code block, the damage data of each bad block are analyzed the location of in correspondence bad block, specially：

If the total number in the n data block and the m encoding block comprising bad block is greater than m, the damage of each bad block is analyzed Bad data is the location of in correspondence bad block.

10. a kind of equipment for restoring data, currently processed data acquisition system is divided into n data block, and the number is calculated According to the encoding block of m redundancy of set, the n and m are the integer greater than 0, which is characterized in that the equipment includes：First point Unit, division unit and recovery unit are analysed, wherein：

First analytical unit, the total number for ought in the n data block and the m encoding block include bad block are big When m, the damage data of each bad block are analyzed the location of in correspondence bad block；

The division unit, the position for analyzing according to will be in the n data block and the m encoding block Each piece is divided into h fritter according to identical division rule, in the institute that the n data block and the m encoding block include Have in fritter, will be closed as a subset the identical fritter in present position in each data block and each encoding block, every height The quantity for the bad fritter that set includes is less than or equal to the m, and the bad fritter refers to the fritter comprising damage data, the h For the integer greater than 1；

The recovery unit, for there are the subclass of bad fritter for one, respectively using it is belonging this there are bad fritters Impaired data in bad fritter in the data recovery subclass of unspoiled fritter in subclass.

11. equipment as claimed in claim 10, which is characterized in that the recovery unit includes：

First acquisition unit is not damaged in the belonging m encoding block for obtaining in the subclass there are bad fritter Fritter data and the belonging n data block in unspoiled fritter data；

First restores subelement, for using in the m encoding block in the data of unspoiled fritter and the n data block The data of unspoiled fritter restore the bad of in the subclass there are bad fritter the and belonging n data block The impaired data that fritter is included.

12. equipment as claimed in claim 10, which is characterized in that the recovery unit includes：

Second acquisition unit, for obtaining in the subclass there are bad fritter, it is in the belonging m encoding block and It is unspoiled small in the data of unspoiled fritter identical with the quantity of bad fritter and the belonging n data block The data of block；

Second restores subelement, the data of unspoiled fritter and the n in the m encoding block for using the acquisition The data of unspoiled fritter in a data block restore impaired number in the bad fritter in the subclass there are bad fritter According to.

13. equipment as claimed in claim 10, which is characterized in that the position that the division unit is also used to analyze according to It sets, by each of the n data block and described m encoding block piece according to identical division rule with pre-set size It is divided into h fritter, the pre-set size is the minimum unit and the m volume of the n data block storing data The size of the common multiple of the minimum unit of code block storing data.

14. equipment as claimed in claim 10, which is characterized in that the position that the division unit is also used to analyze according to It sets, by each of the n data block and described m encoding block piece according to identical division rule with pre-set size It is divided into h fritter, the pre-set size is as unit of a sector and the size including at least one sector.

15. a kind of equipment for restoring data, currently processed data acquisition system is divided into n data block, and the number is calculated According to the encoding block of m redundancy of set, the n and m are the integer greater than 0, which is characterized in that the equipment includes：First point Unit, division unit and recovery unit are analysed, wherein：

First analytical unit, for analyzing when there is bad block in the n data block and the m encoding block The damage data of each bad block are the location of in correspondence bad block；

The recovery unit, for being directed to one, there are the subclass of bad fritter, obtain in the subclass there are bad fritter And the unspoiled fritter in the belonging n data block data, and obtain that described there are the subsets of bad fritter The data of unspoiled fritter in conjunction the and belonging m encoding block use the unspoiled fritter of acquisition Data restore the impaired number that the bad fritter of the belonging n data block in the subclass there are bad fritter includes According to.

16. equipment as claimed in claim 15, which is characterized in that the position that the division unit is also used to analyze according to It sets, by each of the n data block and described m encoding block piece according to identical division rule with pre-set size It is divided into h fritter, the pre-set size is the minimum unit and the m volume of the n data block storing data The size of the common multiple of the minimum unit of code block storing data.

17. equipment as claimed in claim 15, which is characterized in that the position that the division unit is also used to analyze according to It sets, by each of the n data block and described m encoding block piece according to identical division rule with pre-set size It is divided into h fritter, the pre-set size is as unit of a sector and the size including at least one sector.

18. the equipment as described in any one of claim 15-17, which is characterized in that first analytical unit, for working as institute When stating in n data block there are there is bad block in bad block and the m encoding block, the damage data of each bad block are analyzed right It should be the location of in bad block, specially：

If in the n data block and the m encoding block comprising bad block total number be greater than m, first analytical unit, For analyzing the damage data of each bad block the location of in correspondence bad block.

19. the equipment that a kind of data are restored, currently processed data acquisition system is divided into n data block, and the number is calculated According to the encoding block of m redundancy of set, the n and m are the integer greater than 0, which is characterized in that the equipment includes processor, Communication interface, memory and bus：

The communication interface, the storage equipment for the data from the data and the m encoding block for storing the n data block Receive the data of the n data block and the data of the m encoding block；

Batch processing code is stored in the memory；

When the equipment operation that the data are restored, the processor reads the said program code of the memory storage, with So that the method that the equipment perform claim that the data are restored requires 1 to 5 described in any item data to restore.

20. the equipment that a kind of data are restored, currently processed data acquisition system is divided into n data block, and the number is calculated According to the encoding block of m redundancy of set, the n and m are the integer greater than 0, which is characterized in that the equipment includes processor, Communication interface, memory and bus：

Batch processing code is stored in the memory；

When the equipment operation that the data are restored, the processor reads the said program code of the memory storage, with So that the method that the equipment perform claim that the data are restored requires 6 to 9 described in any item data to restore.