CN105930232B - A kind of simple regeneration code restorative procedure using network topological information - Google Patents
A kind of simple regeneration code restorative procedure using network topological information Download PDFInfo
- Publication number
- CN105930232B CN105930232B CN201610312764.3A CN201610312764A CN105930232B CN 105930232 B CN105930232 B CN 105930232B CN 201610312764 A CN201610312764 A CN 201610312764A CN 105930232 B CN105930232 B CN 105930232B
- Authority
- CN
- China
- Prior art keywords
- node
- reparation
- tree
- data block
- optimal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000008929 regeneration Effects 0.000 title claims abstract description 42
- 238000011069 regeneration method Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000008439 repair process Effects 0.000 claims abstract description 58
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 18
- 230000005540 biological transmission Effects 0.000 claims abstract description 7
- 230000008569 process Effects 0.000 claims description 12
- 230000004048 modification Effects 0.000 claims description 8
- 238000012986 modification Methods 0.000 claims description 8
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000003860 storage Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 3
- 235000013399 edible fruits Nutrition 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The present invention discloses a kind of simple regeneration code restorative procedure using network topological information, the specific steps are：1, it is configured according to the distributed memory system where failure node and the simple regeneration code of application, the simple regeneration code coding parameter (n that acquisition system uses, k, f), the link set in the memory node set and memory node set that node is constituted between each node is participated in by node to be repaired, reparation；2, network G (N, E) is repaired in building；3, according to the reparation network established in step 2, a parallel repair of f+1 for being directed to all f+1 loss data blocks using parallel reparation tree developing algorithm building sets Fi(1≤i≤f+1)；4, to reparation tree F obtained in step 3i(1≤i≤f+1) carries out the reparation to data block.The present invention is by optimizing simple regeneration code repair process using network link bandwidth information and the coding/decoding capability of transmission path intermediate node, building is directed to the tree-shaped repair path of particular lost data block, enable data block repair process reasonable employment high-bandwidth link, reduces and repair time delay.
Description
Technical field
The present invention relates to distributed storages and network code field, in particular to a kind of to utilize the simple of network topological information
Regenerate code restorative procedure.
Background technique
With the rise of big data technology, key technology one of of the mass data storage as big data technology is obtained
It is more and more to pay attention to.Distributed memory system is deployed in mostly on cheap business machine, and the node failure in system is one
Kind normality.
For the reliability of data in guarantee system, need to introduce storage redundancy strategy for distributed memory system.It is typical
Storage redundancy strategy be more replication policies and Reed Solomon Coding.Network coding technique was suggested in 2000, its core
Thought thinks that each node in network not only has a routing function, the data that node receives can also be carried out it is linear or
Then nonlinear coded treatment goes out the data forwarding after coding.Using network code, data multicast routing can achieve
Theoretical maximum stream circle, improve data transfer efficiency and network bandwidth availability ratio.
Storage redundancy strategy based on network code is introduced in distributed memory system, using with MDS property
Coding ensures data reliability, and makes optimization for the network overhead and magnetic disc i/o expense of node reparation, and main includes regeneration
Code (Regenerating Code) and part can repair code (Locally Repairable Code).
Simple regeneration code can repair one kind of code as part, and there is magnetic disc i/o when fault-tolerant ability is high, node is repaired to load
The features such as low.The discussion of simple regeneration code is concentrated mainly in coding theory at present：In the recovery of simple regeneration code interior joint
In the process, losing participation data block required when data block is restored all is to pass to failure node by direct link.Failure section
Point carries out XOR operation after receiving specific participation data block, recovers loss data block.This node repair process is ignored
The topology information and link bandwidth information of network.
Summary of the invention
Goal of the invention：In order to overcome the deficiencies in the prior art, the present invention, which provides, a kind of utilizes network topological information
Construct the restorative procedure for simple regeneration code of parallel tree-shaped repair path.
Technical solution：In order to solve the above technical problems, the present invention provides it is a kind of using network topological information it is simple again
Raw code restorative procedure, specific step is as follows for data reparation：
Step 1：It is configured, is obtained according to the distributed memory system where failure node and the simple regeneration code of application
The simple regeneration code coding parameter (n, k, f) that system uses by node to be repaired, repairs the memory node for participating in node and being constituted
Link set in set and memory node set between each node；
Step 2：Network G (N, E) is repaired in building, and wherein N is that node set is made of node to be repaired and participation node,
Meet | N |=2f+1, f+1 node set Vi(1≤i≤f+1), set ViRepresent i-th of loss data block on node to be repaired
Participation node set needed for repairing,E is link set, is met | E | structure between each pair of node in=f (2f+1), N
Build a link；Each of the links in E are all associated with a bandwidth C (e), e ∈ E；
Step 3：According to the reparation network established in step 2, tree developing algorithm building is repaired for all f using parallel
F+1 of+1 loss data block are parallel to repair tree Fi(1≤i≤f+1)；
Step 4：To reparation tree F obtained in step 3i(1≤i≤f+1) carries out the reparation to data block.
Wherein, tree F is repaired in building in step 3i(1≤i≤f+1) specific step is as follows：
Step 3.1：Repair data number of blocks is m, is initialized as 0 to the data block repaired, represents no data block
It is repaired, has constructed successfully to repair and set set F, F is initialized as sky；
Step 3.2：Loss data block is f+1, judges whether the data block f+1 lost and repair data block are equal, such as
Fruit is unequal to enter step 3.3, if equal proof repairs tree set Fi(1≤i≤f+1) is constructed successfully；
Step 3.3：The optimal reparation of f+1-m item is generated to remaining data block and sets OTj(1≤j≤f+1-m)；
Step 3.4：OT is set to optimal reparationj(1≤j≤f+1-m) is ranked up by rule；
Step 3.5：For the reparation tree OT of highest prioritysDistribute link bottleneck bandwidth Bs；
Step 3.6：Link bandwidth is adjusted for OTsLink e, C (e)=C (e)-B of processs；
Step 3.7：The reparation tree OT of corresponding data blocksReparation tree construct complete, by OTsIt is added to reparation tree set F
In, repair data block m=m+1, and be again introduced into step 3.2 for m and judged with loss data block f+1.
Wherein, repaired that specific step is as follows in step 4 to data block：
Step 4.1：A node in OT is set in some given corresponding reparation for losing data block；
Step 4.2：Judge whether root node obtains all data blocks, data block is successfully restored if obtaining, if do not had
Have, is entered step 4.3；
Step 4.3：The data for judging whether to receive child node transmission, if the data for receiving child node transmission into
Enter step 4.4, otherwise carries out judging whether there is again and repair required and not sent data, and if so, being sent out to father node
It send data and is again introduced into step 4.2；
Step 4.4：It judges whether there is and repairs required and not sent data, if there is no then to father node forwarding number
According to and be again introduced into step 4.2, and if so, being sent to father node and again after received data and stored data XOR operation
It is secondary to enter step 4.2.
Wherein, optimal reparation tree OT is generated in step 3.3j(1≤j≤f+1-m) specific step is as follows：
Step 3.3.1：The participation node set V of a given corresponding non-repair data blocki(1≤i≤f+1-m)；
Step 3.3.2：According to the optimal bottleneck path p of the dijkstra's algorithm of modification calculating source node to all nodest
(t∈N)；
Step 3.3.3：For ViIn all nodes optimal bottleneck path pt(t ∈ N) is merged；
Step 3.3.4：Using obtained source node as root, V is generatediIn all nodes optimal reparation tree.
Wherein, the optimal ordering rule for repairing tree is in step 3.4：Optimal reparation tree OT has d side, where each edge
Link has an available bandwidth value, to it is all while according to while where link available bandwidth ascending order arrangement after, obtain d side
An arrangement b1≤b2≤b3≤…≤bd, b1For the bandwidth bottleneck link of the reparation tree；Two corresponding different files to be repaired
The reparation tree OT of block1And OT2, two are repaired the n and m link that tree passes through respectively in network, then OT1< OT2And if only if：
1) there are k (k≤n, k≤m), so that
2) n < m and
Wherein, optimal bottleneck path p of the calculating source node to all nodes in step 3.3.2tThe specific steps of (t ∈ N)
It is as follows：
Step 3.3.2.1：Obtain the available bandwidth C (e), e ∈ E on network information G (N, E) and every remotest places；
Step 3.3.2.2：Source node s is marked, and defines the bottleneck band in the optimal bottleneck path from source node s to node x
Width is widthto (x), bottleneck bandwidth widthto (s)=∞ of the node s to its own；
Step 3.3.2.3：Judge whether all nodes are labeled, obtains if all nodes are all labeled by source
Node s sets out into network the optimal bottleneck path p of each nodet(t ∈ N), if all nodes are not all marked
Then enter step 3.3.2.4；
Step 3.3.2.4：Examine bottleneck bandwidth widthto (j)=max of all flag node k to tail flag node j
{min{widthto(k),C(k,j)}}；
Step 3.3.2.5：Never widthto (i) maximum i is chosen in flag node；
Step 3.3.2.6：It is pre (i) that the node being connected directly with i is searched in marked node；
Step 3.3.2.7：Flag node i simultaneously carries out judging whether all nodes have been marked again by step 3.3.2.3
Note.
Wherein, as follows about optimal bottleneck path merging process in step 3.3.3：
To the optimal bottleneck path p of node i by source node si:s,n1,n2,…,nk,…,i；
To the optimal bottleneck path p of node j by source node sj:s,m1,m2,…,mt,…,j；
Two paths are in respective point nkWith mtIntersect, i.e. nk=mt,nk-1！=mt-1, then merge after tree-shaped knot
Structure is selection s, n1,n2,…,nkOr s, m1,m2,…,mtAs common link, piWith pjUpper nkLink later as sublink,
It is linked to n respectivelykOn.
The coding mode of simple regeneration code is a kind of block encoding process：In (n, k, f) simple regeneration code, input file quilt
It is cut into f block, every piece is recoded into n sub-block using MDS (n, k) coding.Obtained f group coded data block carries out exclusive or
Operation obtains one group of check block.F+1 group data block is placed on n memory node according to certain rule.By simple regeneration
There is f+1 data block on each memory node after code coding.Need to restore all f+1 loss numbers when repairing failure node
According to block, and participation node set needed for repairing each data block is different.This encoding characteristics cause for common correcting and eleting codes
Repairing algorithm with the tree-shaped node of regeneration code can not be applied in the node repair process of simple regeneration code, and therefore, the present invention mentions
Algorithm is repaired for the parallel tree-shaped of simple regeneration code node repair process out.
The present invention is that f+1 loss data block of failure node in (n, k, f) simple regeneration code plans tree-shaped reparation respectively
Path, so that different data block repair process parallelization, reduces node and repair time delay, improve network bandwidth utilization factor.Simultaneously originally
The calculation for inventing the encoding and decoding without changing simple regeneration code uses simple regeneration code as storage redundancy suitable for all
The distributed memory system of strategy.
Compared with prior art, the present invention having the advantages that：
1, the present invention is by the coding/decoding capability using network link bandwidth information and transmission path intermediate node to simple
Regeneration code repair process optimizes, and building is directed to the tree-shaped repair path of particular lost data block, so that data block was repaired
Journey energy reasonable employment high-bandwidth link reduces and repairs time delay.
2, decoding method of the bottom decoding method based on simple regeneration code script in the present invention, makes suitable for all
Use simple regeneration code as the distributed memory system of storage redundancy strategy.
Detailed description of the invention
Fig. 1 is overall framework flow chart of the invention；
Fig. 2 is to generate the optimal flow chart for repairing tree；
Fig. 3 is the flow chart for calculating source node to the optimal bottleneck path of all nodes；
Fig. 4 is the flow chart repaired by losing the reparation tree of data block to data block；
Fig. 5 is the instance graph of (5,3,2) simple regeneration code node reparation.
Specific embodiment
The present invention will be further explained with reference to the accompanying drawing.
The present invention is based on use distributed memory system of the simple regeneration code as storage redundancy strategy to utilize network topology
Information and bandwidth information optimize node repair process.
Data reparation as described in Figure 1 the specific steps are：
Step 1：It is configured, is obtained according to the distributed memory system where failure node and the simple regeneration code of application
The simple regeneration code coding parameter (n, k, f) that system uses by node to be repaired, repairs the memory node for participating in node and being constituted
Link set in set and memory node set between each node；
Step 2：Network G (N, E) is repaired in building, and wherein N is that node set is made of node to be repaired and participation node,
Meet | N |=2f+1, f+1 node set Vi(1≤i≤f+1), set ViRepresent i-th of loss data block on node to be repaired
Participation node set needed for repairing,E is link set, is met | E | structure between each pair of node in=f (2f+1), N
Build a link；Each of the links in E are all associated with a bandwidth C (e), e ∈ E；
Step 3：According to the reparation network established in step 2, tree developing algorithm building is repaired for all using parallel
F+1 of f+1 loss data block are parallel to repair tree Fi(1≤i≤f+1)；As shown in Figure 1, tree F is repaired in buildingi(1≤i≤f+
1) the specific steps are：
Step 3.1：Repair data number of blocks is m, is initialized as 0 to the data block repaired, represents no data block
It is repaired, has constructed successfully to repair and set set F, F is initialized as sky；
Step 3.2：Loss data block is f+1, judges whether the data block f+1 lost and repair data block are equal, such as
Fruit is unequal to enter step 3.3, if equal proof repairs tree set Fi(1≤i≤f+1) is constructed successfully；
Step 3.3：The optimal reparation of f+1-m item is generated to remaining data block and sets OTj(1≤j≤f+1-m)；As shown in Fig. 2,
Generating the step of optimal reparation is set is：
Step 3.3.1：The participation node set V of a given corresponding non-repair data blocki(1≤i≤f+1-m)；
Step 3.3.2：According to the optimal bottleneck path p of the dijkstra's algorithm of modification calculating source node to all nodest
(t∈N)；As shown in figure 3, calculate source node to the optimal bottleneck paths of all nodes the specific steps are：
Step 3.3.2.1：Obtain the available bandwidth C (e), e ∈ E on network information G (N, E) and every remotest places；
Step 3.3.2.2：Source node s is marked, and defines the bottleneck band in the optimal bottleneck path from source node s to node x
Width is widthto (x), bottleneck bandwidth widthto (s)=∞ of the node s to its own；
Step 3.3.2.3：Judge whether all nodes are labeled, obtains if all nodes are all labeled by source
Node s sets out into network the optimal bottleneck path p of each nodet(t ∈ N), if all nodes are not all marked
Then enter step 3.3.2.4；
Step 3.3.2.4：Examine bottleneck bandwidth widthto (j)=max of all flag node k to tail flag node j
{min{widthto(k),C(k,j)}}；
Step 3.3.2.5：Never widthto (i) maximum i is chosen in flag node；
Step 3.3.2.6：It is pre (i) that the node being connected directly with i is searched in marked node；
Step 3.3.2.7：Flag node i simultaneously carries out judging whether all nodes have been marked again by step 3.3.2.3
Note.
Step 3.3.3：For ViIn all nodes optimal bottleneck path pt(t ∈ N) is merged；Optimal bottleneck path
Merging process is as follows：
To the optimal bottleneck path p of node i by source node si:s,n1,n2,…,nk,…,i；
To the optimal bottleneck path p of node j by source node sj:s,m1,m2,…,mt,…,j；
Two paths are in respective point nkWith mtIntersect, i.e. nk=mt,nk-1！=mt-1, then merge after tree-shaped knot
Structure is selection s, n1,n2,…,nkOr s, m1,m2,…,mtAs common link, piWith pjUpper nkLink later as sublink,
It is linked to n respectivelykOn.
Step 3.3.4：Using obtained source node as root, V is generatediIn all nodes optimal reparation tree.
Step 3.4：OT is set to optimal reparationj(1≤j≤f+1-m) is ranked up by rule；The optimal sequence rule for repairing tree
Then it is：Optimal reparation tree OT has d side, and the link where each edge has an available bandwidth value, to it is all while according to while
After the available bandwidth ascending order arrangement of place link, an arrangement b on d side is obtained1≤b2≤b3≤…≤bd, b1For the reparation
The bandwidth bottleneck link of tree；The reparation tree OT of two corresponding different blocks of files to be repaired1And OT2, two reparation trees are respectively by net
N and m link in network, then OT1< OT2And if only if：
1) there are k (k≤n, k≤m), so that
2) n < m and
Step 3.5：For the reparation tree OT of highest prioritysDistribute link bottleneck bandwidth Bs；
Step 3.6：Link bandwidth is adjusted for OTsLink e, C (e)=C (e)-B of processs；
Step 3.7：The reparation tree OT of corresponding data blocksReparation tree construct complete, by OTsIt is added to reparation tree set F
In, repair data block m=m+1, and be again introduced into step 3.2 for m and judged with loss data block f+1.
Step 4：To reparation tree F obtained in step 3i(1≤i≤f+1) carries out the reparation to data block.Such as Fig. 4 institute
Show, by lose data block reparation tree to data block repaired the specific steps are：
Step 4.1：A node in OT is set in some given corresponding reparation for losing data block；
Step 4.2：Judge whether root node obtains all data blocks, data block is successfully restored if obtaining, if do not had
Have, is entered step 4.3；
Step 4.3：The data for judging whether to receive child node transmission, if the data for receiving child node transmission into
Enter step 4.4, otherwise carries out judging whether there is again and repair required and not sent data, and if so, being sent out to father node
It send data and is again introduced into step 4.2；
Step 4.4：It judges whether there is and repairs required and not sent data, if there is no then to father node forwarding number
According to and be again introduced into step 4.2, and if so, being sent to father node and again after received data and stored data XOR operation
It is secondary to enter step 4.2.
The Network status in the node repair process of (5,3,2) simple regeneration code is indicated by taking Fig. 5 as an example.Wherein there are 5 sections
Point is connected between node by network link two-by-two, and each of the links have corresponding bandwidth.
Assuming that node i is node to be repaired, wherein loss data block is respectively with corresponding reparation participation node：
Data block xi, participation node is node i -2, node i -1；
Data block yi+1, participation node is node i -1, node i+1；
Data block si+2, participation node is node i+1, node i+2；
In the algorithmic procedure of Fig. 1 description, m is initialized as 0, the algorithm after 3 wheel three tree-shaped repair paths of grey iterative generation
Terminate.This three repair paths respectively correspond a loss data block.
In first round iteration, three optimal reparations are generated first with optimal reparation tree generation algorithm and are set, process is as follows：
Step 1：The optimal bottle from node i to other nodes is calculated using the dijkstra's algorithm of the modification described by Fig. 3
Neck path：
Initialization：D={ i },Widthto (i)=∞
The first round：D={ i },
Widthto (i-1)=min { widthto (i), C (i, i-1) }=40,
Widthto (i-2)=min { widthto (i), C (i, i-2) }=45,
Widthto (i+1)=min { widthto (i), C (i, i+1) }=30,
Widthto (i+2)=min { widthto (i), C (i, i+2) }=35
Second wheel：D={ i, i-2 },
Third round：D={ i, i-2, i-1 },
Fourth round：D={ i, i-2, i-1, i+1 },
As a result：Widthto (i)=∞, widthto (i-1)=45, widthto (i-2)=45,
Widthto (i+1)=45, widthto (i+2)=35
Obtain be by the optimal bottleneck path of node i to each node：
Node i -2:I → i-2, bottleneck link bandwidth 45；
Node i -1:I → i-2 → i-1, bottleneck link bandwidth 45；
Node i+1:I → i-2 → i-1 → i+1, bottleneck link bandwidth 45；
Node i+2:I → i+2, bottleneck link bandwidth 35；
:Step 2：For each lost block data block, its optimal reparation tree is constructed：
Data block xiReparation to participate in node be i-2, i-1, then its optimal reparations tree is i-1 → i-2 → i；
Data block yi+1Reparation to participate in node be i-1, i+1, then its optimal reparations tree is i+1 → i-1 → i-2 → i；
Data block si+2Reparation to participate in node be i+1, i+2, then its optimal reparations, which is set, is
i+2→i,i+1→i-1→i-2→i；
Step 3：Obtained all optimal reparations are set and are ranked up according to the strategy, the reparation of highest priority is taken out
Tree：
Quicksort, merger sequence scheduling algorithm can be used in sort algorithm.Wait the size relation between the element that sorts according to
The rule is judged.
Got in step 2 three optimal reparation trees are ranked up according to the rule, obtain highest priority most
Excellent reparation tree is data block si+2Optimal reparation tree, the bottleneck bandwidth of the reparation tree is 35.For the optimal chain repairing tree and passing through
Road distributes respective bandwidth and is used for si+2Repair process, then data block si+2Reparation time delay be m/35.
Second wheel iteration, carries out same process on remaining sub-network figure.
A step：Obtained using the dijkstra's algorithm of modification be by the optimal bottleneck path of node i to each node：
Node i -2:I-2 → i+1 → i+2 → i-1 → i, link bottleneck bandwidth 25；
Node i -1:I-1 → i, link bottleneck bandwidth 40；
Node i+1:I+1 → i+2 → i-1 → i, link bottleneck bandwidth are 35；
Node i+2:I+2 → i-1 → i, link bottleneck bandwidth 35；
B step：For remaining data block to be repaired, its optimal reparation tree is constructed：
Data block xiOptimal reparation tree be i-2 → i+1 → i+2 → i-1 → i；
Data block yi+1Optimal reparation tree be i+1 → i+2 → i-1 → i；
C step：Obtained optimal reparation tree is ranked up according to the rule, obtains the optimal reparation of highest priority
Tree is data block xiReparation tree, the reparation tree bottleneck bandwidth be 25.The optimal bandwidth repaired needed for setting is distributed in network
Resource is used for xiReparation, then data block xiReparation time delay be m/25.
Third round iteration carries out same process on remaining sub-network figure.
Data block to be repaired is yi+1, participation node is node i -1, node i+1.
A step：It is obtained using the dijkstra's algorithm of modification by node i to described two optimal bottleneck roads for participating in node
Diameter is：
Node i -1:I-1 → i+1 → i, link bottleneck bandwidth 20；
Node i+1:I+1 → i, link bottleneck bandwidth 30；
B step：For data block y to be repairedi+1, construct its optimal reparation tree：
Data block yi+1Optimal reparation tree be i-1 → i+1 → i；
C step：Selectable optimal reparation tree only has the yi+1Optimal reparation tree, the bottleneck bandwidth of the reparation tree is
20.In the reparation tree, data block yi+1Reparation time delay be m/20.
After algorithm proposed by the present invention operation, 3 parallel tree-shaped repair paths are generated.Utilize this 3 parallel repair way
Diameter repairs failure node i parallel, and the reparation time delay of node i is max { m/35, m/25, m/20 }=m/20.
In the conventional node repair process of simple regeneration code, participate in node data passed to by direct link it is to be repaired
Knot cluster point.In this mode, repair process time delay shown in Fig. 5 is max { 2m/40,2m/30, m/45, m/35 }=m/15.
From the foregoing, it will be observed that node can be effectively reduced in this example using algorithm proposed by the present invention repairs time delay.
The method of the present invention utilizes network topology and link bandwidth information, utilizes the encoding and decoding energy of network code intermediate node
Power optimizes the node repair process of simple regeneration code, reduces the reparation time delay of node.The method of the present invention calculates letter
Single, versatility and scalability are strong, to using node of the simple regeneration code as the distributed memory system for storing redundancy strategy to repair
Multiple efficiency has very big promotion.
The above is only a preferred embodiment of the present invention, it should be pointed out that：For the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (7)
1. a kind of simple regeneration code restorative procedure using network topological information, which is characterized in that the specific steps of data reparation
It is as follows：
Step 1：It is configured according to the distributed memory system where failure node and the simple regeneration code of application, obtains system
The simple regeneration code coding parameter (n, k, f) used by node to be repaired, repairs the memory node set for participating in node and being constituted
And the link set in memory node set between each node；
Step 2：Network G (N, E) is repaired in building, and wherein N is that node set is made of node to be repaired and participation node, is met |
N |=2f+1, f+1 node set Vi(1≤i≤f+1), set ViRepresent i-th of loss data block reparation on node to be repaired
Required participation node set,E is link set, is met | E | one is constructed between each pair of node in=f (2f+1), N
Link；Each of the links in E are all associated with a bandwidth C (e), e ∈ E；
Step 3：According to the reparation network established in step 2, tree developing algorithm building is repaired for all f+1 using parallel
F+1 parallel repair for losing data block set Fi(1≤i≤f+1)；
Step 4：To reparation tree F obtained in step 3i(1≤i≤f+1) carries out the reparation to data block.
2. a kind of simple regeneration code restorative procedure using network topological information according to claim 1, which is characterized in that
Tree F is repaired in building in the step 3i(1≤i≤f+1) specific step is as follows：
Step 3.1：Repair data number of blocks is m, is initialized as 0 to the data block repaired, represents no data block and obtain
It repairs, has constructed successfully to repair and set set F, F is initialized as sky；
Step 3.2：Loss data block is f+1, judges whether the data block f+1 lost and repair data block are equal, if not
It is equal to enter step 3.3, if equal proof repairs tree set Fi(1≤i≤f+1) is constructed successfully；
Step 3.3：The optimal reparation of f+1-m item is generated to remaining data block and sets OTj(1≤j≤f+1-m)；
Step 3.4：OT is set to optimal reparationj(1≤j≤f+1-m) is ranked up by rule；
Step 3.5：For the reparation tree OT of highest prioritysDistribute link bottleneck bandwidth Bs；
Step 3.6：Link bandwidth is adjusted for OTsLink e, C (e)=C (e)-B of processs；
Step 3.7：The reparation tree OT of corresponding data blocksReparation tree construct complete, by OTsIt is added in reparation tree set F,
Repair data block m=m+1, and be again introduced into step 3.2 for m and judged with data block f+1 is lost.
3. a kind of simple regeneration code restorative procedure using network topological information according to claim 1, which is characterized in that
Repaired that specific step is as follows in the step 4 to data block：
Step 4.1：The optimal reparation tree OT of some given corresponding loss data blockjIn the reparation tree OT of one of (1≤j≤f+1-m)
A node；
Step 4.2：Judging whether root node obtains all data blocks, data block is successfully restored if obtaining, if not
To then entering step 4.3；
Step 4.3：Judge whether to receive the data that child node is sent, step is entered if the data for receiving child node transmission
Rapid 4.4, it otherwise carries out judging whether there is again and repairs required and not sent data, and if so, sending number to father node
According to and be again introduced into step 4.2；
Step 4.4：Judge whether there is reparation needed for and not sent data, if there is no then to father node forwarding data simultaneously
Be again introduced into step 4.2, and if so, be sent to after received data and stored data XOR operation father node and again into
Enter step 4.2.
4. a kind of simple regeneration code restorative procedure using network topological information according to claim 2, which is characterized in that
Optimal reparation tree OT is generated in the step 3.3j(1≤j≤f+1-m) specific step is as follows：
Step 3.3.1：The participation node set V of a given corresponding non-repair data blocki(1≤i≤f+1-m)；
Step 3.3.2：According to the optimal bottleneck path p of the dijkstra's algorithm of modification calculating source node to all nodest(t∈
N)；
Step 3.3.3：For ViIn all nodes optimal bottleneck path pt(t ∈ N) is merged；
Step 3.3.4：Using obtained source node as root, V is generatediIn all nodes optimal reparation tree.
5. a kind of simple regeneration code restorative procedure using network topological information according to claim 2, which is characterized in that
The optimal ordering rule for repairing tree is in the step 3.4：Optimal reparation tree OT has d side, and the link where each edge has
One available bandwidth value, to it is all while according to while where link the arrangement of available bandwidth ascending order after, obtain one of d side row
Arrange b1≤b2≤b3≤...≤bd, b1For the bandwidth bottleneck link of the reparation tree；Two correspond to repairing for different blocks of files to be repaired
OT is set again1And OT2, two are repaired the n and m link that tree passes through respectively in network, then OT1< OT2And if only if：
1) there are k (k≤n, k≤m), so that
2) n < m and
6. a kind of simple regeneration code restorative procedure using network topological information according to claim 4, which is characterized in that
Optimal bottleneck path p of the calculating source node to all nodes in the step 3.3.2t(t ∈ N) specific step is as follows：
Step 3.3.2.1：Obtain the available bandwidth C (e), e ∈ E in network information G (N, E) and each edge；
Step 3.3.2.2：Source node s is marked, and the bottleneck bandwidth for defining the optimal bottleneck path from source node s to node x is
Bottleneck bandwidth widthto (the s)=∞ of widthto (x), node s to its own；
Step 3.3.2.3：Judge whether all nodes are labeled, obtains if all nodes are all labeled by source node
S sets out into network the optimal bottleneck path p of each nodet(t ∈ N), if all nodes are not all marked into
Enter step 3.3.2.4；
Step 3.3.2.4：Examine bottleneck bandwidth widthto (j)=max { min of all flag node k to tail flag node j
{widthto(k),C(k,j)}}；
Step 3.3.2.5：Never widthto (i) maximum i is chosen in flag node；
Step 3.3.2.6：It is pre (i) that the node being connected directly with i is searched in marked node；
Step 3.3.2.7：Flag node i simultaneously carries out judging whether all nodes are labeled again by step 3.3.2.3.
7. a kind of simple regeneration code restorative procedure using network topological information according to claim 4, which is characterized in that
It is as follows about optimal bottleneck path merging process in the step 3.3.3：
To the optimal bottleneck path p of node i by source node si:s,n1,n2,…,nk,…,i；
To the optimal bottleneck path p of node j by source node sj:s,m1,m2,…,mt,…,j；
Two paths are in respective point nkWith mtIntersect, i.e. nk=mt,nk-1！=mt-1, then merge after tree be
Select s, n1,n2,…,nkOr s, m1,m2,…,mtAs common link, piWith pjUpper nkLink later is as sublink, respectively
It is linked to nkOn.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610312764.3A CN105930232B (en) | 2016-05-12 | 2016-05-12 | A kind of simple regeneration code restorative procedure using network topological information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610312764.3A CN105930232B (en) | 2016-05-12 | 2016-05-12 | A kind of simple regeneration code restorative procedure using network topological information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105930232A CN105930232A (en) | 2016-09-07 |
| CN105930232B true CN105930232B (en) | 2018-11-30 |
Family
ID=56834799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610312764.3A Expired - Fee Related CN105930232B (en) | 2016-05-12 | 2016-05-12 | A kind of simple regeneration code restorative procedure using network topological information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105930232B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109194444A (en) * | 2018-09-01 | 2019-01-11 | 哈尔滨工程大学 | A kind of balanced binary tree restorative procedure based on network topology |
| CN109828723B (en) * | 2019-02-13 | 2020-05-05 | 山东大学 | Distributed storage system and accurate data restoration method and device thereof |
| CN109857585A (en) * | 2019-02-20 | 2019-06-07 | 哈尔滨工程大学 | A kind of correcting and eleting codes failure node reconstructing method based on balanced binary tree |
| CN111143108B (en) * | 2019-12-09 | 2023-05-02 | 成都信息工程大学 | Coding and decoding method and device for reducing array code Xcode repair |
| CN111224747A (en) * | 2019-12-19 | 2020-06-02 | 湖北大学 | Coding method capable of reducing repair bandwidth and disk reading overhead and repair method thereof |
| CN111385200B (en) * | 2020-03-04 | 2022-03-04 | 中国人民解放军国防科技大学 | Control method and device for data block repair |
| CN111949656B (en) * | 2020-07-25 | 2022-07-08 | 苏州浪潮智能科技有限公司 | Method and system for realizing pairing check of multiple concurrent data based on B + tree |
| CN113938376B (en) * | 2021-11-25 | 2023-08-01 | 桂林电子科技大学 | Fault node repairing method in distributed storage system |
| CN114595092B (en) * | 2022-04-28 | 2022-09-20 | 阿里云计算有限公司 | Distributed storage system, data reconstruction method, device and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461781A (en) * | 2014-12-01 | 2015-03-25 | 华中科技大学 | Data block reconstruction method based on erasure codes |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2359168A1 (en) * | 2000-10-25 | 2002-04-25 | John Doucette | Design of meta-mesh of chain sub-networks |
| US8483096B2 (en) * | 2008-07-22 | 2013-07-09 | The Regents Of The University Of California | Scalable commodity data center network architecture |
-
2016
- 2016-05-12 CN CN201610312764.3A patent/CN105930232B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461781A (en) * | 2014-12-01 | 2015-03-25 | 华中科技大学 | Data block reconstruction method based on erasure codes |
Non-Patent Citations (5)
| Title |
|---|
| A scalable, commodity data center network architecture;AI-Fares M, Loukissas A, Vahdat A;《ACM SIGCOMM Computer Communication Review》;20081031;全文 * |
| Dcell: A scalable and fault-tolerant network structure for data centers;Guo C, Wu H, Tan K, Shi L, Zhang Y, Lu S;《ACM SIGCOMM Computer Communication Review》;20081031;全文 * |
| Tree-Structured data regeneration in distributed storage systems with regenerating codes;Li J, Yang S, Wang X, Li B;《In: Proc. of the IEEE INFOCOM 2010》;20101231;全文 * |
| Tree-Structured parallel regeneration for multiple data losses in distributed storage systems based on erasure codes;Sun W, Wang Y, Pei X;《China Communications》;20131231;全文 * |
| 一种基于再生码的数据修复算法;张小玲;《广东通信技术》;20150430;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105930232A (en) | 2016-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105930232B (en) | A kind of simple regeneration code restorative procedure using network topological information | |
| Li et al. | Coding for distributed fog computing | |
| Li et al. | Tree-structured data regeneration in distributed storage systems with regenerating codes | |
| US9722637B2 (en) | Construction of MBR (minimum bandwidth regenerating) codes and a method to repair the storage nodes | |
| CN102075198B (en) | Quasi-cyclic low-density parity check convolution code coding-decoding system and coding-decoding method thereof | |
| CN103124182A (en) | File download and streaming system | |
| CN105515590B (en) | A kind of effective low complex degree serially offsets list polarization code coding method | |
| CN110190926B (en) | Erasure code updating method and system based on network computing | |
| CN105262564A (en) | Two-dimensional distribution design method for distributed fountain code | |
| Li et al. | Compressed coded distributed computing | |
| CN106776129A (en) | A kind of restorative procedure of the multinode data file based on minimum memory regeneration code | |
| CN101562556A (en) | Method, device and system for reducing network coding cost | |
| Gerami et al. | Partial repair for wireless caching networks with broadcast channels | |
| CN112035059A (en) | Single-point failure recovery method for distributed storage system, electronic equipment and storage medium | |
| CN107896137A (en) | A kind of sort method suitable for the division of polarization code decoding path | |
| CN108347306A (en) | Class Partial Reconstruction code coding and node failure restorative procedure in distributed memory system | |
| CN102420616B (en) | Error correction method by using quasi-cyclic LDPC code based on Latin square | |
| CN104782101B (en) | Coding, reconstruct and restoration methods for the selfreparing code of distributed network storage | |
| Yang et al. | Capacity analysis of linear operator channels over finite fields | |
| CN111786683A (en) | Low-complexity polar code multi-code block decoder | |
| CN106998242A (en) | The unequal loss protection erasure code method of space communication distributed dynamic network topology | |
| CN102325009A (en) | Forward error correction-based method for reliably transmitting network coding multicast data stream | |
| Zorgui et al. | Centralized multi-node repair in distributed storage | |
| Li et al. | Architectures for coded mobile edge computing | |
| CN110781025B (en) | Symmetrical partial repetition code construction and fault node repairing method based on complete graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20170308 Address after: No. 163 Qixia Xianlin Avenue District of Nanjing City, Jiangsu province 210023 Applicant after: Nanjing University Applicant after: State Grid Corporation of China Applicant after: State Grid Shanghai Municipal Electric Power Company Applicant after: Nanjing Nari Co., Ltd. Address before: No. 163 Nanjing City, Jiangsu province 210023 Xianlin Avenue Applicant before: Nanjing University |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181130 Termination date: 20200512 |