CN105933386B - A kind of storage system construction method and device - Google Patents
A kind of storage system construction method and device Download PDFInfo
- Publication number
- CN105933386B CN105933386B CN201610211840.1A CN201610211840A CN105933386B CN 105933386 B CN105933386 B CN 105933386B CN 201610211840 A CN201610211840 A CN 201610211840A CN 105933386 B CN105933386 B CN 105933386B
- Authority
- CN
- China
- Prior art keywords
- node
- check
- storage system
- vector
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of storage system construction method and device, which has that node is dynamically adapted and the characteristic of high disaster tolerance, scheme include：Determine storage nodes parameter；Information node and check-node are numbered in order；Respectively for each check-node in each check-node, { 0,1 } random vector is generated；And encoding relation is established with information node for each check-node in each check-node respectively, including：Determine that check-node corresponds to each element that random vector intermediate value is 1；Determine each element for being respectively 1 with value corresponding each information node in position in the random vector；Corresponding each information node is successively subjected to XOR operation, as a result the data as updated check-node.The cost of operation is reduced when constructing storage system using scheme provided in an embodiment of the present invention, improves computational efficiency, and consume with low redundancy and realize high reliability, and there is brilliant node dynamic adjustment capability.
Description
Technical field
The present invention relates to the coding tissue and technical field of memory of electronic information file more particularly to a kind of storage system structures
Construction method and device.
Background technique
In recent years, with the fast development of internet, cloud computing, mobile terminal and Internet of Things, global metadata amount is 2010
The ZB epoch are entered year, and annual also in sustainable growth.The data-centered information age is marched toward in the whole world already, and how
Efficiently, preservation data reliably, safe have become the foundation stone of information age, while also becoming the matter of utmost importance of field of storage.
Currently, distributed memory system gradually replaces traditional single-point to deposit just using internet as the novel memory module of medium
Storage mode.It can be with inexpensive architecture system, and obtains highly reliable mass data storage effect, has become each " big data "
Company's preferred option.
Current distributed memory system still mainly guarantees data reliability to replicate disaster recovery method.It has storage operation letter
Singly, the advantages that read operation is quick；But data redudancy is high, especially when disaster tolerance parameter or data storage capacity are continuously increased, is based on
The storage system cost of replication strategy will be continuously increased.
Increasingly apparent for the defect of replication strategy High redundancy, distributed memory system disaster recovery method just gradually switchs to adopt
With based on Reed Solomon (RS) correcting and eleting codes strategy.The need of RS code any disaster tolerance parameter since itself encoding characteristics can be realized
It asks, and amount of redundant data can be made far below replication strategy.
However, the calculating of RS code is built upon in finite field, operation cost is big, limits its popularization.
Summary of the invention
The embodiment of the present invention provides a kind of storage system construction method and device, to solve existing in the prior art deposit
The cost that operation is carried out between storage system interior joint is big, the low problem of computational efficiency.
The embodiment of the present invention provides a kind of storage system construction method, and the storage system includes information node and verification section
Point, for storing data, the check-node is for restoring data, and each node in the storage system for the information node
Memory capacity is identical, the method includes：
Determine storage nodes parameter { n, k, T }, wherein n is the node total number of the storage system, and k is described deposits
The information node number of storage system, n-k are the check-node number of the storage system, and T is that individual element value in { 0,1 } is 1
Probability；Wherein n, k are positive integer, 0 < T < 1；
K information node is numbered in order as { D1,D2,…,Dk, and by n-k check-node number in order for
{P1,P2,…,Pn-k, wherein the primary data of n-k check-node is zero；
Respectively for each check-node in n-k check-node, { 0,1 } random vector g that length is 1 × k is generatedj
=(aj1,aj2,…,ajk), 1≤j≤n-k, a ∈ { 0,1 }, gjFor check-node PjCorresponding random vector, wherein each in vector
Element ajiWith probability T value for 1；
Respectively for each check-node in n-k check-node, following steps are executed：
Determine check-node PjCorresponding vector gjThe each element a that intermediate value is 1ji；
Determine each element a for being respectively 1 with valuejiIn vector gjThe corresponding each information node D in middle positioni；
By check-node PjWith corresponding each information node DiXOR operation is successively carried out, as a result as updated
Pj。
Further, further include：
In i-th of information node D of the storage systemiStoring data change Δ DiAfterwards, it is verified for n-k
Each check-node P in nodejIf check-node PjCorresponding vector gjI-th of element ajiValue be 1, then by the verification
Node PjWith the variation delta D of the information nodeiXOR operation is carried out, updated P is as a result used asj, otherwise, keep verification section
Point PjIt is constant.
Further, further include：
C1, when needing to add the identical l information node { D of memory capacity within the storage systemk+1,Dk+2,…,
Dk+lWhen, respectively for each check-node in n-k check-node, generate { 0,1 } random vector h that length is 1 × lj
=(bj1,bj2,…,bjl), 1≤j≤n-k, b ∈ { 0,1 }, hjFor check-node PjCorresponding random vector, wherein each in vector
Element bjiWith probability T value for 1；
C2 executes following steps respectively for each check-node in n-k check-node：
C21 determines check-node PjCorresponding vector hjThe each element b that intermediate value is 1ji；
C22 determines each element b for being respectively 1 with valuejiIn vector hjThe corresponding each information node D in middle positionk+i；
C23, by check-node PjWith corresponding each information node Dk+iXOR operation is successively carried out, as a result as more
P after newj；
C3, by l information node { Dk+1,Dk+2,…,Dk+lBe added in the storage system, and respectively will at random to
Measure hjCorrespondence is added to random vector gjTail portion, obtain updated random vector gj。
Further, further include before in the step C2：
It determines l information node { D to be addedk+1,Dk+2,…,Dk+lWhether not stored data, if it is not, then
Execute the step C2；And
If it is, directly by l information node { Dk+1,Dk+2,…,Dk+lBe added in the storage system, Yi Jifen
Not by random vector hjCorrespondence is added to random vector gjTail portion, obtain updated random vector gj。
Further, further include：
When needing to delete l information node { C within the storage system1,C2,…,ClWhen, determine l letter to be deleted
Cease node { C1,C2,…,ClNumber in information node set；
Respectively for each check-node in n-k check-node, { 0,1 } vector f that length is 1 × l is generatedj=
(cj1,cj2,…,cjl), 1≤j≤n-k, fjFor check-node PjCorresponding vector, wherein cjiFor information node CiIn information section
Number in point set is in gjThe element of middle corresponding position；
Respectively for each check-node in n-k check-node, following steps are executed：
Determine check-node PjCorresponding vector fjThe each element c that intermediate value is 1ji；
Determine each element c for being respectively 1 with valuejiIn vector fjThe corresponding each information node C in middle positioni；
By check-node PjWith corresponding each information node CiXOR operation is successively carried out, as a result as updated
Pj；
By l information node { C1,C2,…,ClDeleted from the storage system, and respectively from random vector gjIn
Delete each information node { C1,C2,…,ClNumber corresponding position element, obtain updated random vector gj。
Further, in each check-node being directed in n-k check-node respectively, generate length for 1 × l 0,
1 } vector fj=(cj1,cj2,…,cjl) before, further include：
The l information node { C that determination will delete1,C2,…,ClWhether not stored data, if it is not, then execute
Respectively for each check-node in n-k check-node, { 0,1 } vector f that length is 1 × l is generatedj=(cj1,cj2,…,
cjl) the step for；And
If it is, directly by l information node { C1,C2,…,ClDeleted from the storage system, and respectively from
Random vector gjIt is middle to delete each information node { C1,C2,…,ClNumber corresponding position element, obtain it is updated at random to
Measure gj。
Further, further include：
When needing to add the identical l check-node { P of memory capacity within the storage systemn-k+1,Pn-k+2,…,
Pn-k+lWhen, respectively for each check-node in l check-node, generate { 0,1 } random vector that length is 1 × k
gn-k+j=(an-k+j,1,an-k+j,2,…,an-k+j,k), 1≤j≤l, a ∈ { 0,1 }, gn-k+jFor check-node Pn-k+jIt is corresponding random
Vector, wherein each element a in vectorn-k+j,iWith probability T value for 1；
Respectively for each check-node in l check-node, following steps are executed：
Determine check-node Pn-k+jCorresponding vector gn-k+jThe each element a that intermediate value is 1n-k+j,i；
Determine each element a for being respectively 1 with valuen-k+j,iIn vector gn-k+jThe corresponding each information node in middle position
Di；
By check-node Pn-k+jWith corresponding each information node DiXOR operation is successively carried out, is as a result used as after updating
Pn-k+j。
Further, further include：
It determines check-node to be deleted, the check-node to be deleted is deleted from the storage system.
Further, the storage system also has disaster tolerance parameter { t, δ }, wherein t+ δ is the verification of the storage system
Number of nodes, the storage system allow the complete error of at most t node, and δ is volume needed for the storage system guarantees high redundancy ability
Acromere points, and t+ δ < k, t+ δ+k=n, the method further include：
When the storing data of l information node of the storage system is lost, from the storage system remaining n-l
K+ δ node is arbitrarily chosen in node, and including a check-node of δ 'A information node of k+ δ-δ ', l
≤t；
{ 0, the 1 } random vector of the corresponding 1 × k of a check-node of δ ' is taken out, δ ' × k { 0,1 } square is arranged as
Battle array Rδ'×k, wherein matrix jth V&V of behavior node{ 0,1 } random vector of corresponding 1 × k, 1≤j≤δ '；
Structural matrix Hδ'×(k+δ'), matrix Hδ'×(k+δ')Left part is divided into matrix Rδ'×k, right half is unit battle array Iδ'×δ'；
Based on a information node of known k+ δ-δ ' and a check-node of δ ', vector β is constructed(k+δ')×1, wherein β(k+δ')×1It is left
Playing k is information node { D1,D2,…,Dk, wherein unknown information node is replaced with variable X, right end is
For relational expression Hδ'×(k+δ')·β(k+δ')×1=0 is solved, the storage number for the l information node lost
According to；
By the data recovery storage of l obtained information node into the storage system.
The embodiment of the present invention also provides a kind of storage system construction device, which is characterized in that the storage system includes letter
Node and check-node are ceased, for storing data, the check-node is for restoring data, and the storage for the information node
The memory capacity of each node is identical in system, and described device includes：
Parameter determination unit, for determining storage nodes parameter { n, k, T }, wherein n is the section of the storage system
Point sum, k are the information node number of the storage system, and T is the individual element probability that value is 1 in { 0,1 }；Wherein n, k
It is positive integer, 0 < T < 1；
Numbered cell, for numbering in order k information node for { D1,D2,…,Dk, and n-k verification is saved
Point numbers in order as { P1,P2,…,Pn-k, wherein the primary data of n-k check-node is zero；
Vector generation unit, each check-node for being directed in n-k check-node respectively, generation length is 1 × k
{ 0,1 } random vector gj=(aj1,aj2,…,ajk), 1≤j≤n-k, a ∈ { 0,1 }, gjFor check-node PjIt is corresponding random
Vector, wherein each element a in vectorjiWith probability T value for 1；
Encoding relation establishes unit, and each check-node for being directed in n-k check-node respectively executes following step
Suddenly：
Determine check-node PjCorresponding vector gjThe each element a that intermediate value is 1ji；
Determine each element a for being respectively 1 with valuejiIn vector gjThe corresponding each information node D in middle positioni；
By check-node PjWith corresponding each information node DiXOR operation is successively carried out, as a result as updated
Pj。
Beneficial effect of the present invention includes：
Using above-mentioned storage system construction method provided in an embodiment of the present invention, during constructing storage system, only
It needs to carry out simple XOR operation in system between each node, to reduce the cost of operation, improves calculating effect
Rate.
Other features and advantage will illustrate in the following description, also, partly become from specification
It obtains it is clear that being understood and implementing the application.The purpose of the application and other advantages can be by written explanations
Specifically noted structure is achieved and obtained in book, claims and attached drawing.
Detailed description of the invention
Attached drawing is used to provide further understanding of the present invention, and constitutes part of specification, is implemented with the present invention
Example is used to explain the present invention together, is not construed as limiting the invention.In the accompanying drawings：
Fig. 1 is the architecture diagram of storage system provided in an embodiment of the present invention；
Fig. 2 is the flow chart of storage system construction method provided in an embodiment of the present invention；
Fig. 3 is the schematic diagram that storage system is constructed in the embodiment of the present invention；
Fig. 4 is the flow chart provided in an embodiment of the present invention for adding information node within the storage system；
Fig. 5 is the schematic diagram for adding information node in the embodiment of the present invention within the storage system；
Fig. 6 is the flow chart provided in an embodiment of the present invention that information node is deleted from storage system；
Fig. 7 is the schematic diagram for deleting information node in the embodiment of the present invention from storage system；
Fig. 8 is the flow chart provided in an embodiment of the present invention for adding check-node within the storage system；
Fig. 9 is the schematic diagram for adding check-node in the embodiment of the present invention from storage system；
Figure 10 is the flow chart for restoring the storing data lost in storage system in the embodiment of the present invention；
Figure 11 is the structural schematic diagram of storage system construction device provided in an embodiment of the present invention.
Specific embodiment
In order to provide the operation cost reduced in storage system between each node, and the realization side of raising computational efficiency
Case, the embodiment of the invention provides a kind of storage system construction method and devices, below in conjunction with Figure of description to of the invention
Preferred embodiment is illustrated, it should be understood that preferred embodiments described herein are only used to illustrate and explain the present invention, not
For limiting the present invention.And in the absence of conflict, the features in the embodiments and the embodiments of the present application can be mutual
Combination.
The embodiment of the present invention provides a kind of storage system construction method, as shown in Figure 1, the constructed storage system includes
Information node and check-node, for storing data, check-node respectively saves in storage system information node for restoring data
The memory capacity of point is identical.
Also, to storage medium or storage tool, there is no restriction in application process for the storage system, the letter of storing data
Breath node can be sector, USB flash disk, hard disk or even be independent rack etc..
Further, each node can be connected by large-scale internet in the storage system, also can be by small-scale
LAN connection, connect the network type of each node there is no limit.Also, each node can be placed in differently in storage system
Domain also can be the different location in same rack, also without limitation.
As shown in Fig. 2, storage system construction method provided in an embodiment of the present invention, may include steps of：
Step 21 determines storage nodes parameter { n, k, T }, wherein n is the node total number of storage system, and k is storage
The information node number of system, n-k are the check-node number of storage system, and T is the individual element probability that value is 1 in { 0,1 }；
Wherein n, k are positive integer, 0 < T < 1.
Step 22 numbers in order k information node for { D1,D2,…,Dk, and by n-k check-node by suitable
Sequence number is { P1,P2,…,Pn-k, wherein the primary data of n-k check-node is zero.
Step 23, respectively for each check-node in n-k check-node, generate length for 1 × k { 0,1 } with
Machine vector gj=(aj1,aj2,…,ajk), 1≤j≤n-k, a ∈ { 0,1 }, gjFor check-node PjCorresponding random vector, wherein
Each element a in vectorjiWith probability T value for 1.
Step 24, each check-node being directed in n-k check-node respectively, establish encoding relation with information node,
Following steps can specifically be executed：
Step A1, check-node P is determinedjCorresponding vector gjThe each element a that intermediate value is 1ji；
Step B1, each element a for being respectively 1 with value is determinedjiIn vector gjThe corresponding each information node in middle position
Di；
Step C1, by check-node PjEach information node D corresponding with thisiXOR operation is successively carried out, as a result as update
P afterwardsj。
Specifically, for the framework of the storage system shown in Fig. 3, to above-mentioned storage system construction method shown in Fig. 2 into
Row detailed description.
Where it is assumed that storage system parameter index is { n=10, k=6 }, i.e., wherein share 6 information nodes and 4 schools
Test node.
According to above-mentioned steps 22,6 information nodes are numbered in order as { D1,D2,D3,D4,D5,D6, and by 4 schools
Node is tested to number in order as { P1,P2,P3,P4, wherein the primary data of 4 check-nodes is zero.
According to above-mentioned steps 23, respectively for each check-node in 4 check-nodes, generating length is 1 × 6
{ 0,1 } random vector gj=(aj1,aj2,…,aj6), 1≤j≤4, a ∈ { 0,1 }, gjFor check-node PjCorresponding random vector,
Wherein, each element a in vectorjiWith probability T value for 1.
For example, specifically, as shown in Figure 3, g1=(1,0,1,1,1,0), g2=(0,1,1,0,0,0), g3=(0,1,
0,0,1,0), g4=(1,1,0,0,1,1).
Then it according to above-mentioned steps 24, respectively for each check-node in 4 check-nodes, is established with information node
Encoding relation, specifically, below with check-node P1For be described：
According to step A1, check-node P1Corresponding random vector g1=(1,0,1,1,1,0), each element that intermediate value is 1
Including the 1st, 3,4,5 element；
According to step B1, each element for being respectively 1 with value is in vector g1The corresponding each information node in middle position includes：
D1、D3、D4And D5；
According to step C1, by check-node P1Each information node D corresponding with this1、D3、D4、D5XOR operation is successively carried out,
I.e.Since the primary data of each check-node is zero, so, check-node P1Most
Whole operation result can also be expressed asAs a result it is used as updated P1, so as to complete P is directed to1
Encoding relation foundation.
In the embodiment of the present invention, the XOR operation between node can be the binary system in two nodes in same memory location
The nodulo-2 addition of bit symbol.
It completes to be directed to P using identical method2、P3And P4Encoding relation foundation, to complete the structure of the storage system
It builds.
After the building for completing the storage system, as i-th of information node D of the storage systemiStoring data hair
Raw changes delta DiAfterwards, for each check-node P in n-k check-nodejIf check-node PjCorresponding vector gjI-th
A element ajiValue be 1, then can be by check-node PjWith information node variation delta DiXOR operation is carried out, i.e.,As a result it is used as updated Pj, to complete the update of nodes encoding relationship, otherwise, keep check-node Pj
It is constant.
For example, storage system shown in Fig. 3 is directed to, if data are stored in information node D2, then it is directed to each check-node { P1,
P2,P3,P4Corresponding random vector, the random vector that second element value is 1 includes g2、g3And g4, then correspondingly, being directed to school
Test node P2、P3And P4It is required to be updated, specific renewal process is detailed in above content, is no longer described in detail herein.
Using above-mentioned storage system construction method provided in an embodiment of the present invention, during constructing storage system, only
It needs to carry out simple XOR operation in system between each node, to reduce the cost of operation, improves calculating effect
Rate.
It, can also be according to actual needs to information therein for the above-mentioned storage system of building in the embodiment of the present invention
Number of nodes is adjusted, including addition information node and deletion information node, is described in detail with reference to the accompanying drawing.
When needing to add information node within the storage system, as shown in figure 4, may include steps of：
Step 41 determines the identical l information node { D of memory capacity for needing to add within the storage systemk+1,
Dk+2,…,Dk+l}。
Step 42, respectively for each check-node in n-k check-node, generate length for 1 × l { 0,1 } with
Machine vector hj=(bj1,bj2,…,bjl), 1≤j≤n-k, b ∈ { 0,1 }, hjFor check-node PjCorresponding random vector, wherein
Each element b in vectorjiWith probability T value for 1.
Step 43, each check-node being directed in n-k check-node respectively, establish coding with new information node and close
System, can specifically execute following steps：
Step A2, check-node P is determinedjCorresponding vector hjThe each element b that intermediate value is 1ji；
Step B2, each element b for being respectively 1 with value is determinedjiIn vector hjThe corresponding each information node in middle position
Dk+i；
Step C2, by check-node PjEach information node D corresponding with thisk+iXOR operation is successively carried out, as a result as more
P after newj。
Step 44, by l information node { Dk+1,Dk+2,…,Dk+lBe added in the storage system, and respectively will be random
Vector hjCorrespondence is added to random vector gjTail portion, obtain updated random vector gj。
Specifically, for the framework of the still storage system shown in Fig. 3, shown in Fig. 4 add within the storage system to above-mentioned
The method of information node is added to be described in detail.
On the basis of storage system shown in Fig. 3, need to add 3 information node { D7,D8,D9}。
According to above-mentioned steps 42, respectively for each check-node in 4 check-nodes, generating length is 1 × 3
{ 0,1 } random vector hj=(bj1,bj2,bj3), 1≤j≤4, b ∈ { 0,1 }, hjFor check-node PjCorresponding random vector,
In, each element b in vectorjiWith probability T value for 1.
For example, specifically, as shown in Figure 5, h1=(1,0,0), h2=(0,1,1), h3=(0,1,0), h4=(1,1,
1)。
According to above-mentioned steps 43, respectively for each check-node in 4 check-nodes, established with new information node
Encoding relation, specifically, below with check-node P1For be described：
According to step A2, check-node P1Corresponding random vector h1=(1,0,0), each element that intermediate value is 1 include the
1 element；
According to step B2, each element for being respectively 1 with value is in vector h1The corresponding each information node in middle position includes：
D7；
According to step C2, by check-node P1Information node D corresponding with this7XOR operation is carried out, i.e.,
Based in process shown in above-mentioned Fig. 2In this step, check-node P1Final operation result
It can be expressed asAs a result it is used as updated P1, so as to complete P is directed to1Coding close
The update of system.
Then according to above-mentioned steps 44, as shown in figure 5, by 3 information node { D7,D8,D9It is added to the storage system
In, and respectively by random vector hjCorrespondence is added to random vector gjTail portion, obtain updated random vector gj, i.e., more
It is respectively after new：g1=(1,0,1,1,1,0,1,0,0), g2=(0,1,1,0,0,0,0,1,1), g3=(0,1,0,0,1,0,0,
1,0), g4=(1,1,0,0,1,1,1,1,1).
In the embodiment of the present invention, when adding information node into storage system, if by l information node to be added
{Dk+1,Dk+2,…,Dk+lNot stored data, do not influence the volume between original node of storage system due to information node to be added
Code relationship, it is possible to above-mentioned steps 43 are not executed, directly by l information node { Dk+1,Dk+2,…,Dk+lIt is added to storage system
In system, and respectively by random vector hjCorrespondence is added to random vector gjTail portion, obtain updated random vector gj。
When needing to delete information node from storage system, as shown in fig. 6, may include steps of：
Step 61, when need from storage system delete l information node { C1,C2,…,ClWhen, determine l to be deleted
A information node { C1,C2,…,ClNumber in information node set.
Step 62, respectively for each check-node in n-k check-node, generate length for 1 × l { 0,1 } to
Measure fj=(cj1,cj2,…,cjl), 1≤j≤n-k, fjFor check-node PjCorresponding vector, wherein cjiFor information node Ci?
Number in information node set is in gjThe element of middle corresponding position.
Step 63, each check-node being directed in n-k check-node respectively, update and close with the coding of information node
System, can specifically execute following steps：
Step A3, check-node P is determinedjCorresponding vector fjThe each element c that intermediate value is 1ji；
Step B3, each element c for being respectively 1 with value is determinedjiIn vector fjThe corresponding each information node in middle position
Ci；
Step C3, by check-node PjEach information node C corresponding with thisiXOR operation is successively carried out, as a result as update
P afterwardsj。
Step 64, by l information node { C1,C2,…,ClDeleted from storage system, and respectively from random vector gj
It is middle to delete each information node { C1,C2,…,ClNumber corresponding position element, obtain updated random vector gj。
Specifically, for the framework of the still storage system shown in Fig. 3, shown in fig. 6 deleted from storage system to above-mentioned
Except the method for information node is described in detail.
On the basis of storage system shown in Fig. 3, need to delete 2 information node { D1,D4, correspond to { C1,C2}。
According to above-mentioned steps 62, respectively for each check-node in 4 check-nodes, generating length is 1 × 2
{ 0,1 } vector fj=(cj1,cj2), 1≤j≤4, fjFor check-node PjCorresponding vector, wherein cjiFor information node CiBelieving
The number in node set is ceased in gjThe element of middle corresponding position.
For example, specifically, as shown in Figure 7, { D1,D4Number in information node set is respectively 1 and 4, then distinguish
From check-node { P1,P2,P3,P41 × 6 { 0,1 } random vector g1、g2、g3And g4In extract the 1st, 4 bits element, obtain
f1=(1,1), f2=(0,0), f3=(0,0), f4=(1,0).
According to above-mentioned steps 63, respectively for each check-node in 4 check-nodes, the volume with information node is updated
Code relationship, specifically, below with check-node P1For be described：
According to step A3, check-node P1Corresponding vector f1=(1,1), each element that intermediate value is 1 include the 1st, 2
Element；
According to step B3, each element c for being respectively 1 with valuejiIn vector fjThe corresponding each information node packet in middle position
It includes：{C1,C2, correspond to { D1,D4}；
According to step C3, by check-node P1Each information node { C corresponding with this1,C2Successively carry out XOR operation, i.e., and believe
Cease node { D1,D4Successively carry out XOR operation, i.e.,Based in process shown in above-mentioned Fig. 2
In this step, check-node P1Final operation result can also be expressed as
As a result it is used as updated P1, so as to complete P is directed to1Encoding relation update.
Then according to above-mentioned steps 64, as shown in fig. 7, by 2 information node { C1,C2, i.e. { D1,D4From storage system
Middle deletion, and respectively from random vector gjIt is middle to delete each information node { C1,C2Number in information node set corresponds to position
The element set obtains updated random vector gj, that is, it is respectively after updating：g1=(0,1,1,0), g2=(1,1,0,0), g3
=(1,0,1,0), g4=(1,0,1,1).
In the embodiment of the present invention, when deleting information node from storage system, if the l information node that will be deleted
{C1,C2,…,ClNot stored data, the coding not influenced between original node of storage system due to information node to be deleted closed
System, it is possible to above-mentioned steps 62 and step 63 are not executed, directly by l information node { C1,C2,…,ClFrom storage system
It deletes, and respectively from random vector gjIt is middle to delete each information node { C1,C2,…,ClNumber pair in information node set
The element for answering position obtains updated random vector gj。
Currently, in the prior art, there is part distributed memory system application RAID (Redundant Arrays of
Independent Disks, disk array) coding techniques, it is able to achieve the direct xor operation of each storing data, so that redundancy
Data volume is far below replication strategy.However, system parameter is very limited, is not appropriate for data since RAID coding techniques limits
The ever-expanding trend of storage size.
On the other hand, not only storage data quantity radix is big for big data era, it also shows explosive growth situation simultaneously.
However, the existing distributed storage method based on coding mode, system parameter adjustability is poor, once it is determined that, it is difficult to it changes；Data
Storage mode is complicated, by storage mode after first piecemeal re-encoding so that storage system Expansion is poor, lead to not to adapt to
The dynamic rapid growth of storing data.
And storage system based on the embodiment of the present invention, believed using above-mentioned shown in Fig. 4 added into storage system
The process and the above-mentioned process shown in fig. 6 that information node is deleted from storage system for ceasing node, can be according to practical need
It asks, flexibly quickly storage system is extended, to preferably adapt to the dynamic rapid growth of data to be stored.
It, can also be according to actual needs to verification therein for the above-mentioned storage system of building in the embodiment of the present invention
Number of nodes is adjusted, including addition check-node and deletion check-node, to realize the redundancy ability to storage system
Adjustment, be described in detail with reference to the accompanying drawing.
When needing to add check-node within the storage system, as shown in figure 8, may include steps of：
Step 81, determination need to add the identical l check-node { P of memory capacity within the storage systemn-k+1,
Pn-k+2,…,Pn-k+l}。
It is random to generate { 0,1 } that length is 1 × k for step 82, each check-node being directed in l check-node respectively
Vector gn-k+j=(an-k+j,1,an-k+j,2,…,an-k+j,k), 1≤j≤l, a ∈ { 0,1 }, gn-k+jFor check-node Pn-k+jIt is corresponding
Random vector, wherein each element a in vectorn-k+j,iWith probability T value for 1.
Step 83, each check-node being directed in l check-node respectively, establish encoding relation with information node, have
Body can execute following steps：
Step A4, check-node P is determinedn-k+jCorresponding vector gn-k+jThe each element a that intermediate value is 1n-k+j,i；
Step B4, each element a for being respectively 1 with value is determinedn-k+j,iIn vector gn-k+jThe corresponding each letter in middle position
Cease node Di；
Step C4, by check-node Pn-k+jEach information node D corresponding with thisiXOR operation is successively carried out, as a result conduct
Updated Pn-k+j。
Specifically, storage system will increase 2 check-node { P newly on the basis of storage system shown in Fig. 35,P6}。
According to above-mentioned steps 82, respectively for each check-node in 2 check-nodes, generating length is 1 × 6
{ 0,1 } random vector gj=(aj1,aj2,…,aj6), 5≤j≤6, a ∈ { 0,1 }, gjFor check-node PjCorresponding random vector,
Wherein, each element a in vectorjiWith probability T value for 1.
For example, specifically, as shown in Figure 9, g5=(0,1,0,1,0,1), g6=(1,1,0,0,0,1).
Then it according to above-mentioned steps 83, respectively for each check-node in 2 check-nodes, is established with information node
Encoding relation, specifically, below with check-node P5For be described：
According to step A4, check-node P5Corresponding random vector g5=(0,1,0,1,0,1), each element that intermediate value is 1
Including the 2nd, 4,6 element；
According to step B4, each element for being respectively 1 with value is in vector g5The corresponding each information node in middle position includes：
D2、D4And D6；
According to step C4, by check-node P5Each information node D corresponding with this2、D4、D6XOR operation is successively carried out, i.e.,Since the primary data of 2 newly-increased check-nodes is zero, so, check-node P5Most
Whole operation result can also be expressed asAs a result it is used as updated P5, so as to complete P is directed to5Volume
The foundation of code relationship.
It is directed to P in the same way6Encoding relation is established, result is：
In the embodiment of the present invention, when needing to delete check-node, check-node to be deleted is determined, be then from storage
The check-node to be deleted is deleted in system, no longer citing is described herein.
In the embodiment of the present invention, above-mentioned storage system can also have disaster tolerance parameter { t, δ }, wherein t+ δ is storage system
Check-node number, storage system allows the complete error of at most t node, and δ is that storage system guarantees additional needed for high redundancy ability
Number of nodes, and t+ δ < k, t+ δ+k=n.
Based on above-mentioned disaster tolerance parameter { t, δ }, when the quantity for the information node that storing data is lost in storage system is not more than t
When, the storing data of loss can be recovered, with reference to the accompanying drawing, the process flow for restoring to lose data is retouched in detail
It states, as shown in Figure 10, can specifically include following steps：
Step 101, when l information node of storage system storing data lose when, determine lose l information node
Number in information node set, l≤t.
Step 102 arbitrarily chooses k+ δ node from the remaining n-l node of storage system, and including a school δ '
Test nodeA information node of k+ δ-δ '.
Step 103 takes out { 0, the 1 } random vector of the corresponding 1 × k of a check-node of δ ', is arranged as δ ' × k
{ 0,1 } matrix Rδ'×k, wherein matrix jth V&V of behavior node{ 0,1 } random vector of corresponding 1 × k, 1≤j≤δ '.
Step 104, structural matrix Hδ'×(k+δ'), matrix Hδ'×(k+δ')Left part is divided into matrix Rδ'×k, right half is unit battle array
Iδ'×δ'。
Step 105 is based on a information node of known k+ δ-δ ' and a check-node of δ ', constructs vector β(k+δ')×1, wherein
β(k+δ')×1K are information node { D from left to right1,D2,…,Dk, wherein unknown information node is replaced with variable X, right end is
Step 106 is directed to relational expression Hδ'×(k+δ')·β(k+δ')×1=0 is solved, the l information node lost
Storing data.
Step 107, by the data recovery storage of l obtained information node into the storage system.
Specifically, on the basis of storage system shown in Fig. 3, it is known that each parameter of current storage system is { n=10, k=
6, δ=1, t=3 }, wherein there are k=6 information node ,+t=4 check-nodes of δ, it is ensured that any t=3 node damage,
δ=1 is the necessary redundant node number for guaranteeing high probability and restoring.
Assuming that information node { D1,D5Storage loss of data or damage, according to step 102, from remaining 8 nodes
7 nodes are taken out, for example, being respectively { D2,D3,D4,P1,P2,P3,P4}。
According to step 103, by 4 check-node { P1,P2,P3,P4{ 0,1 } random vector take out, structural matrixWherein matrix jth V&V of behavior node{ 0,1 } of corresponding 1 × k is random
Vector, 1≤j≤4.
According to step 104, structural matrix
According to step 105, it is based on known 3 information nodes and 4 check-node { D2,D3,D4,P1,P2,P3,P4, structure
Make vector β(k+δ')×1=β10×1={ X1,D2,D3,D4,X2,X3,P1,P2,P3,P4, wherein { X1,X2Correspond to { the D lost1,
D5, X3Corresponding D6；Although D6Storing data is not lost, but D6Not selected to carry out recovery operation, value is still unknown.
According to step 106, H4×10With β10×1Meet relationship H4×10)·β10×1=0, it is solved for the relationship, both may be used
With the 2 information node { D lost1,D5Storing data, and then by obtain 2 information node { D1,D5Storage
Data are restored in storage system.
In the embodiment of the present invention, for the relational expression H in above-mentioned steps 106δ'×(k+δ')·β(k+δ')×1=0, work as T=0.5
When, there is the probability of solution to be greater than 1-2δ, so, work as T=0.5, when δ=20, can guarantee that the probability higher than 0.999999 has solution, with
δ constantly increases, and has the probability of solution also constantly to increase, so, preferably, as T=0.5, δ=20.
Also, in practical applications, when k, t value are much larger than δ, the memory space of δ redundancy check node is relatively whole
It is negligible for a storage system.
Based on the same inventive concept, the storage system construction method provided according to that above embodiment of the present invention, correspondingly, this
It invents another embodiment and additionally provides a kind of storage system construction device, structural schematic diagram is as shown in figure 11, the storage system
System includes information node and check-node, and for storing data, the check-node is used to restore data the information node, and
The memory capacity of each node is identical in the storage system, and described device includes：
Parameter determination unit 111, for determining storage nodes parameter { n, k, T }, wherein n is the storage system
Node total number, k be the storage system information node number, T is the individual element probability that value is 1 in { 0,1 }；Wherein
N, k are positive integer, 0 < T < 1；
Numbered cell 112, for numbering in order k information node for { D1,D2,…,Dk, and n-k is verified
Node numbers in order as { P1,P2,…,Pn-k, wherein the primary data of n-k check-node is zero；
Vector generation unit 113, each check-node for being directed in n-k check-node respectively, generating length is 1
{ 0,1 } random vector g of × kj=(aj1,aj2,…,ajk), 1≤j≤n-k, a ∈ { 0,1 }, gjFor check-node PjIt is corresponding with
Machine vector, wherein each element a in vectorjiWith probability T value for 1；
Encoding relation establishes unit 114, and each check-node for being directed in n-k check-node respectively executes such as
Lower step：
Determine check-node PjCorresponding vector gjThe each element a that intermediate value is 1ji；
Determine each element a for being respectively 1 with valuejiIn vector gjThe corresponding each information node D in middle positioni；
By check-node PjWith corresponding each information node DiXOR operation is successively carried out, as a result as updated
Pj。
Further, the function of above-mentioned each unit can correspond to the respective handling step in process shown in Fig. 1 to Figure 10,
This is repeated no more.
Storage system construction device provided by embodiments herein can be realized by a computer program.Art technology
Personnel are it should be appreciated that above-mentioned module division mode is only one of numerous module division modes, if being divided into it
His module or non-division module all should be in the protection scopes of the application as long as storage system construction device has above-mentioned function
Within.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of storage system construction method, which is characterized in that the storage system includes information node and check-node, described
Information node for storing data, for restoring data, and in the storage system, hold the check-node by the storage of each node
Measure it is identical, the method includes：
Determine storage nodes parameter { n, k, T }, wherein n is the node total number of the storage system, and k is storage system
The information node number of system, n-k be the storage system check-node number, T be individual element value in { 0,1 } be 1 it is general
Rate；Wherein n, k are positive integer, 0 < T < 1；
K information node is numbered in order as { D1,D2,…,Dk, and n-k check-node is numbered in order as { P1,
P2,…,Pn-k, wherein the primary data of n-k check-node is zero；
Respectively for each check-node in n-k check-node, { 0,1 } random vector g that length is 1 × k is generatedj=
(aj1,aj2,…,ajk), 1≤j≤n-k, a ∈ { 0,1 }, gjFor check-node PjCorresponding random vector, wherein each member in vector
Plain ajiWith probability T value for 1；
Respectively for each check-node in n-k check-node, following steps are executed：
Determine check-node PjCorresponding vector gjThe each element a that intermediate value is 1ji；
Determine each element a for being respectively 1 with valuejiIn vector gjThe corresponding each information node D in middle positioni；
By check-node PjWith corresponding each information node DiXOR operation is successively carried out, updated P is as a result used asj。
2. the method as described in claim 1, which is characterized in that further include：
In i-th of information node D of the storage systemiStoring data change Δ DiAfterwards, for n-k check-node
In each check-node PjIf check-node PjCorresponding vector gjI-th of element ajiValue be 1, then by the check-node
PjWith the variation delta D of the information nodeiXOR operation is carried out, updated P is as a result used asj, otherwise, keep check-node Pj
It is constant.
3. the method as described in claim 1, which is characterized in that further include：
C1, when needing to add the identical l information node { D of memory capacity within the storage systemk+1,Dk+2,…,Dk+lWhen,
Respectively for each check-node in n-k check-node, { 0,1 } random vector h that length is 1 × l is generatedj=(bj1,
bj2,…,bjl), 1≤j≤n-k, b ∈ { 0,1 }, hjFor check-node PjCorresponding random vector, wherein each element b in vectorji
With probability T value for 1；
C2 executes following steps respectively for each check-node in n-k check-node：
C21 determines check-node PjCorresponding vector hjThe each element b that intermediate value is 1ji；
C22 determines each element b for being respectively 1 with valuejiIn vector hjThe corresponding each information node D in middle positionk+i；
C23, by check-node PjWith corresponding each information node Dk+iXOR operation is successively carried out, as a result as updated
Pj；
C3, by l information node { Dk+1,Dk+2,…,Dk+lBe added in the storage system, and respectively by random vector hj
Correspondence is added to random vector gjTail portion, obtain updated random vector gj。
4. method as claimed in claim 3, which is characterized in that before the step C2, further include：
It determines l information node { D to be addedk+1,Dk+2,…,Dk+lWhether not stored data, if it is not, then execute
The step C2；And
If it is, directly by l information node { Dk+1,Dk+2,…,Dk+lBe added in the storage system, and respectively will
Random vector hjCorrespondence is added to random vector gjTail portion, obtain updated random vector gj。
5. the method as described in claim 1, which is characterized in that further include：
When needing to delete l information node { C within the storage system1,C2,…,ClWhen, determine l information section to be deleted
Point { C1,C2,…,ClNumber in information node set；
Respectively for each check-node in n-k check-node, { 0,1 } vector f that length is 1 × l is generatedj=(cj1,
cj2,…,cjl), 1≤j≤n-k, fjFor check-node PjCorresponding vector, wherein cjiFor information node CiIn information node collection
Number in conjunction is in gjThe element of middle corresponding position；
Respectively for each check-node in n-k check-node, following steps are executed：
Determine check-node PjCorresponding vector fjThe each element c that intermediate value is 1ji；
Determine each element c for being respectively 1 with valuejiIn vector fjThe corresponding each information node C in middle positioni；
By check-node PjWith corresponding each information node CiXOR operation is successively carried out, updated P is as a result used asj；
By l information node { C1,C2,…,ClDeleted from the storage system, and respectively from random vector gjIt is middle to delete respectively
Information node { C1,C2,…,ClNumber corresponding position element, obtain updated random vector gj。
6. method as claimed in claim 5, which is characterized in that in each verification section being directed in n-k check-node respectively
Point generates { 0,1 } vector f that length is 1 × lj=(cj1,cj2,…,cjl) before, further include：
The l information node { C that determination will delete1,C2,…,ClWhether not stored data, if it is not, then executing difference
For each check-node in n-k check-node, { 0,1 } vector f that length is 1 × l is generatedj=(cj1,cj2,…,cjl)
The step for；And
If it is, directly by l information node { C1,C2,…,ClDeleted from the storage system, and respectively from random
Vector gjIt is middle to delete each information node { C1,C2,…,ClNumber corresponding position element, obtain updated random vector gj。
7. the method as described in claim 1, which is characterized in that further include：
When needing to add the identical l check-node { P of memory capacity within the storage systemn-k+1,Pn-k+2,…,Pn-k+l}
When, respectively for each check-node in l check-node, generate { 0,1 } random vector g that length is 1 × kn-k+j=
(an-k+j,1,an-k+j,2,…,an-k+j,k), 1≤j≤l, a ∈ { 0,1 }, gn-k+jFor check-node Pn-k+jCorresponding random vector,
Wherein, each element a in vectorn-k+j,iWith probability T value for 1；
Respectively for each check-node in l check-node, following steps are executed：
Determine check-node Pn-k+jCorresponding vector gn-k+jThe each element a that intermediate value is 1n-k+j,i；
Determine each element a for being respectively 1 with valuen-k+j,iIn vector gn-k+jThe corresponding each information node D in middle positioni；
By check-node Pn-k+jWith corresponding each information node DiXOR operation is successively carried out, as a result as updated
Pn-k+j。
8. the method as described in claim 1, which is characterized in that further include：
It determines check-node to be deleted, the check-node to be deleted is deleted from the storage system.
9. the method as described in claim 1, which is characterized in that the storage system also has disaster tolerance parameter { t, δ }, wherein t
+ δ is the check-node number of the storage system, and the storage system allows the complete error of at most t node, and δ is storage system
Additional nodes number needed for blanket insurance demonstrate,proves high redundancy ability, and t+ δ < k, t+ δ+k=n, the method further include：
When the storing data of l information node of the storage system is lost, determine the l information node lost in information section
Number in point set, l≤t arbitrarily choose k+ δ node from the remaining n-l node of the storage system, and wherein wrap
Include a check-node of δ 'A information node of k+ δ-δ ', l≤t；
{ 0, the 1 } random vector of the corresponding 1 × k of a check-node of δ ' is taken out, δ ' × k { 0,1 } matrix is arranged as
Rδ'×k, wherein matrix jth V&V of behavior node{ 0,1 } random vector of corresponding 1 × k, 1≤j≤δ '；
Structural matrix Hδ'×(k+δ'), matrix Hδ'×(k+δ')Left part is divided into matrix Rδ'×k, right half is unit battle array Iδ'×δ'；
Based on a information node of known k+ δ-δ ' and a check-node of δ ', vector β is constructed(k+δ')×1, wherein β(k+δ')×1K from left to right
Position is information node { D1,D2,…,Dk, wherein unknown information node is replaced with variable X, right end is
For relational expression Hδ'×(k+δ')·β(k+δ')×1=0 is solved, the storing data for the l information node lost；
By the data recovery storage of l obtained information node into the storage system.
10. a kind of storage system construction device, which is characterized in that the storage system includes information node and check-node, institute
State information node for storing data, the check-node for restoring data, and in the storage system each node storage
Capacity is identical, and described device includes：
Parameter determination unit, for determining storage nodes parameter { n, k, T }, wherein n is that the node of the storage system is total
Number, k are the information node number of the storage system, and T is the individual element probability that value is 1 in { 0,1 }；Wherein n, k are
Positive integer, 0 < T < 1；
Numbered cell, for numbering in order k information node for { D1,D2,…,Dk, and n-k check-node is pressed
Serial number is { P1,P2,…,Pn-k, wherein the primary data of n-k check-node is zero；
Vector generation unit, each check-node for being directed in n-k check-node respectively, generating length is 1 × k's
{ 0,1 } random vector gj=(aj1,aj2,…,ajk), 1≤j≤n-k, a ∈ { 0,1 }, gjFor check-node PjIt is corresponding at random to
Amount, wherein each element a in vectorjiWith probability T value for 1；
Encoding relation establishes unit, and each check-node for being directed in n-k check-node respectively executes following steps：
Determine check-node PjCorresponding vector gjThe each element a that intermediate value is 1ji；
Determine each element a for being respectively 1 with valuejiIn vector gjThe corresponding each information node D in middle positioni；
By check-node PjWith corresponding each information node DiXOR operation is successively carried out, updated P is as a result used asj。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610211840.1A CN105933386B (en) | 2016-04-06 | 2016-04-06 | A kind of storage system construction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610211840.1A CN105933386B (en) | 2016-04-06 | 2016-04-06 | A kind of storage system construction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105933386A CN105933386A (en) | 2016-09-07 |
| CN105933386B true CN105933386B (en) | 2018-11-27 |
Family
ID=56840200
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610211840.1A Active CN105933386B (en) | 2016-04-06 | 2016-04-06 | A kind of storage system construction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105933386B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110278222B (en) * | 2018-03-15 | 2021-09-14 | 华为技术有限公司 | Method, system and related device for data management in distributed file storage system |
| CN109460185B (en) * | 2018-10-29 | 2022-03-18 | 锐捷网络股份有限公司 | Method and device for building block group CKG, virtual storage pool and storage medium |
| CN111435323B (en) * | 2019-01-15 | 2023-06-20 | 阿里巴巴集团控股有限公司 | Information transmission method, device, terminal, server and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102420616B (en) * | 2011-11-16 | 2013-08-14 | 西安电子科技大学 | Error correction method by using quasi-cyclic LDPC code based on Latin square |
| CN102624866B (en) * | 2012-01-13 | 2014-08-20 | 北京大学深圳研究生院 | Data storage method, data storage device and distributed network storage system |
-
2016
- 2016-04-06 CN CN201610211840.1A patent/CN105933386B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102420616B (en) * | 2011-11-16 | 2013-08-14 | 西安电子科技大学 | Error correction method by using quasi-cyclic LDPC code based on Latin square |
| CN102624866B (en) * | 2012-01-13 | 2014-08-20 | 北京大学深圳研究生院 | Data storage method, data storage device and distributed network storage system |
Non-Patent Citations (2)
| Title |
|---|
| "Efficient software implementations of large finite fields GF (2 n </sup>) for secure storage applications";Jianqiang Luo; Kevin D. Bowers; Alina Oprea; Lihao Xu;;《ACM Transactions on Storage (TOS)》;20120229;全文 * |
| "Heuristics for optimizing matrix-based erasure codes for fault-tolerant storage systems";James S. Plank; Catherine D. Schuman; B. Devin Robison;《IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012)》;20120809;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105933386A (en) | 2016-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103336785B (en) | A kind of distributed storage method based on network code and device thereof | |
| CN104052576B (en) | Data recovery method based on error correcting codes in cloud storage | |
| CN111553473B (en) | Data redundancy method and neural network processor for executing the same | |
| US10860555B2 (en) | Method and apparatus for two tier data deduplication using weighted graphs | |
| US9436722B1 (en) | Parallel checksumming of data chunks of a shared data object using a log-structured file system | |
| CN104871138B (en) | For the method and system that multidimensional RAID reconstruction and defect are avoided | |
| CN106201338A (en) | Date storage method and device | |
| CN105933386B (en) | A kind of storage system construction method and device | |
| CN101685672A (en) | Least significant bit page recovery method used in multi-level cell flash memory device | |
| CN110750382A (en) | Minimum storage regeneration code coding method and system for improving data repair performance | |
| WO2018112980A1 (en) | Storage controller, data processing chip, and data processing method | |
| CN107844272A (en) | A kind of cross-packet coding and decoding method for improving error correcting capability | |
| CN111831223B (en) | Fault-tolerant coding method, device and system for improving expandability of data deduplication system | |
| CN106484559A (en) | A kind of building method of check matrix and the building method of horizontal array correcting and eleting codes | |
| WO2016082156A1 (en) | Metadata recovery method and apparatus | |
| US20170359090A1 (en) | Method of operating decoder for reducing computational complexity and method of operating data storage device including the decoder | |
| US20100262755A1 (en) | Memory systems for computing devices and systems | |
| CN108268337A (en) | Method for managing data in memory device, memory device and controller thereof | |
| CN111078460A (en) | Fast erasure code calculation method | |
| CN108132854A (en) | A kind of correcting and eleting codes coding/decoding method that can restore data element and redundant elements simultaneously | |
| CN103838649B (en) | Method for reducing calculation amount in binary coding storage system | |
| CN105808170A (en) | RAID6 (Redundant Array of Independent Disks 6) encoding method capable of repairing single-disk error by minimum disk accessing | |
| CN107153661A (en) | A kind of storage, read method and its device of the data based on HDFS systems | |
| CN109358980A (en) | A kind of pair of data update and single disk error repairs friendly RAID6 coding method | |
| CN104461373B (en) | A kind of RAID data processing method and processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |