CN102843212B - Coding and decoding processing method and device - Google Patents
Coding and decoding processing method and device Download PDFInfo
- Publication number
- CN102843212B CN102843212B CN201210275397.6A CN201210275397A CN102843212B CN 102843212 B CN102843212 B CN 102843212B CN 201210275397 A CN201210275397 A CN 201210275397A CN 102843212 B CN102843212 B CN 102843212B
- Authority
- CN
- China
- Prior art keywords
- decoding
- dimension
- data
- encoding
- correcting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0064—Concatenated codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0078—Avoidance of errors by organising the transmitted data in a format specifically designed to deal with errors, e.g. location
- H04L1/0079—Formats for control data
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Error Detection And Correction (AREA)
Abstract
The invention provides a kind of coding and decoding processing method and device, the method includes: treats codec data and carries out higher dimensional formats process, wherein, multidimensional at least two dimension；In a predetermined sequence, the at least two dimension in the every dimension treating codec data after processing higher dimensional formatsization carries out reed-solomon RS correcting and eleting codes encoding and decoding and processes, pass through the present invention, solve and prior art correlation technique exists when allowing anti-more corrupted data, need to increase amount of calculation, and the problem affecting codec rate and performance, and then reached on the premise of not reducing original encoding rate and memory space utilization rate, system survivability is dramatically increased, improves coding rate and the effect of decoding speed simultaneously.
Description
Technical field
The present invention relates to the communications field, in particular to a kind of coding and decoding processing method and device.
Background technology
Cloud storage refers to by functions such as cluster application, grid or distributed file systems, by various differences a large amount of in network
The storage device of type gathers collaborative work by application software, common externally offer data storage and Operational Visit function
One system.In cloud computing environment, file is typically fragmented and is saved in multiple cloud storage server.When data communication,
The data needing communication also can be divided into multiple burst, sends the other side piecewise to.
When data store and during communication, in order to solve the problem of reliability, typically use Read-Solomon (Reed-Solomon,
Referred to as RS) correcting and eleting codes (Erasure Codes, referred to as EC) technology, after document No., it is divided into the m that size is identical
Individual burst and n verification burst, carry out storing or communication respectively.For file storage or communication recipient, as long as obtaining it
In arbitrarily m burst, original or data can be recovered by decoding, therefore can resist n burst damage or loss,
Substantially increase the reliability of system.Storing for computer documents, the memory space utilization rate of correcting and eleting codes system is m/(m+n),
Be significantly larger than copy storage mode, therefore RS correcting and eleting codes exchanges storage capacity for by computing capability, significantly reduce carrying cost with
O&M cost.
Nineteen sixty, Reed (I.S.Reed) and Saloman (G.Solomon) propose a kind of method constructing correcting and eleting codes, use and be somebody's turn to do
The correcting and eleting codes of method is referred to as Reed-Solomon code, is called for short RS code.Correcting and eleting codes based on RS coding techniques structure is then referred to as
RS correcting and eleting codes.One (n, k) correcting and eleting codes is that k source data is encoded to n (n > k) individual data so that appoint with in these n data
Meaning k is according to original k the source data of equal restructural.Use m burst and n verification burst correcting and eleting codes system be exactly (m+n,
M) correcting and eleting codes.
Reed Solomon code mainly comprises the coding generated based on generalized circular matrix, is Vandermonde code (Vandermond Code),
With the coding generated based on Cauchy matrix, it is Cauchy's code (Cauchy Code).Their computing is based on finite field Jia Luohua
(Galois) territory is carried out.They can arbitrarily arrange m and n value when realizing, thus obtains higher space utilisation.
But no matter generalized circular matrix and the RS correcting and eleting codes system of Cauchy matrix, has a common shortcoming, it is simply that amount of calculation is relatively big,
Coding, decoding speed are slower.According to existing open mathematical knowledge, above two RS correcting and eleting codes when encoding and decoding, amount of calculation and
Time complexity is O (m^2), and when solving the inverse matrix of generator matrix, if use Gauss-when the elimination be optimal algorithm,
Amount of calculation and time complexity are O (m^3), if employing k redundant block during decoding, then decoding algorithm operand is O (mk).
For the file of a length of L, decoding algorithm operand is O (Lk).The redundant block k of decoding speed and use is directly proportional, therefore,
In actual use, the redundant block value of use can not be the biggest.In commercial system, burst quantity m is usually no more than 10 at present, verification
Sheet n is usually no more than 6.In order to preferably use RS correcting and eleting codes system in computer communication field, general employing specialized hardware is real
Existing codec functions, improves encoding and decoding speed.
On the other hand, using the cloud storage system of civilian cheap hard disk, and in P2P dynamic memory-communication environment, it would be desirable to
In the case of not affecting encoding rate and decoding performance, anti-more corrupted data, i.e. require that verification burst n is sufficiently large, and use
RS correcting and eleting codes encoding and decoding algorithm performance do not decline.In such a mode, simple is impracticable by putting forward high n-value, can cause meter
The quick growth of calculation amount, causes hydraulic performance decline to arrive unpractical stage.
Therefore, exist in the related when allowing anti-more corrupted data, need to increase amount of calculation, and impact is compiled and solved
Bit rate and the problem of performance.
Summary of the invention
The invention provides a kind of coding and decoding processing method and device, at least to solve prior art correlation technique to exist when allowing
During anti-more corrupted data, need to increase amount of calculation, and the problem affecting codec rate and performance.
According to an aspect of the invention, it is provided a kind of coding and decoding processing method, including: treat codec data and carry out multidimensional
Formatting processes, wherein, and described multidimensional at least two dimension；In a predetermined sequence, waiting after processing higher dimensional formatsization is compiled
At least two dimension in every dimension of decoding data carries out Read-Solomon RS correcting and eleting codes encoding and decoding and processes.
Preferably, treat codec data to carry out higher dimensional formatsization and process and comprise determining that described data to be encoded are formatted
The deblocking size processed；In the case of performing coded treatment, described waiting is compiled by the described deblocking size according to determining
Code data carry out benefit and cut process；In the case of performing decoding process, data to be decoded are stored in the described deblocking determined big
The relevant position of little data block is decoded processing.
Preferably, in a predetermined sequence, in the every dimension treating codec data after higher dimensional formatsization being processed at least
Two dimension carries out described RS correcting and eleting codes encoding and decoding process and includes: in the case of performing coded treatment, goes dimension step by step according to described multidimensional
The mode of degree, the every dimension treating codec data after processing higher dimensional formatsization carries out described RS correcting and eleting codes coded treatment；
In the case of performing decoding process, adding the mode of dimension step by step according to described multidimensional, waiting after processing higher dimensional formatsization is compiled
Every dimension of decoding data carries out described RS correcting and eleting codes decoding process.
Preferably, in a predetermined sequence, in the every dimension treating codec data after higher dimensional formatsization is processed extremely
After few two dimension carries out described RS correcting and eleting codes encoding and decoding process, also include: according to the physical resource of storage server, to carrying out institute
State the data obtained after RS correcting and eleting codes processes to store；Or, send and carry out acquisition after described RS correcting and eleting codes processes
Data.
Preferably, according to the physical resource of storage server, the data carrying out obtaining after described RS correcting and eleting codes processes are deposited
Storage includes: the part verification data carried out after described RS correcting and eleting codes processes in the described data of acquisition are stored in individually storage
On node.
Preferably, in the case of described multidimensional is three-dimensional, in a predetermined sequence, at least one in the following manner to multidimensional
At least two dimension in the every dimension treating codec data after formatting process carries out described RS correcting and eleting codes encoding and decoding and processes: position
In storage server same file access client FAC to higher dimensional formatsization process after treat the most one-dimensional of codec data
Degree carries out described RS correcting and eleting codes encoding and decoding and processes；After described FAC completes the first order encoding and decoding corresponding to the first dimension, storage
The node that calculates in territory completes corresponding to the second level corresponding to the second dimension and third dimension the 3rd to the data after first order encoding and decoding
The encoding and decoding of level；After described FAC completes the first order encoding and decoding corresponding to the first dimension, the calculating node in storage territory is to first
Data after level encoding and decoding complete second level encoding and decoding corresponding to the second dimension, and memory node is to the number after the encoding and decoding of the second level
According to the third level encoding and decoding completed corresponding to third dimension.
According to a further aspect in the invention, it is provided that a kind of encoding and decoding processing means, including: the first processing module, it is used for treating
Codec data carries out higher dimensional formats process, wherein, described multidimensional at least two dimension；Second processing module, for according in advance
Fixed order, at least two dimension in the every dimension treating codec data after processing higher dimensional formatsization carries out Read-Solomon
RS correcting and eleting codes encoding and decoding process.
Preferably, described first processing module includes: first determines unit, is used for determining described data to be encoded are carried out form
The deblocking size that change processes；First processing unit, in the case of performing coded treatment, according to the described number determined
According to piecemeal size, described data to be encoded are carried out benefit and cut process；In the case of performing decoding process, data to be decoded are stored in
The relevant position of the data block of the described deblocking size determined is decoded processing.
Preferably, described second processing module includes: the second processing unit, is used in the case of performing coded treatment, according to
Described multidimensional goes the mode of dimension step by step, in the every dimension treating codec data at least two after processing higher dimensional formatsization
Dimension carries out described RS correcting and eleting codes coded treatment；3rd processing unit, in the case of performing decoding process, according to described many
Dimension adds the mode of dimension step by step, and the every dimension treating codec data after processing higher dimensional formatsization carries out described RS correcting and eleting codes
Decoding process；
Preferably, described second processing module includes at least one of: in a predetermined sequence, the most at least it
At least two dimension in the every dimension treating codec data after a pair multidimensional formatting process carries out described RS correcting and eleting codes and compiles solution
Code processes: fourth processing unit, in the case of described multidimensional is three-dimensional, by being positioned at the same literary composition of storage server
The every dimension treating codec data after higher dimensional formatsization is processed by part access client FAC carries out described RS correcting and eleting codes volume
Decoding process；5th processing unit, in the case of described multidimensional is three-dimensional, completes the first dimension institute by described FAC
After corresponding first order encoding and decoding, the data after first order encoding and decoding are completed corresponding to the second dimension by the node that calculates in storage territory
The encoding and decoding of the third level corresponding to the second level and third dimension；6th processing unit, is used in the case of described multidimensional is three-dimensional,
After completing the first order encoding and decoding corresponding to the first dimension by described FAC, the calculating node in storage territory is to first order encoding and decoding
After data complete second level encoding and decoding corresponding to the second dimension, and memory node completes to the data after the encoding and decoding of the second level
Third level encoding and decoding corresponding to three dimensionality.
By the present invention, employing is treated codec data and is carried out higher dimensional formats process, wherein, described multidimensional at least two dimension；
In a predetermined sequence, to higher dimensional formatsization process after the every dimension treating codec data at least two dimension carry out Reed-
Saloman RS correcting and eleting codes encoding and decoding process, and solve and exist when allowing anti-more corrupted data in prior art correlation technique,
Need to increase amount of calculation, and affect codec rate and performance problem, and then reached do not reducing original encoding rate
On the premise of memory space utilization rate, system survivability is dramatically increased, improves coding rate and decoding speed simultaneously
Effect.
Accompanying drawing explanation
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, and the present invention shows
Meaning property embodiment and explanation thereof are used for explaining the present invention, are not intended that inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the flow chart of coding and decoding processing method according to embodiments of the present invention；
Fig. 2 is the structured flowchart of encoding and decoding processing means according to embodiments of the present invention；
Fig. 3 is the preferred structure block diagram of the first processing module 22 in decoding processing device according to embodiments of the present invention；
Fig. 4 is the preferred structure block diagram one of the second processing module 24 in decoding processing device according to embodiments of the present invention；
Fig. 5 is the preferred structure block diagram two of the second processing module 24 in decoding processing device according to embodiments of the present invention；
Fig. 6 is the structural representation of based on RS correcting and eleting codes multistage coding/decoding system according to embodiments of the present invention.
Detailed description of the invention
Below with reference to accompanying drawing and describe the present invention in detail in conjunction with the embodiments.It should be noted that in the case of not conflicting,
Embodiment in the application and the feature in embodiment can be mutually combined.
Providing a kind of coding and decoding processing method in the present embodiment, Fig. 1 is coding and decoding processing method according to embodiments of the present invention
Flow chart, as it is shown in figure 1, this flow process comprises the steps:
Step S102, treats codec data and carries out higher dimensional formats process, wherein, this multidimensional at least two dimension；
Step S104, in a predetermined sequence, in the every dimension treating codec data after higher dimensional formatsization is processed extremely
Few two dimension carries out Read-Solomon RS correcting and eleting codes encoding and decoding and processes.
By above-mentioned steps, the data treating encoding and decoding carry out staged care, relative to directly treating encoding and decoding in correlation technique
Data process, and can utilize storage system or the computing capability of distributing communication system and storage capacity, classification easily
Process make at different levels between separate, not only solve and prior art correlation technique exist when allowing anti-more data to damage
In bad time, need to increase amount of calculation, and affect codec rate and performance problem, and then reached do not reduce original
On the premise of encoding rate and memory space utilization rate, system survivability is dramatically increased, improves coding rate simultaneously and conciliate
The effect of code speed.
When treating codec data and carrying out higher dimensional formatsization process, it is possible, firstly, to according to physical resource, determine and treat coded number
According to formatting the deblocking size of process, i.e. when follow-up encoding and decoding process, carry out in units of this deblocking size
Process；And carrying out encoding and during decoding process, performed operation is different, such as, in the feelings performing coded treatment
Under condition, according to this deblocking size determined, data to be encoded are carried out benefit and cut process, for example, if by be encoded
After data format, the deblocking size divided is a*b*c, when this size of data deficiencies to be encoded, then leads to
The operation crossing zero padding carrys out completion, and when data to be encoded are more than this size, then big by data to be encoded are cut into this
Little perform the encoding operation；In the case of performing decoding process, data to be decoded are stored in the described deblocking size determined
The relevant position of data block be decoded processing, illustrate with above-mentioned example equally: for the data to be decoded of storage,
It is a*b*c from there is reading deblocking size position, it is impossible to select the data more than this size, and big in data to be decoded
When above-mentioned size, can be with the data to be decoded of prioritizing selection data division；It addition, for the data to be decoded received, can
It is decoded processing in the relevant position of the data block that they put into the deblocking size logic as a*b*c by number.
It should be noted that in a predetermined sequence, in the every dimension treating codec data after higher dimensional formatsization is processed
At least two dimension to carry out RS correcting and eleting codes different according to the process of encoding and decoding, perform different orders, such as, performing coded treatment
In the case of, go the mode of dimension step by step according to multidimensional, the every dimension treating codec data after higher dimensional formatsization is processed
Carry out RS correcting and eleting codes coded treatment；And in the case of performing decoding process, add the mode of dimension step by step according to multidimensional, to multidimensional
The every dimension treating codec data after formatting processes carries out RS correcting and eleting codes decoding process.
Preferably, in a predetermined sequence, in the every dimension treating codec data after higher dimensional formatsization is processed extremely
After few two dimension carries out RS correcting and eleting codes encoding and decoding process, also include: according to the physical resource of storage server, to stating RS
The data that correcting and eleting codes obtains after processing store；Or, send the data carrying out obtaining after RS correcting and eleting codes process.It addition,
Part verification data in the data that can also obtain after carrying out RS correcting and eleting codes process are stored on single memory node.
For to the main body performed by encoding and decoding, can have various ways, the multidimensional with above-mentioned indication is come for three-dimensional example below
Illustrate, in the case of carrying out being normalized to three-dimensional by the data treating encoding and decoding, can in a predetermined sequence, by with
The every dimension treating codec data after higher dimensional formatsization is processed by least one under type carries out described RS correcting and eleting codes volume solution
Code process: (1) be positioned at storage server same file access client FAC to higher dimensional formatsization process after waiting compile solve
Every dimension of code data carries out described RS correcting and eleting codes encoding and decoding and processes；(2) this FAC completes first corresponding to the first dimension
After level encoding and decoding, the node that calculates in storage territory completes the second level corresponding to the second dimension and the to the data after first order encoding and decoding
The encoding and decoding of the third level corresponding to three dimensionality；(3) after FAC completes the first order encoding and decoding corresponding to the first dimension, in storage territory
The node that calculates the data after first order encoding and decoding are completed second level encoding and decoding corresponding to the second dimension, and memory node is to the
Data after two grades of encoding and decoding complete the third level encoding and decoding corresponding to third dimension.Above-mentioned various processing mode can be according to specifically
Needs select flexibly.
Additionally providing a kind of encoding and decoding processing means in the present embodiment, this device is used for realizing above-described embodiment and the side of being preferable to carry out
Formula, had carried out repeating no more of explanation.As used below, term " module " can realize the software of predetermined function
And/or the combination of hardware.Although the device described by following example preferably realizes with software, but hardware, or software
The realization of the combination with hardware also may and be contemplated.
Fig. 2 is the structured flowchart of encoding and decoding processing means according to embodiments of the present invention, as in figure 2 it is shown, this device includes first
Processing module 22 and the second processing module 24, illustrate this device below.
First processing module 22, is used for treating codec data and carries out higher dimensional formats process, wherein, this multidimensional at least two dimension；
Second processing module 24, is connected to above-mentioned first processing module 22, in a predetermined sequence, higher dimensional formats is processed it
After the every dimension treating codec data at least two dimension carry out Read-Solomon RS correcting and eleting codes encoding and decoding process.
Fig. 3 is the preferred structure block diagram of the first processing module 22 in decoding processing device according to embodiments of the present invention, such as Fig. 3 institute
Showing, this first processing module 22 includes that first determines unit 32 and the first processing unit 34, below to this first processing module 22
Illustrate.
First determines unit 32, for determining the deblocking size that data to be encoded format process；First processes list
Unit 34, is connected to above-mentioned first and determines unit 32, in the case of performing coded treatment, big according to the deblocking determined
Little data to be encoded are carried out benefit cut process；In the case of performing decoding process, data to be decoded are stored in the data determined and divide
The relevant position of the data block of block size is decoded processing.
Fig. 4 is the preferred structure block diagram one of the second processing module 24 in decoding processing device according to embodiments of the present invention, such as Fig. 4
Shown in, this second processing module 24 includes the second processing unit 42 and the 3rd processing unit 44, below to this second processing module
24 illustrate.
Second processing unit 42, for, in the case of performing coded treatment, going the mode of dimension step by step, to multidimensional according to multidimensional
At least two dimension in the every dimension treating codec data after formatting process carries out RS correcting and eleting codes coded treatment；3rd processes
Unit 44, is connected to above-mentioned second processing unit 42, for, in the case of performing decoding process, adding dimension step by step according to multidimensional
Mode, to higher dimensional formatsization process after the every dimension treating codec data carry out RS correcting and eleting codes decoding process；
Fig. 5 is the preferred structure block diagram two of the second processing module 24 in decoding processing device according to embodiments of the present invention, such as Fig. 5
Shown in, this second processing module 24 includes at least one of: at fourth processing unit the 52, the 5th processing unit the 54, the 6th
Reason unit 56, illustrates this second processing module 24 below.
Fourth processing unit, in the case of multidimensional is three-dimensional, accesses visitor by being positioned at the same file of storage server
Family end FAC higher dimensional formatsization is processed after the every dimension treating codec data at least two dimension carry out described RS and entangle and delete
Code encoding and decoding process；5th processing unit, in the case of multidimensional is three-dimensional, completes the first dimension institute by described FAC
After corresponding first order encoding and decoding, the data after first order encoding and decoding are completed corresponding to the second dimension by the node that calculates in storage territory
The encoding and decoding of the third level corresponding to the second level and third dimension；6th processing unit is in the case of multidimensional is three-dimensional, logical
After crossing the first order encoding and decoding that described FAC completes corresponding to the first dimension, after the calculating node in storage territory is to first order encoding and decoding
Data complete second level encoding and decoding corresponding to the second dimension, and memory node completes the 3rd to the data after the encoding and decoding of the second level
Third level encoding and decoding corresponding to dimension.
Provide a kind of when file storage or data communication in the present embodiment, multistage decoding method based on RS correcting and eleting codes,
The method is used to realize data or the multistage encoding and decoding of file.In this multistage decoding method based on RS correcting and eleting codes, treat
Coded data carries out higher dimensional formats, such as, two dimension or 3 dimensional format, and the most on request, every dimension is carried out by order
RS correcting and eleting codes encoding operation, composition multi-dimensional data is grouped and verifies piecemeal accordingly, more preferably, can be according to the thing of cloud storage
Reason resource situation stores, thus can part verification data be stored on single memory node.In the present embodiment with
Illustrate as a example by three grades of encoding and decoding.
Step wants S1, coding: three grades of codings in the present embodiment comprise the steps:
(1) data to be encoded are carried out 3 dimensional format
Burst quantity mi of three grades of coding parameters, and verification burst quantity ni can be determined according to the physical resource situation of cloud storage,
I=1,2,3.Wherein, m1+n1 is disk sum in single storage server, and m2+n2 is storage number of servers in Single Cabinet,
M3+n3 is the rack quantity in same storage territory.
Data to be encoded carry out 3 dimensional format, and i.e. according to the mode of m1*m2*m3, composition looks like cuboid in logic
Data block.If data deficiencies m1*m2*m3, decline passes through zero padding polishing；If file is relatively big, then according to
The piecemeal of m1*m2*m3 cuts, and is cut into multiple cuboids in logic, wherein, a length of m1, a width of m2, a height of m3.
(2) plane coding
To deblocking according to planar dimensions, the m1*m2 data of each layer are carried out RS (m3+n3, n3) coding.After coding,
The height of deblocking becomes m3+n3 from m3, i.e. creates extra n3 layer verification data.
(3) row coding
By each layer data of the data of m3+n3 layer, in column direction, each column data in m3 row is carried out RS (m2+n2,
N2) coding.After coding, the line number of deblocking every layer becomes m2+n2 from m2, i.e. creates extra n2 row verification data.
(4) row coding
By each layer data of m3+n3 layer data piecemeal, every layer has m2+n2 row, by line direction, carries out each row
RS (m1+n1, n1) encodes.After coding, data often row becomes m1+n1 data, and final data block size becomes
(m1+n1)*(m2+n2)*(m3+n3)。
For above-described three grades of codings, both can by cloud storage same file access client (File Access Client,
Referred to as FAC) carry out, it is also possible to performed the coding of the first order by FAC, then be sent to store in territory by the data after coding
Calculating node in each rack, is carried out the process of the second level and the third level by calculating node.Calculate node and can individually complete the
Two grades and the coding of the third level, it is possible to only complete the coding of the second level, then data are sent to memory node by row, then by depositing
Storage node carries out third level coding, places the data in each disk of memory node.
(5) data storage or communication
For cloud storage mode, by coded data block m3+n3 layer, each layer data is sent to each machine in same storage territory
In cabinet.(m1+n1) * (m2+n2) data received for each rack, store to by row in each storage server in rack.
For each storage server, the m1+n1 data often gone are stored in m1+n1 disk.
For data communication mode, all data blocks after coding are numbered in order, is sent to the other side one by one.
Step S2: decoding: three grades of decodings in the present embodiment comprise the steps:
(1) data to be decoded are taken
For the decoding of cloud storage, when file access client FAC needs decoding, the metadata according to data or file is believed
Breath, from m3+n3 rack, the storage server of each rack m2+m2, on the m1+n1 disk of each storage server,
Fetch data by number.For each row of data on each storage server, at least get m1 data, if can select to be more than
M1 data, the data (the most as far as possible selecting former data) of m1 numbering before prioritizing selection.For each rack, at least take m2
The partial data of row, m2*m1 data altogether, the data of m2 row before prioritizing selection；For multiple racks, at least return
Complete m1*m2 the data of m3 rack, the data of m3 rack before prioritizing selection.
For data communication mode, to all data blocks received, they are put into by number logic
(m1+n1) relevant position in the cuboid of * (m2+n2) * (m3+n3) is decoded.
(2) row decoding
The cubical data of each (m1+n1) * (m2+n2) * (m3+n3) for receiving, first carry out RS(m1+n1 by row,
M1) decoding, recovers the data often gone.If the data that this row receives are less than m1, then this row decodes unsuccessfully.
(3) row decoding
For in each plane, decoded by row, at least need to recover the partial data of any m2 row, just can carry out row decoding.
For each plane, if there being m2 row partial data, then each column data to these row, by carrying out RS(m2+n2,
M2) decoding, it is possible to obtain all of initial data in this plane.If the partial data obtained is less than m2 row, then arrange decoding
Failure, it is impossible to obtain the initial data in this plane.
(4) plane decoding
For the data of m3+n3 plane, decoded by row, at least need to recover the data of m3 plane, just can carry out
Plane decodes.If there being the data of m3 plane, then decoded by (m3+n3, m3), all initial m3 can be obtained
The data of individual plane, i.e. obtain all primary datas.
For above-described three grades of coding system examples, in cloud storage, its three grades decoding processes, both can be deposited by whole clouds
Same file access client FAC of storage is carried out, it is also possible to is calculated node by storage rack and carries out the two or three grade of decoding (at once
Decoding and row decoding), then carried out first order decoding (plane decoding) by FAC, it is also possible to entered by storage rack memory node respectively
Row third level decoding (row decoding), calculates node and carries out two grades of decodings (row decoding), and FAC carries out first order decoding (planar solution
Code).In practice, what encoding and decoding flow process specifically used, depends on actual motion environment.Such as, if having only to two
Level encoding and decoding, then have only to simply split data into m3 layer plane when FAC encodes, and second level coding is encoded by row
Complete, it is not necessary to do row coding again.
Based on RS correcting and eleting codes the multistage coding/decoding system used for the embodiment of the present invention, its encoding rate and memory space utilize
Rate P3=m1*m2*m3/ (m1+n1) (m2+n2) (m3+n3).Can accommodate n3 rack to damage, each rack accommodates n2 storage
Node damages, and it is bad that each node accommodates n1 block adjustment debit.For any disk sum S3min that can accommodate damage,
S3min=(n1+1) (n2+1) (n3+1)-1, the most at least accommodates any S3min block adjustment debit bad.At most accommodate damage dish S3max,
S3max=(m1+n1)(m2+n2)(m3+n3)-m1m2m3。
If two-stage coding/decoding system, its encoding rate and memory space utilization rate P2=m1*m2/ (m1+n1) (m2+n2), at least
Anti-S2min block adjustment debit is bad, bad to multi-resistance S2max block adjustment debit, wherein, and S2min=(n1+1) (n2+1)-1, S2max=
(m1+n1)(m2+n2)-m1m2.For k level coding/decoding system, Pk=m1*...mk/ (m1+n1) * ... (mk+nk),
Skmin=(n1+1) ... (nk+1)-1, Skmax=(m1+n1) ... (mk+nk)-m1..mk.
For a typical cloud storage system, a rack has 16 2U to store server, each storage server 12
SATA hard disc, or 8 4U storage servers, 24 SATA hard disc of each storage server, each rack separately has 4
Calculate node.One storage territory can include 8-16 rack.If using three grades of coding/decoding systems, m3=7, n3=1, m2=7,
N2=1, m1=21, n1=3, can calculate encoding rate P3=7*7*21/8*8*24=66.99%, can accommodate the most any
2*2*4-1=15 block disk failures, at most accommodates 8*8*24-7*7*21=507 adjustment debit bad.If using two-stage coding system, by 8
Individual rack 8 storage server, altogether 64 the storage server composition second level, m1=21, n1=3, m2=61, n2=3,
Then encoding rate P2=61*21/64*24=83.40%, at least can anti-4*4-1=15 block disk failures, to multi-resistance 64*24-61*21=255
Block disk failures.
For the tertiary storage system in the example above, if all using RS correcting and eleting codes, then according to existing knowledge, if three grades
Employ k1, k2, k3 block check data in decoding respectively, then file decoding amount of calculation is O (k1), O (k2), O (k3).For
In above-mentioned example, in the case of anti-at least 15 pieces of adjustment debits, decoding the amount of calculation is O (1)+O (1)+O (3), and single-stage RS is compiled
Under decoding situation, decoding calculating quantity O (5) is suitable.Under the above-mentioned example case using two grades of RS encoding and decoding, at anti-at least 15 pieces
In the case of adjustment debit, decoding the amount of calculation is that under O (3)+O (3), and single-stage RS encoding and decoding situation, decoding calculating quantity O (6) is suitable.
Therefore, use multilevel memory system, decoding speed can be greatly improved.In turn, in single-stage RS correcting and eleting codes coding/decoding system,
If supporting any 15 pieces of adjustment debits, meeting the encoding rate of 83.40% simultaneously, needing to carry out the encoding and decoding of RS (90,15), decoding
Time solve inverse matrix performance and decoding performance is the lowest.
Based on RS correcting and eleting codes the multistage decoding method provided for the embodiment of the present invention, owing to multistage encoding and decoding system is permissible
Combine with the storage hardware of cloud storage, therefore, it can constitute distributed type assemblies encoding and decoding, make full use of in storage rack and calculate
Node, the computing capability of memory node.When coding, FAC is used only to carry out third level RS(m3+n3, m3) coding, complete
Produce after m3+n3 layer data after one-tenth, i.e. return to application and encode successfully, store rack subsequently and fall into a trap operator node and memory node again
Carry out the second layer and ground floor coding.So In the view of application, it is only necessary to carry out RS(m3+n3, m3) coding i.e. can return to coding
Success, it is only necessary to additionally encode the content of n3/ (m3+n3).Decoding time, by storage rack fall into a trap operator node and memory node complete
Become the decoding operation of the second level and the first order, after producing complete datum plane, then send FAC to and do the decoding of the third level.Deposit
The storage node m3*m2/mono-decoding data to whole file, calculates node only to whole file m3/mono-data
It is decoded, therefore the decoding operating time of the second level and the first order, is far smaller than the first order decoding that FAC is carried out.So exist
In distributed multi-stage coding/decoding system, in application, the scramble time is RS(m3+n3, m3) scramble time, the decoding time is
Slightly larger than RS(m3+n3, m3), i.e. slightly larger than O (n3).
For two-stage coding/decoding system, storage rack calculating node and complete first order encoding and decoding, FAC completes second level encoding and decoding.
In application, the scramble time is RS(m2+n2, m2) scramble time, the decoding time is slightly larger than RS(m2+n2, m2),
I.e. slightly larger than O (n2).For the encoding and decoding under data communication mode, the multistage encoding and decoding system thought that the embodiment of the present invention is provided
May be used without, i.e. the second level therein and first order encoding and decoding task, encoding and decoding Task-decomposing, are sent to by the FAC of the third level
Other computers in distributed system are carried out, thus form more efficient distributed multi-stage coding/decoding system, can resist more simultaneously
Corrupted data or loss.
In actual use, the coded data of each plane, can leave in and individually store in rack.m3+n3
Layer data, is stored in m3+n3 storage rack.The wherein redundant data of n3 layer plane, is stored in single n3 and deposits
In storage rack.For same storage rack, m2+n2 row data, it is respectively stored in the storage server of m2+n2, wherein
The verification data of n2 row, are all saved in single n2 storage server.Under system is in the read-only pattern do not write, and it is
System disk make mistakes negligible amounts time, verification data storage rack, frame on verification storage server can close or fall-back,
To save system operation expense.
Based on RS correcting and eleting codes the multistage coding/decoding system that the embodiment of the present invention is provided, the RS encoding and decoding of each of which rank
Algorithm is independent, it is therefore possible to use following mutation improves performance further: such as: (1) reduce FAC encoding and decoding that
The check number n of one-level, for two grades of coding/decoding systems, reduces n2 value；For three grades of coding/decoding systems, reduce n3 value.So
The encoding and decoding time that application is felt can reduce accordingly.The most such as, that one-level of FAC encoding and decoding uses more efficient encoding and decoding to calculate
Method, owing to the encoding and decoding time of application perception depends primarily on that one-level of FAC, therefore, the RS algorithm used in FAC can
To use more efficient algorithm.Such as, that one-level of FAC, according to two-stage encoding and decoding or three grades of encoding and decoding systems, by correspondence
Verification burst quantity n3 or n2 are set to 1, and the XOR algorithm of XOR type so can be used to replace original RS correcting and eleting codes
Algorithm.Owing to XOR algorithm realizes simple efficient, the encoding-decoding efficiency of application perception can be improved further.
Based on RS correcting and eleting codes the multistage coding/decoding system that above-described embodiment and preferred implementation are provided, can be real by software
Existing, by the most independent multistage RS correcting and eleting codes system, data are carried out multistage encoding and decoding, data or file are entered for unit by group
Row coding and decoding.Data also mutual redundancy backup between group and group.Use the embodiment of the present invention be not required for quantity to be grouped and
In group, data bulk is equal, can in any combination, more preferably, data bulk in marshalling quantity and group, can be with cloud storage physics
Equipment point territory situation carries out correspondence, facilitates cloud storage system to be managed and realizes.Use the multistage encoding and decoding of the embodiment of the present invention
System, can utilize cloud storage system or the computing capability of distributing communication system and storage capacity easily, forms one
Distributed coding/decoding system, is effectively improved encoding and decoding performance.It addition, the multistage coding/decoding system that the embodiment of the present invention is provided,
Not limiting the kind of correcting and eleting codes, coding/decoding systems at different levels the most independently realize, and are independent of each other.Therefore, it can for user's property
Energy requirement, uses the optimization encoding and decoding algorithm of XOR type, improves encoding and decoding performance further.With original single-stage RS encoding and decoding
System is compared, and uses the present invention, on the premise of not reducing original encoding rate and memory space utilization rate, can increase considerably and is
System fault-tolerant ability, improves coding rate and decoding speed simultaneously, is well suited for using cloud storage system and the P2P of cheap civil disk
The scene of dynamic storage system.Additionally, the multistage decoding method using the embodiment of the present invention to be provided, data can will be verified
Concentrate in some rack of cloud storage, frame.Under system is in the read-only pattern do not write, and system disk makes mistakes quantity relatively
Time few, in verification data rack, frame, verification storage server can be closed or fall-back, saves system operation expense.
Below in conjunction with the accompanying drawings, the embodiment of the present invention is illustrated.
Fig. 6 is the structural representation of based on RS correcting and eleting codes multistage coding/decoding system according to embodiments of the present invention, as shown in Figure 6,
This system uses three grades of encoding and decoding to process data.
In three grades of RS correcting and eleting codes coding/decoding systems of this use, three grades coding be respectively RS(m1+n1, m1), RS(m2+n2,
M2), RS(m3+n3, m3) correcting and eleting codes.Data to be encoded, logically carry out piecemeal by m1*m2*m3, often go
M1, an each column m2 former data, the data of total m3 layer.If end of file piecemeal is less than m1*m2*m3,0 can be filled
Make it meet m1*m2*m3 to divide, it is possible to individually store end of file by copy mode.
First carry out the first order plane coding during document No., use RS(m3+n3, m3) coding.After end-of-encode, produce
The verification datum plane of n3 layer, each verification datum plane contains m1*m2 verification data, and each verification data are by m3 layer
In former datum plane, each data are calculated according to RS correcting and eleting codes.First order plane coding, is completed by FAC.As
Really FAC has been served only for first order coding, then can return coding success message to application after completing.
Carry out second level row coding subsequently, use RS(m2+n2, m2) coding.For the m1*m2 in each plane former number
According to, m1 individually row can be formed.Each row are carried out RS(m2+n2, m2) coding, each row produce n2 verification
Data.The n3 layer verification data produced for first order plane coding, carry out second level row coding too, produce the second level
Verification data.After the row coding of the second level, every layer plane data will produce extra n2 row data.Second level coding can be by FAC
Complete, it is possible to after being completed first order coding by FAC, the data of each plane are sent to each storage rack and calculate node, by respectively
Individual storage rack calculates node and completes.
Finally carry out third level row coding, use RS(m1+n1, m1) coding.For the m1*m2 in each plane former number
According to, m2 individually row can be formed.Each row is carried out RS(m1+n1, m1) coding, each row produces n1 verification
Data.For the row of n2 the extra verification data composition that second level row coding produces, carry out third level row volume too；
For first order plane coding produce n3 layer verification data, then complete the second level row coding after, carry out third level row coding equally.
After completing third level coding, deblocking is become (m1+n1) * (m2+n2) * (m3+n3) from m1*m2*m3.Third level row coding can
To be calculated node completed by FAC or each storage rack, it is possible to first by each row of data of each plane, be sent to store rack
On each memory node, each memory node calculate.After calculating, m1+n1 the data often gone will be stored in
On m1+n1 independent disk of memory node.So, the data of each data plane layer, deposit in different storage racks；
The each row of data of same datum plane, is stored in memory nodes different under same rack；Each data in each row of data, deposit
It is stored in the different disk under same memory node.
The decoding process of file and cataloged procedure are the most contrary.First third level row decoding is carried out, as long as obtaining on each memory node
Obtain m1 data in magnetic disk, this grade of row decoding can be completed, it is thus achieved that the initial data of one's own profession；When carrying out second level row decoding, only
M2 row to be had completes row decoding, can complete this grade of row decoding, it is thus achieved that the initial data of this plane；Put down carrying out the first order
During the decoding of face, as long as there being the data of m3 plane, whole initial data can be obtained.
Under the multistage RS encoding and decoding system that the embodiment of the present invention is provided, at most can resist n3 rack corrupted data or fall
Electricity.For each rack, n2 memory node of multi-resistance damages or power down.For each memory node, multi-resistance n1
Disk failures.If rack, memory node are all normal, whole system resistant to arbitrary (n1+1) (n2+1) (n3+1)-1 disk failures.
Take m3=7, n3=1, m2=7, n2=1, m1=21, n1=3, whole system can be calculated and have disk 8*8*24=1536
Block, encoding rate P=7*7*21/8*8*24=66.99%, the most any 15 pieces of disk failures can be accommodated, at most accommodate
8*8*24-7*7*21=507 adjustment debit is bad, i.e. accommodates up to the disk failures of 33%.For three grades of encoding and decoding systems of this example, three grades
Encoding and decoding amount of calculation be respectively O (1), O (1), Q (3), the amount of calculation is equivalent to the O (5) of one-level encoding and decoding system, but diamagnetic
Adjustment debit bad quantity increases to 15 from 5.
If using two-stage coding system, 8 racks 8 are stored server, altogether 64 the storage server composition second level,
Whole system has disk 64*24=1536 block, m1=21, n1=3, m2=61, n2=3, then encoding rate
P2=61*21/64*24=83.40%, at least can anti-4*4-1=15 block disk failures, to multi-resistance 64*24-61*21=255 block disk damage
Bad, i.e. accommodate up to the disk failures of 16.6%.Compare with three grades of coding systems, under same anti-15 pieces of disk failures premises,
Improve encoding rate, but reduce maximum disk damage-retardation quantity.For two grades of encoding and decoding systems of this example, two grades of encoding and decoding calculate
Amount is respectively O (3), O (3), and the amount of calculation is equivalent to the O (6) of one-level encoding and decoding system, but anti-disk failures quantity increases from 6
To 15.
Therefore, by above-described embodiment and preferred implementation, improve codec rate the most significantly, and, significantly carry
High system survivability.
Obviously, those skilled in the art should be understood that each module of the above-mentioned present invention or each step can be with general calculating
Device realizes, and they can concentrate on single calculating device, or is distributed on the network that multiple calculating device is formed,
Alternatively, they can realize with calculating the executable program code of device, it is thus possible to be stored in storing device
In perform by calculating device, and in some cases, can with the order being different from herein perform shown or described by step
Suddenly, or they are fabricated to respectively each integrated circuit modules, or the multiple modules in them or step are fabricated to single
Integrated circuit modules realizes.So, the present invention is not restricted to the combination of any specific hardware and software.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for those skilled in the art
For, the present invention can have various modifications and variations.All within the spirit and principles in the present invention, any amendment of being made, etc.
With replacement, improvement etc., should be included within the scope of the present invention.
Claims (10)
1. a coding and decoding processing method, it is characterised in that including:
Treat codec data and carry out higher dimensional formats process, wherein, described multidimensional at least two dimension；
In a predetermined sequence, at least two dimension in the every dimension treating codec data after higher dimensional formatsization being processed
Carry out Read-Solomon RS correcting and eleting codes encoding and decoding to process.
Method the most according to claim 1, it is characterised in that treat codec data carry out higher dimensional formatsization process include:
Determine the deblocking size that described data to be encoded are formatted process；
In the case of performing coded treatment, described data to be encoded are mended by the described deblocking size according to determining
Cut process；In the case of performing decoding process, data to be decoded are stored in the data of the described deblocking size determined
The relevant position of block is decoded processing.
Method the most according to claim 1, it is characterised in that in a predetermined sequence, after processing higher dimensional formatsization
Treat that at least two dimension in every dimension of codec data carries out described RS correcting and eleting codes encoding and decoding process and includes:
In the case of performing coded treatment, go the mode of dimension according to described multidimensional step by step, higher dimensional formats is processed it
After the every dimension treating codec data carry out described RS correcting and eleting codes coded treatment；
In the case of performing decoding process, add the mode of dimension according to described multidimensional step by step, higher dimensional formats is processed it
After the every dimension treating codec data carry out described RS correcting and eleting codes decoding process.
Method the most according to claim 1, it is characterised in that in a predetermined sequence, after processing higher dimensional formatsization
The every dimension treating codec data at least two dimension carry out described RS correcting and eleting codes encoding and decoding process after, also include:
According to the physical resource of storage server, the data carrying out obtaining after described RS correcting and eleting codes processes are stored；
Or,
Send the data carrying out obtaining after described RS correcting and eleting codes processes.
Method the most according to claim 4, it is characterised in that according to the physical resource of storage server, to carrying out described RS
The data that correcting and eleting codes obtains after processing carry out storage and include: will carry out the described number obtained after described RS correcting and eleting codes processes
Part verification data according to are stored on single memory node.
Method the most according to any one of claim 1 to 5, it is characterised in that in the case of described multidimensional is three-dimensional,
In a predetermined sequence, each of codec data is treated after at least one in the following manner higher dimensional formatsization being processed
At least two dimension in dimension carries out described RS correcting and eleting codes encoding and decoding and processes:
Be positioned at storage server same file access client FAC to higher dimensional formatsization process after treat encoding and decoding number
According to every dimension at least two dimension carry out described RS correcting and eleting codes encoding and decoding process；
After described FAC completes the first order encoding and decoding corresponding to the first dimension, the first order is compiled by the calculating node in storage territory
Decoded data complete the encoding and decoding of the third level corresponding to the second level corresponding to the second dimension and third dimension；
After described FAC completes the first order encoding and decoding corresponding to the first dimension, the first order is compiled by the calculating node in storage territory
Decoded data complete second level encoding and decoding corresponding to the second dimension, and memory node is to the number after the encoding and decoding of the second level
According to the third level encoding and decoding completed corresponding to third dimension.
7. an encoding and decoding processing means, it is characterised in that including:
First processing module, is used for treating codec data and carries out higher dimensional formats process, and wherein, described multidimensional is at least
Two dimension；
Second processing module, in a predetermined sequence, treats the every of codec data after processing higher dimensional formatsization
At least two dimension in dimension carries out Read-Solomon RS correcting and eleting codes encoding and decoding and processes.
Device the most according to claim 7, it is characterised in that described first processing module includes:
First determines unit, for determining the deblocking size that described data to be encoded format process；
First processing unit, in the case of performing coded treatment, according to the described deblocking size determined to institute
State data to be encoded to carry out benefit and cut process；In the case of performing decoding process, data to be decoded are stored in described in determining
The relevant position of the data block of deblocking size is decoded processing.
Device the most according to claim 7, it is characterised in that described second processing module includes:
Second processing unit, for, in the case of performing coded treatment, going the mode of dimension step by step according to described multidimensional,
At least two dimension in the every dimension treating codec data after processing higher dimensional formatsization carries out described RS correcting and eleting codes and compiles
Code processes；
3rd processing unit, for, in the case of performing decoding process, adding the mode of dimension step by step according to described multidimensional,
At least two dimension in the every dimension treating codec data after processing higher dimensional formatsization carries out described RS correcting and eleting codes solution
Code processes；
10. according to the device according to any one of claim 7 to 9, it is characterised in that described second processing module include with down to
One of few: in a predetermined sequence, treat each of codec data after in the following manner higher dimensional formatsization being processed
At least two dimension in dimension carries out described RS correcting and eleting codes encoding and decoding and processes:
Fourth processing unit, in the case of described multidimensional is three-dimensional, by being positioned at the same literary composition of storage server
The every dimension treating codec data after higher dimensional formatsization is processed by part access client FAC carries out described RS and entangles and delete
Code encoding and decoding process；
5th processing unit, in the case of described multidimensional is three-dimensional, completes the first dimension by described FAC right
After the first order encoding and decoding answered, it is right that the calculating node in storage territory completes the second dimension institute to the data after first order encoding and decoding
Answer the encoding and decoding of the third level corresponding to the second level and third dimension；
6th processing unit, in the case of described multidimensional is three-dimensional, completes the first dimension by described FAC right
After the first order encoding and decoding answered, it is right that the calculating node in storage territory completes the second dimension institute to the data after first order encoding and decoding
Answer second level encoding and decoding, and memory node completes the third level corresponding to third dimension to the data after the encoding and decoding of the second level
Encoding and decoding.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210275397.6A CN102843212B (en) | 2012-08-03 | 2012-08-03 | Coding and decoding processing method and device |
| PCT/CN2013/080767 WO2014019549A1 (en) | 2012-08-03 | 2013-08-02 | Coding/decoding processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210275397.6A CN102843212B (en) | 2012-08-03 | 2012-08-03 | Coding and decoding processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102843212A CN102843212A (en) | 2012-12-26 |
| CN102843212B true CN102843212B (en) | 2016-10-26 |
Family
ID=47370301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210275397.6A Active CN102843212B (en) | 2012-08-03 | 2012-08-03 | Coding and decoding processing method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102843212B (en) |
| WO (1) | WO2014019549A1 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102843212B (en) * | 2012-08-03 | 2016-10-26 | 南京中兴新软件有限责任公司 | Coding and decoding processing method and device |
| CN105338035B (en) * | 2014-08-07 | 2018-12-28 | 苏宁易购集团股份有限公司 | Method for secure storing and device based on RS correcting and eleting codes |
| WO2017107107A1 (en) * | 2015-12-23 | 2017-06-29 | Intel Corporation | Techniques to recover data in a network storage system |
| CN109426590A (en) * | 2017-09-01 | 2019-03-05 | 阿里巴巴集团控股有限公司 | Method for the method for back end storing data and for restoring data |
| CN108132854B (en) * | 2018-01-15 | 2020-11-17 | 成都信息工程大学 | Erasure code decoding method capable of simultaneously recovering data elements and redundant elements |
| CN109274717B (en) * | 2018-08-22 | 2021-08-24 | 泰康保险集团股份有限公司 | Block chain based shared storage method, device, medium and electronic equipment |
| CN114296648B (en) * | 2021-12-24 | 2023-08-08 | 天翼云科技有限公司 | Maintenance method, device, equipment and readable medium for distributed cloud storage data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1855794A (en) * | 2005-04-21 | 2006-11-01 | 华为技术有限公司 | Data transmission protection based on correcting and eleting codes |
| CN101339524A (en) * | 2008-05-22 | 2009-01-07 | 清华大学 | Magnetic disc fault tolerance method of large scale magnetic disc array storage system |
| US8127212B2 (en) * | 2007-09-26 | 2012-02-28 | Hewlett-Packard Development Company, L.P. | System and method for determining the fault-tolerance of an erasure code |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102843212B (en) * | 2012-08-03 | 2016-10-26 | 南京中兴新软件有限责任公司 | Coding and decoding processing method and device |
-
2012
- 2012-08-03 CN CN201210275397.6A patent/CN102843212B/en active Active
-
2013
- 2013-08-02 WO PCT/CN2013/080767 patent/WO2014019549A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1855794A (en) * | 2005-04-21 | 2006-11-01 | 华为技术有限公司 | Data transmission protection based on correcting and eleting codes |
| US8127212B2 (en) * | 2007-09-26 | 2012-02-28 | Hewlett-Packard Development Company, L.P. | System and method for determining the fault-tolerance of an erasure code |
| CN101339524A (en) * | 2008-05-22 | 2009-01-07 | 清华大学 | Magnetic disc fault tolerance method of large scale magnetic disc array storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102843212A (en) | 2012-12-26 |
| WO2014019549A1 (en) | 2014-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102843212B (en) | Coding and decoding processing method and device | |
| US10108819B1 (en) | Cross-datacenter extension of grid encoded data storage systems | |
| US9959167B1 (en) | Rebundling grid encoded data storage systems | |
| US10162704B1 (en) | Grid encoded data storage systems for efficient data repair | |
| US9998539B1 (en) | Non-parity in grid encoded data storage systems | |
| US9904589B1 (en) | Incremental media size extension for grid encoded data storage systems | |
| CN111149093B (en) | Data encoding, decoding and repairing method of distributed storage system | |
| CN106100801B (en) | A kind of non-homogeneous erasure code method of cloud storage system | |
| CN102833040B (en) | Decoding processing method, device and coding/decoding system | |
| CN107656832A (en) | A kind of correcting and eleting codes method of low data reconstruction expense | |
| US8392805B2 (en) | Non-MDS erasure codes for storage systems | |
| JP2018508073A (en) | Data removal, allocation and reconstruction | |
| WO2023151290A1 (en) | Data encoding method and apparatus, device, and medium | |
| CN106484559A (en) | A kind of building method of check matrix and the building method of horizontal array correcting and eleting codes | |
| Datta et al. | An overview of codes tailor-made for better repairability in networked distributed storage systems | |
| Venkatesan et al. | Effect of codeword placement on the reliability of erasure coded data storage systems | |
| Esmaili et al. | The core storage primitive: Cross-object redundancy for efficient data repair & access in erasure coded storage | |
| WO2024001494A1 (en) | Data storage method, single-node server, and device | |
| Qiu et al. | Ec-fusion: An efficient hybrid erasure coding framework to improve both application and recovery performance in cloud storage systems | |
| CN113687975A (en) | Data processing method, device, equipment and storage medium | |
| WO2019227465A1 (en) | Erasure coding with overlapped local reconstruction codes | |
| Ivanichkina et al. | Mathematical methods and models of improving data storage reliability including those based on finite field theory | |
| Estrada-Galinanes et al. | Alpha entanglement codes: practical erasure codes to archive data in unreliable environments | |
| CN116501553B (en) | Data recovery method, device, system, electronic equipment and storage medium | |
| CN108614749A (en) | A kind of data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20160919 Address after: Yuhuatai District of Nanjing City, Jiangsu province 210012 Bauhinia Road No. 68 Applicant after: Nanjing Zhongxing New Software Co., Ltd. Address before: 518057 Nanshan District science and technology, Guangdong Province, South Road, No. 55, No. Applicant before: ZTE Corporation |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180426 Address after: No. 55, Nanshan District science and technology road, Nanshan District, Shenzhen, Guangdong Patentee after: ZTE Corporation Address before: 210012 No. 68 Bauhinia Road, Yuhuatai District, Jiangsu, Nanjing Patentee before: Nanjing Zhongxing New Software Co., Ltd. |