CN106469172A - The data-updating method of distributed file system and device - Google Patents
The data-updating method of distributed file system and device Download PDFInfo
- Publication number
- CN106469172A CN106469172A CN201510512344.5A CN201510512344A CN106469172A CN 106469172 A CN106469172 A CN 106469172A CN 201510512344 A CN201510512344 A CN 201510512344A CN 106469172 A CN106469172 A CN 106469172A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- index
- back end
- updating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of data-updating method of distributed file system of disclosure and device, wherein said method includes：Acquisition updates the data, and by the described afterbody updating the data write current data node, and updates corresponding index information in described back end；By the described afterbody updating the data and writing described check-node, and update corresponding index information in described check-node.Achieve high-performance in the data updating process of distributed file system, be not take up computing resource and the consistent effect of data by the application.
Description
Technical field
The application is related to field of computer technology, and the data of more particularly, to a kind of distributed file system updates
Method and apparatus.
Background technology
With the continuous development of Internet, the data on the Internet becomes explosive and increases.Cost is also more next
Higher.A lot of Distribute file systems are reached to data encoding by Erasure Code (correcting and eleting codes) algorithm
Purpose to reduces cost.In distributed file system, application Erasure Code update algorithm exists very
Many problems are it is impossible to apply in production environment, but business realizing operates in the renewal be unableing to do without file again,
So it is also more and more urgent how to solve the problems, such as that Erasure Code update algorithm is brought.
Assume that the parameter that Erasure Code encodes is：Back end is M, and checkpoint is N, then existing
The shortcoming of technique algorithm has：
(1) performance is low；
A read-write disk is often：M-1 number evidence read by disk, to disk write M+N number evidence；
B network data transmission amount is big：Receive M-1 number evidence from network, send M-1+N number to network
According to；
(2) computing resource waste is serious, and CPU needs to calculate M number evidence；
(3) when partly failing in renewal process, data is irrecoverable；For example, after the completion of calculating, toward disk
On when writing data, if there are unsuccessfully arising that problems.
In sum, in Distribute file system, application Erasure Code update algorithm carries out data renewal
Lead to the problems such as cost is high, performance is low, concordance is difficult to ensure that it is therefore necessary to propose improved technology
Means solve the above problems.
Content of the invention
The main purpose of the application is to provide a kind of data-updating method of distributed file system and dress
Put, updated with overcoming Distribute file system present in prior art to carry out data and lead to cost height, performance
The problems such as low, concordance is difficult to ensure that.
A kind of data-updating method of distributed file system, described distribution are proposed according to the embodiment of the present application
Formula file system includes multiple back end and at least one check-node, and methods described includes：Obtain
Update the data, by the described afterbody updating the data write current data node, and in described back end
Update corresponding index information；By the described afterbody updating the data and writing described check-node, and described
Corresponding index information is updated in check-node.
Wherein, described update corresponding index information in described back end, including：Update described number
According to the occupied end position of the memory space of node；Find and update the data corresponding former number with described
According to index, by this index modification be described in the index that updates the data.
Wherein, also include：Update the data described and the described mark updating the data corresponding former data
Knowledge information, the identification information of current data node send to described check-node, wherein, described data section
The identification information of point includes：Disk mark, IP address, port information.
Wherein, described update corresponding index information in described check-node, including：Update described school
Test the end position of occupied memory space in node.
Wherein, also include：Gone out relative with this back end according to the identification information-enquiry of described back end
The index information answered；According to the described identification information updating the data corresponding former data, in described number
According to the index finding former data in the index information of node, will update the data described in this index modification
Index.
A kind of data update apparatus of distributed file system are also proposed according to the embodiment of the present application, described point
Cloth file system includes multiple back end and at least one check-node, and described device includes：
Acquisition module, updates the data for obtaining；Back end update module, for by described renewal number
According to the afterbody of write current data node, and update corresponding index information in described back end；School
Test node updates module, for by the described afterbody updating the data and writing described check-node, and described
Corresponding index information is updated in check-node.
Wherein, described back end update module is additionally operable to, and updates the memory space quilt of described back end
The end position taking；Find and the described index updating the data corresponding former data, this is indexed
The index updating the data described in being revised as.
Wherein, also include：Data transmission blocks, for updating the data described updating the data with described
The corresponding identification information of former data, the identification information of current data node send to described check-node,
Wherein, the identification information of described back end includes：Disk mark, IP address, port information.
Wherein, described check-node update module is additionally operable to, and updates occupied storage in described check-node
The end position in space.
Wherein, described check-node update module is additionally operable to, and is looked into according to the identification information of described back end
Ask out the index information corresponding with this back end；Update the data corresponding former data according to described
Identification information, the index information of described back end finds the index of former data, this is indexed
The index updating the data described in being revised as.
According to the technique scheme of the application, by write back end and verification section will be updated the data
The afterbody of point, solves application Erasure Code update algorithm with disk space and carries out data renewal presence
Defect, effectively achieve in the data updating process of distributed file system high-performance, be not take up count
Calculate resource and the consistent effect of data.
Brief description
Accompanying drawing described herein is used for providing further understanding of the present application, constitutes of the application
Point, the schematic description and description of the application is used for explaining the application, does not constitute to the application's
Improper restriction.In the accompanying drawings：
Fig. 1 is the Organization Chart of the distributed file system according to the embodiment of the present application；
Fig. 2 is the schematic diagram of the storage management of the back end according to the embodiment of the present application；
Fig. 3 is the schematic diagram of the storage management of the check-node according to the embodiment of the present application；
Fig. 4 is the flow chart of the data-updating method of the distributed file system according to the embodiment of the present application；
Fig. 5 is the schematic diagram that the data according to the embodiment of the present application updates；
Fig. 6 is the structure of the data update apparatus of the distributed file system according to one embodiment of the application
Block diagram；
Fig. 7 is the structure of the data update apparatus of the distributed file system according to another embodiment of the application
Block diagram.
Specific embodiment
Purpose, technical scheme and advantage for making the application are clearer, concrete below in conjunction with the application
Embodiment and corresponding accompanying drawing are clearly and completely described to technical scheme.Obviously, retouched
The embodiment stated is only some embodiments of the present application, rather than whole embodiments.Based in the application
Embodiment, it is all that those of ordinary skill in the art are obtained under the premise of not making creative work
Other embodiment, broadly falls into the scope of the application protection.
There is provided a kind of data-updating method of distributed file system according to the embodiment of the present application.
The application is built upon on distributed file system, and with reference to Fig. 1, distributed file system is by extremely
A few main controlled node (or referred to as control node), at least one client and multiple memory node composition,
It is in communication with each other by network between the device of above-mentioned three types.Each node is an independent physics
Machine, main controlled node is mainly used in positioning memory node, and each memory node is responsible for one piece of disk.
Wherein, memory node includes storing original data storage node and storage verification data memory node two
Type：
Storage original data storage node, referred to as back end, for storing initial data.Original number
It is cut into different data patch according to according to predefined size (such as 1M), each fritter is with respect to current
Back end has a unique numbering (ID).
The function of back end mainly includes：(1) receive and forward from main controlled node, other data section
Point and/or the network data of client；(2) data on hyperdisk and index information.
In actual applications, the index information of back end specifically includes：Magnetic disc ID, coding original position
With end-of-encode position, the end position (incremented by successively) of occupied memory space, all data blocks
Index.With reference to Fig. 2, each data block 21 corresponds to an index 22, and the index of data block includes：Number
According to block identification (ID), data block starting position, end of data block position.An enforcement in the application
In example, can be with rapidly locating file (data block) in back end by the index information of data block
In position.
With reference to Fig. 3, store verification data memory node, referred to as check-node, Erasure is passed through in storage
32 are updated the data produced by the verification data 31 of Code algorithm generation and more new file, wherein, will
Update the data the afterbody of write check-node.
The data type that check-node and back end not only store is different, and the index information storing
Type also different.As shown in figure 3, check-node would generally be in index data starting position 33
The record coding original position of check-node, end position, the occupied end position of memory space (according to
Secondary incremental), then the position 44 after starting position 33 stores the index of other all back end
Information.As shown in figure 3, record respectively in check-node the disk mark of each back end, IP address,
The index of the data block corresponding verification data block of port information (disc information) and this back end.
Wherein, the index of verification data block can include：Verification data block identification (ID), verification data block are opened
Beginning position, verification data agllutination beam position.In actual applications, the index by verification data block is permissible
Quick position in check-node for the location and check data block.
With reference to Fig. 4, Fig. 4 illustrates the data of the distributed file system according to one embodiment of the application more
The flow chart of new method.
Step S402, acquisition updates the data, by the described afterbody updating the data write current data node,
And update corresponding index information in described back end；
To update the data after the afterbody writing back end, the memory space of back end changes, because
This needs the end position of the occupied memory space to back end to be updated；And, data updates
Afterwards in addition it is also necessary to update the position of the index sensing of former data, specifically：By the described beginning updating the data
Location updating is that N, end position are updated to N and add update the data length, the occupied end of memory space
Location updating adds for N and updates the data length；Wherein, N is the occupied stop bits of memory space before renewal
Put.
Then, by described update the data with the described identification information updating the data corresponding former data,
The identification information of current data node sends to described check-node, wherein, the mark of described back end
Information includes：Disk mark, IP address, port information.
Step S404, by the described afterbody updating the data and writing described check-node, and in described verification section
Corresponding index information is updated in point.
Check-node receives after updating the data, and data supplementing to the afterbody of oneself then needs to update
The index information of check-node, specifically：Gone out and this number according to the identification information-enquiry of described back end
According to the corresponding index information of node；According to the described identification information updating the data corresponding former data,
Find the index of former data in the index information of described back end, by this index modification be described more
The index of new data.
First occupied for memory space end position is updated to N and adds new data length；Then according to institute
The identification information-enquiry stating back end goes out the index information related to this back end, by described renewal number
According to starting position be updated to N, end position is updated to N and adds update the data length.
The example updating with reference to the data that Fig. 5 describes distributed file system in detail.
In an embodiment of the application, described distributed file system can be TFS (Taobao File
System, Taobao's file system), TFS system manages data file with data block (Block) for unit
Storage, each data block has a globally unique identification information (ID).When storing small documents,
Multiple small documents can be stored in same data block, TFS system supports the renewal to file simultaneously
Operation.TFS system, in storage file, sets up the index information according to this document., index information is not
Carry out Erasure Code coding, be copied directly in the back end (check-node) at check block place.
Only Erasure Code coding is carried out to the truthful data of each data block in an encoding process, not right
The index information coding of each data block.The index information of each data block can copy to each check block (school
Test node), and store in the index file of check block.
Update the data flow process to briefly describe, setting distributed file system include 3 back end and
1 check-node (for only illustrating 2 back end in simple and clear Fig. 5), coding starting position is 0, knot
Beam position is M, and the occupied end position of memory space is N.Assume to back end 1, numbering is
2 data block is updated, and obtains in advance and updates the data (new data), with reference to Fig. 5, to back end
The more new technological process of 12 number blocks (i.e. former data) is as follows：
(1) new data is appended to the afterbody of oneself by back end 1, updates the related rope of 2 number blocks
Fuse ceases.In back end 1, the index of 2 number blocks is index 2, before the update, index
2 point to data block 2 (i.e. shown in figure phantom line segments)；After renewal, needs will index 2 sensings and update number
According to, that is, the starting position of index 2 is updated to N, end position is updated to N+ new data length (i.e.
Shown in figure real segment), the occupied end position of memory space is updated to N+ new data length；
(2) back end 1 passes through network by new data, the identification information of former data and back end
1 relevant information (magnetic disc ID, IP address, port numbers PORT) is sent to check-node 1；
(3) in check-node 1, before the update, index 2 sensing data block 2 (i.e. in figure dotted line
Shown in section)；After renewal, after check-node 1 receives new data, first new data is appended to certainly
Oneself afterbody, update the occupied end position of memory space is N+ new data length simultaneously；By magnetic
Disk ID, IP address, port numbers inquire the index information related to back end 1, update the data block 2
Index information, index 2 point to update the data (i.e. shown in figure real segment).
Following technique effect is capable of according to embodiments herein：
(1) high-performance；
Read-write disk number of times only account for original scheme (2/ (M-1+M+N)=
2/ (3-1+3+1)=2/6=33.33%；
Network biography amount only accounts for (2/ (M-1+M+N)=2/ (3-1+3+1)=2/6=33.33% of original scheme；
(2) it is not take up computing resource it is only necessary in disk end supplemental data；
(3) data that user see consistent situation occurs when partly failing in renewal process.
Also provide a kind of data update apparatus of distributed file system according to the embodiment of the present application, described point
Cloth file system includes multiple back end and at least one check-node.
Fig. 6 is the structure of the data update apparatus of the distributed file system according to one embodiment of the application
Block diagram, as shown in fig. 6, described device includes：
Acquisition module 610, updates the data for obtaining；
Back end update module 620, for by described update the data write current data node afterbody,
And update corresponding index information in described back end；
Further, described back end update module 620 updates the memory space quilt of described back end
The end position taking；Find and the described index updating the data corresponding former data, this is indexed
The index updating the data described in being revised as.Specifically, the described starting position updating the data is updated to
N, end position be updated to N add update the data length, the occupied end position of memory space is updated to
N adds and updates the data length；Wherein, N is the end position of occupied memory space before renewal.
Check-node update module 630, for by the described afterbody updating the data and writing described check-node,
And update corresponding index information in described check-node.
Further, described check-node update module 630 updates occupied storage in described check-node
The end position in space；Gone out corresponding with this back end according to the identification information-enquiry of described back end
Index information；According to the described identification information updating the data corresponding former data, in described data
Find the index of former data in the index information of node, by this index modification be described in the rope that updates the data
Draw.Specifically, occupied for memory space end position is updated to N and adds new data length；According to
The identification information-enquiry of described back end goes out the index information related to this back end；By described renewal
The starting position of data is updated to N, end position is updated to N and adds update the data length.
With reference to Fig. 7, Fig. 7 is that the data of the distributed file system according to another embodiment of the application updates
The structured flowchart of device, described device includes：Acquisition module 710, back end update module 720,
Check-node update module 730, data transmission blocks 740.
Wherein, acquisition module 710, back end update module 720, check-node update module 730
Respectively with the acquisition module 610 shown in Fig. 6, back end update module 620, check-node update module
630 are similar to, and repeat no more.
As shown in fig. 7, data transmission blocks 740, for updating the data described and described renewal number
Send to described verification section according to the identification information of the identification information of corresponding former data, current data node
Point, wherein, the identification information of described back end includes：Disk mark, IP address, port information.
The operating procedure of the method for the present invention is corresponding with the architectural feature of device, can be cross-referenced, no longer
Repeat one by one.
In sum, the technique scheme according to the application, by updating the data write back end
And the afterbody of check-node, application Erasure Code update algorithm is solved with disk space and enters line number
According to update exist defect, effectively achieve in the data updating process of distributed file system high-performance,
It is not take up computing resource and the consistent effect of data.
It will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer
Program product.Therefore, the application using complete hardware embodiment, complete software embodiment or can combine soft
The form of the embodiment of part and hardware aspect.And, the application can adopt and wherein comprise one or more
Have computer usable program code computer-usable storage medium (including but not limited to disk memory,
CD-ROM, optical memory etc.) the upper computer program implemented form.
One typical configure, computing device include one or more processors (CPU), input/
Output interface, network interface and internal memory.
Internal memory potentially includes the volatile memory in computer-readable medium, random access memory
(RAM) and/or the form such as Nonvolatile memory, such as read only memory (ROM) or flash memory (flash
RAM).Internal memory is the example of computer-readable medium.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by
Any method or technique is realizing information Store.Information can be computer-readable instruction, data structure,
The module of program or other data.The example of the storage medium of computer includes, but are not limited to phase transition internal memory
(PRAM), static RAM (SRAM), dynamic random access memory (DRAM),
Other kinds of random access memory (RAM), read only memory (ROM), electrically erasable
Read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read only memory
(CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, tape magnetic
Disk storage or other magnetic storage apparatus or any other non-transmission medium, can be used for storage can be counted
The information that calculation equipment accesses.Define according to herein, computer-readable medium does not include temporary computer can
Read media (transitory media), the such as data signal of modulation and carrier wave.
Also, it should be noted term " inclusion ", "comprising" or its any other variant be intended to non-
The comprising of exclusiveness, so that include a series of process of key elements, method, commodity or equipment not only
Including those key elements, but also include other key elements being not expressly set out, or also include for this
Process, method, commodity or the intrinsic key element of equipment.In the absence of more restrictions, by language
The key element that sentence "including a ..." limits is it is not excluded that including the process of described key element, method, business
Also there is other identical element in product or equipment.
The foregoing is only embodiments herein, be not limited to the application.For this area
For technical staff, the application can have various modifications and variations.All spirit herein and principle it
Interior made any modification, equivalent substitution and improvement etc., should be included in claims hereof scope
Within.
Claims (10)
1. a kind of data-updating method of distributed file system, described distributed file system includes multiple
Back end and at least one check-node are it is characterised in that methods described includes：
Acquisition updates the data, by the described afterbody updating the data write current data node, and in described number
According to updating corresponding index information in node；
By the described afterbody updating the data and writing described check-node, and update in described check-node right
The index information answered.
2. method according to claim 1 it is characterised in that described in described back end more
Newly corresponding index information, including：
Update the occupied end position of memory space of described back end；
Find and the described index updating the data corresponding former data, by this index modification be described more
The index of new data.
3. method according to claim 1 is it is characterised in that also include：
Update the data described and the described identification information updating the data corresponding former data, current number
Send to described check-node, wherein, the identification information bag of described back end according to the identification information of node
Include：Disk mark, IP address, port information.
4. method according to claim 3 it is characterised in that described in described check-node more
Newly corresponding index information, including：
Update the end position of occupied memory space in described check-node.
5. method according to claim 4 is it is characterised in that also include：
The index information corresponding with this back end is gone out according to the identification information-enquiry of described back end；
According to the described identification information updating the data corresponding former data, in the rope of described back end
Find the index of former data in fuse breath, by this index modification be described in the index that updates the data.
6. a kind of data update apparatus of distributed file system, described distributed file system includes many
Individual back end and at least one check-node are it is characterised in that described device includes：
Acquisition module, updates the data for obtaining；
Back end update module, for by described update the data write current data node afterbody, and
Corresponding index information is updated in described back end；
Check-node update module, for by the described afterbody updating the data and writing described check-node, and
Corresponding index information is updated in described check-node.
7. device according to claim 6 it is characterised in that described back end update module also
For updating the occupied end position of memory space of described back end；Find and described renewal
The index of the corresponding former data of data, by this index modification be described in the index that updates the data.
8. device according to claim 6 is it is characterised in that also include：
Data transmission blocks, for updating the data corresponding former data by described updating the data with described
Identification information, the identification information of current data node sends to described check-node, wherein, described number
Identification information according to node includes：Disk mark, IP address, port information.
9. device according to claim 8 it is characterised in that described check-node update module also
For updating the end position of occupied memory space in described check-node.
10. device according to claim 10 is it is characterised in that described check-node update module
It is additionally operable to, the index letter corresponding with this back end is gone out according to the identification information-enquiry of described back end
Breath；According to the described identification information updating the data corresponding former data, in the rope of described back end
Find the index of former data in fuse breath, by this index modification be described in the index that updates the data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510512344.5A CN106469172B (en) | 2015-08-19 | 2015-08-19 | The data-updating method and device of distributed file system |
| PCT/CN2016/094322 WO2017028721A1 (en) | 2015-08-19 | 2016-08-10 | Data update method and device in distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510512344.5A CN106469172B (en) | 2015-08-19 | 2015-08-19 | The data-updating method and device of distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106469172A true CN106469172A (en) | 2017-03-01 |
| CN106469172B CN106469172B (en) | 2019-07-23 |
Family
ID=58050718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510512344.5A Active CN106469172B (en) | 2015-08-19 | 2015-08-19 | The data-updating method and device of distributed file system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106469172B (en) |
| WO (1) | WO2017028721A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110245166B (en) * | 2019-05-21 | 2023-09-26 | 创新先进技术有限公司 | Data checking method and device |
| CN114398659A (en) * | 2021-10-28 | 2022-04-26 | 上海哔哩哔哩科技有限公司 | Resource checking method, device, equipment and storage medium |
| CN114676166B (en) * | 2022-05-26 | 2022-10-11 | 阿里巴巴（中国）有限公司 | Data processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101996250A (en) * | 2010-11-15 | 2011-03-30 | 中国科学院计算技术研究所 | Hadoop-based mass stream data storage and query method and system |
| CN102681793A (en) * | 2012-04-16 | 2012-09-19 | 华中科技大学 | Local data updating method based on erasure code cluster storage system |
| CN102799679A (en) * | 2012-07-24 | 2012-11-28 | 河海大学 | Hadoop-based massive spatial data indexing updating system and method |
| US8726129B1 (en) * | 2004-07-23 | 2014-05-13 | Hewlett-Packard Development Company, L.P. | Methods of writing and recovering erasure coded data |
| CN104102558A (en) * | 2014-07-13 | 2014-10-15 | 中国人民解放军国防科学技术大学 | Erasure code based file appending method |
| CN104376053A (en) * | 2014-11-04 | 2015-02-25 | 南京信息工程大学 | Storage and retrieval method based on massive meteorological data |
-
2015
- 2015-08-19 CN CN201510512344.5A patent/CN106469172B/en active Active
-
2016
- 2016-08-10 WO PCT/CN2016/094322 patent/WO2017028721A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8726129B1 (en) * | 2004-07-23 | 2014-05-13 | Hewlett-Packard Development Company, L.P. | Methods of writing and recovering erasure coded data |
| CN101996250A (en) * | 2010-11-15 | 2011-03-30 | 中国科学院计算技术研究所 | Hadoop-based mass stream data storage and query method and system |
| CN102681793A (en) * | 2012-04-16 | 2012-09-19 | 华中科技大学 | Local data updating method based on erasure code cluster storage system |
| CN102799679A (en) * | 2012-07-24 | 2012-11-28 | 河海大学 | Hadoop-based massive spatial data indexing updating system and method |
| CN104102558A (en) * | 2014-07-13 | 2014-10-15 | 中国人民解放军国防科学技术大学 | Erasure code based file appending method |
| CN104376053A (en) * | 2014-11-04 | 2015-02-25 | 南京信息工程大学 | Storage and retrieval method based on massive meteorological data |
Non-Patent Citations (1)
| Title |
|---|
| 朱媛媛 等: "基于GE码的HDFS优化方案", 《计算机应用》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017028721A1 (en) | 2017-02-23 |
| CN106469172B (en) | 2019-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| DE112012005037B4 (en) | Manage redundant immutable files using deduplications in storage clouds | |
| CN107219997B (en) | Method and device for verifying data consistency | |
| CN107066498B (en) | Key value KV storage method and device | |
| US20210049711A1 (en) | Method of automatically transmitting data information and device of automatically transmitting data information | |
| CN106649788A (en) | Database data transmission method and device | |
| CN107016047A (en) | Document query, document storing method and device | |
| CN109447820B (en) | Data processing method, device, computer equipment and storage medium | |
| CN106325826A (en) | Configuration file detection method and apparatus | |
| CN105550306A (en) | Multi-copy data reading/writing method and system | |
| CN110968554A (en) | Block chain storage method, storage system and storage medium based on file chain blocks | |
| CN106469172A (en) | The data-updating method of distributed file system and device | |
| CN106469152A (en) | A kind of document handling method based on ETL and system | |
| CN107341070A (en) | A kind of random write method and system based on correcting and eleting codes | |
| CN106649061A (en) | Method and device for recording user's operation log in web system | |
| CN107102908A (en) | Data verification method, data fault tolerance method and device | |
| CN107357588B (en) | Object code generation method and device | |
| CN104090948A (en) | Method, device and system for processing mass data of nuclear power station | |
| CA3138761A1 (en) | Method, device, computer equipment and storage medium for verifying inter-system data admission | |
| US10664349B2 (en) | Method and device for file storage | |
| CN104765775B (en) | A kind of log preservation method and device | |
| CN105988986A (en) | Information processing method and device | |
| CN117522094A (en) | Seal management method, platform, electronic equipment and storage medium | |
| CN112784566A (en) | Document generation method, device, equipment and storage medium | |
| CN106649108A (en) | Generation method and device of test data | |
| CN106293671B (en) | Method and device for generating component template |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |