CN102622412A - Method and device of concurrent writes for distributed file system - Google Patents
Method and device of concurrent writes for distributed file system Download PDFInfo
- Publication number
- CN102622412A CN102622412A CN2012100370141A CN201210037014A CN102622412A CN 102622412 A CN102622412 A CN 102622412A CN 2012100370141 A CN2012100370141 A CN 2012100370141A CN 201210037014 A CN201210037014 A CN 201210037014A CN 102622412 A CN102622412 A CN 102622412A
- Authority
- CN
- China
- Prior art keywords
- writes
- data block
- address
- server
- concurrent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and device of concurrent writes for a distributed file system. A metadata server maintains storage resource information of a file storage server and distributes adjacent storage addresses for concurrently written data according to the maintained storage resource information when receiving concurrent write requests from multiple file access servers. Concurrently written and dispersed file data blocks from a file access client are stored in continuous sectors of a single hard disk drive, and accordingly the concurrently written file data blocks can be continuously written into the hard disk drive at high speed, and the concurrent write capacity of files is improved evidently.
Description
Technical field
The present invention relates to the Computer Applied Technology field, relate in particular to concurrent wiring method and device in a kind of distributed file system.
Background technology
Along with the fast development of internet and multimedia industry, the media file storage of magnanimity is had higher requirement to memory technology and system.For guaranteeing high reliability, high capacity and cost performance, IT vendor and Internet service merchant adopt distributed file system to store data usually.Distributed file system adopts the mode of redundant storage to guarantee to store the reliability of data, remedies the unreliable of hardware with highly reliable software, thereby cheap magnanimity distributed storage and computing system reliably are provided.
As shown in Figure 1; In distributed file system; General main meta data server, document storage server and the file access server of comprising; Wherein: meta data server mainly is in charge of information such as file path, file name, and the distribution situation of file data blocks on document storage server; Document storage server mainly is in charge of the storage of file data blocks on local hard drive, normally on hard disk, creates local file system, then data block is kept on the local hard drive as file; File access server (being client) is responsible for providing the user capture interface, through with meta data server interaction metadata information, and with document storage server interaction data block message, for the user provides the file access service.
Under above-mentioned implementation pattern, the performance of distributed file system usually receives the very big influence of local file system performance.Its reason is that local file system though can guarantee the continuous dispensing of file name, can not guarantee data block file depositing continuously at hard disk sector when carrying out the file storage of data block.In general, to the random read-write poor performance of hard disk sector, and the sequential read write performance is fine.
In sum; To provide file cocurrent to write fashionable when distributed file system; How the distribution of concurrent writing data blocks is continuous on the control documents storage server, to promote the writing speed of hard disk on the document storage server, is to be badly in need of the technical matters that solves in the prior art.
Summary of the invention
The technical matters that the present invention solves provides concurrent wiring method and device in a kind of distributed file system, can significantly promote the file cocurrent write performance of distributed file system.
For solving the problems of the technologies described above, the invention provides the concurrent wiring method in a kind of distributed file system,
The storage resources information of meta data server maintenance documentation storage server;
Said meta data server receives concurrent the writing when asking of a plurality of file access servers, is the adjacent memory address of the said concurrent data allocations that writes according to the said storage resources information of safeguarding.
Further, said method also comprises:
Said meta data server will be included in for the adjacent memory address of the said concurrent data allocations that writes and send to document storage server in the metadata information;
When said document storage server receives the data block that request that the file access server sends writes, be cached to data block and write in the formation; And according to the said memory address that comprises in the said metadata information data block that said data block writes in the formation is sorted, when waiting to satisfy Writing condition, said data block is write the data block that memory address is adjacent in the formation write hard disk in regular turn.
Further, said storage resources information comprises: document storage server information, the hard disk information of document storage server and hard disc physical block message;
Said meta data server is the said adjacent memory address of the said concurrent data allocations that writes, and is specially: the continuous hard disc physical block address in the same hard disk of same document storage server.
Further, saidly satisfy Writing condition, specifically comprise:
When said data block is write data block in the formation and sorts,, then judge and satisfy Writing condition if the adjacent data block of memory address reaches predefined number threshold value, or exceeds pre-set time interval.
Further, the storage resources information of said meta data server maintenance documentation storage server specifically comprises:
To the idle hard disc physical piece on the same hard disk, the arrangement in order that the address is continuous, calculated address bunch;
And the address bunch number according to the idle hard disc physical piece that is comprised at interval carried out descending sort.
Further, said meta data server is the adjacent memory address of the said concurrent data allocations that writes, and comprising: from the maximum address of the idle hard disc physical piece number that comprises bunch, choose continuous hard disc physical block address.
Further, the storage resources information of said meta data server maintenance documentation storage server also comprises:
When the hard disc physical piece on the hard disk discharged, said meta data server bunch upgraded idle hard disc physical piece and address.
In addition, the present invention also provides the concurrent writing station in a kind of distributed file system, and said device comprises SRM module and the concurrent processing module that writes in the meta file server, wherein:
Said SRM module is used for, the storage resources information of maintenance documentation storage server;
The said concurrent processing module that writes is used for, and receives concurrent the writing when asking of a plurality of file access servers, and the said storage resources information of safeguarding according to said SRM module is the adjacent memory address of the said concurrent data allocations that writes.
Further, said device also comprises the data block writing module in the document storage server,
The said concurrent processing module that writes also is used for, and will be included in for the adjacent memory address of the said concurrent data allocations that writes and send to said document storage server in the metadata information;
Said data block writing module is used for, and when receiving the data block that request that the file access server sends writes, is cached to data block and writes in the formation; And according to the said memory address that comprises in the said metadata information data block that said data block writes in the formation is sorted, when waiting to satisfy Writing condition, said data block is write the data block that memory address is adjacent in the formation write hard disk in regular turn.
Further, said SRM module is used for, and safeguards following storage resources information: document storage server information, the hard disk information of document storage server and hard disc physical block message;
The said concurrent processing module that writes is used for, and the said adjacent memory address for the said concurrent data allocations that writes comprises: the continuous hard disc physical block address in the same hard disk of same document storage server.
Further; Said data block writing module is used for; When said data block is write data block in the formation and sorts,, then judge and satisfy Writing condition if the adjacent data block of memory address reaches predefined number threshold value, or exceeds pre-set time interval.
Further, said SRM module is used for, in such a way the storage resources information of maintenance documentation storage server: to the idle hard disc physical piece on the same hard disk, and the arrangement in order that the address is continuous, calculated address bunch; And the address bunch number according to the hard disc physical piece that is comprised at interval carried out descending sort; And, when the hard disc physical piece on the hard disk discharges, to idle hard disc physical piece and address bunch upgrading;
The said concurrent processing module that writes is used for, and for the adjacent memory address of the said concurrent data allocations that writes the time, from the maximum address of the idle hard disc physical piece number that comprises bunch, chooses continuous hard disc physical block address.
Promote the method and the device of file cocurrent write performance in a kind of distributed file system provided by the present invention based on naked dish mode; File data blocks through user's request on the file access client of concurrent and dispersion is write is stored in the contiguous sector of single hard disk; Make these data blocks can write hard disk at high speed; Through the mode of " distributed written document; the centralized hard disk that writes ", realize the leap of hard disk streamability, make the file cocurrent write performance be significantly improved.
Description of drawings
Accompanying drawing described herein is used to provide further understanding of the present invention, constitutes the application's a part, and illustrative examples of the present invention and explanation thereof are used to explain the present invention, do not constitute improper qualification of the present invention.In the accompanying drawings:
Fig. 1 is the concurrent synoptic diagram that writes processing in the distributed file system;
Fig. 2 is the schematic block diagram of the concurrent writing station in the distributed file system of the embodiment of the invention.
Embodiment
This embodiment provides the concurrent wiring method in a kind of distributed file system, specifically adopts following technical scheme:
Meta data server is safeguarded the storage resources information of each document storage server;
When meta data server is received the concurrent request that writes; To be a plurality of file access servers initiate a plurality of data simultaneously in one that sets short time when writing request, is the adjacent memory address of the said concurrent data allocations that writes (also being the memory location) according to the said storage resources information of safeguarding.
Wherein, said storage resources information comprises: document storage server information, hard disk information, and hard disc physical block message.
Wherein, said document storage server information comprises the information of each document storage server, identifies according to the network address of document storage server;
Hard disk information identifies according to the hard disk sequence number;
The hard disc physical block message identifies according to the hard disc physical block number.
Further, said meta data server is the said adjacent memory address of the said concurrent data allocations that writes, and is meant:
Continuous hard disc physical piece in the same hard disk of same document storage server.
Further, said meta data server is safeguarded the storage resources information of each document storage server, specifically comprises:
To the idle hard disc physical piece on the same hard disk, the arrangement in order that the address is continuous, calculated address bunch;
And the address bunch number according to the idle hard disc physical piece that comprises at interval carried out descending sort.
Further, said meta data server is safeguarded the storage resources information of each document storage server, also comprises:
When the hard disc physical piece on the hard disk discharged, said meta data server bunch upgraded idle hard disc physical piece and address.
Further; Said document storage server is according to the metadata information of said meta data server; When the data block that the request of receiving the transmission of file access server writes; Be cached to data block and write in the formation, and the data block that said data block is write in the formation sorts according to hard disc physical block address to be written;
When waiting to satisfy Writing condition, hard disc physical block address continuous data block is write in the hard disk earlier.
Particularly, Writing condition can comprise following implementation: one of which, write morely when concurrent, and when the data block of continuous physical address reaches predefined number threshold value, judge and satisfy Writing condition after the ordering; Its two, write lessly when concurrent, ordering back surpasses pre-set time interval, even the data block of continuous physical address does not reach predefined number threshold values yet, also judges and satisfies Writing condition, writes.
In addition, this embodiment also provides the concurrent writing system in a kind of distributed file system, comprises meta data server, document storage server and file access server etc.Wherein:
Said meta data server; Be used for being in charge of the file metadata information of distributed file system; Comprise file directory/filename, and file data blocks is the distributed intelligence on the storage server (being the stored position information in the hard disk of file data blocks at storage server).And idle hard disc physical piece organized according to continuity; When a plurality of client users initiate the file write operation simultaneously, will be stored in for the data block that each client is distributed in the adjacent hard disc physical piece of same hard disk of same storage server as far as possible.
Said document storage server is used for the metadata information that sends according to meta data server, the access site on hard disk of file control data piece, and carry out transmission of data blocks with the file access server.
Wherein, different with prior art is, in this embodiment; Document storage server is when stores file data blocks; On hard disk, no longer create local file system, but adopt naked dish mode management method, the hard disc physical block size of each operation is consistent with the file data block size.Because, then can't guarantee data block depositing continuously at hard disk sector if still on hard disk, create local file system.
In addition, when document storage server is received the data block of file access server transmission, at once this data block is not write the hard disc physical piece, write formation but be cached to data block to data block earlier; Then the data block that writes formation is sorted according to hard disc physical block address to be written, the data block of preferentially the hard disc physical sequence of addresses being arranged writes hard disk.
Said file access server is used to be responsible for providing access interface to the user, and the file service function is provided.In the time of user's file data, obtain file data blocks document storage server information to be deposited alternately with meta data server, send data block is accomplished data block to document storage server transmission then.
For making the object of the invention, technical scheme and advantage clearer, hereinafter will combine accompanying drawing that embodiments of the invention are elaborated.Need to prove that under the situation of not conflicting, embodiment among the application and the characteristic among the embodiment be combination in any each other.
As shown in Figure 1, in the distributed file system of the present invention, mainly comprise meta data server, document storage server and file access server etc.
Meta data server wherein is used for being in charge of the file metadata information of distributed file system, comprises file directory/filename, the distribution of file data blocks on document storage server.Wherein storage resources has been carried out the triode reason: the first order is a document storage server, identifies according to the network address of document storage server; The second level is hard disk, identifies according to the hard disk sequence number; The third level is the hard disc physical piece, is the integral multiple of sector, specifically can be consistent with file data block size (the file data block size is fixed), identify according to the hard disc physical block number.Simultaneously, the idle hard disc physical piece on the same hard disk is organized according to continuity: continuous address of formation, address bunch, arrange in order; Address that simultaneously will be at interval bunch also gathers, according to the number descending sort of physical block.In the time of meta data server application hard disc physical piece store data, preferentially bunch choose, and carry out continuous dispensing from the maximum address of idle hard disc physical piece number.When discharging the hard disc physical piece, then idle hard disc physical piece is managed, carry out that bunch merging of necessary address is handled etc.For example; When three client users initiate the write operation of file A, file B, file C simultaneously; Meta data server is selected bunch suitable physical address bunch Pa-Ph, and the data block that these concurrent operations are used is stored in the adjacent hard disc physical piece of same hard disk of same storage server.
Said document storage server is used for according to the main control server metadata information, the access site on hard disk of file control data piece, and carry out transmission of data blocks with the file access server.File data blocks adopts naked dish mode management method on storage server, the hard disc physical block size of each operation is consistent with the file data block size.When document storage server is received the data block of file access server transmission, at once this data block is not write the hard disc physical piece, but earlier data-block cache is write formation to data block; The data block that then data block is write in the formation sorts according to the hard disc physical block address to be written in the metadata information, and the data block of preferentially the hard disc physical sequence of addresses being arranged writes disk.For example; Three data block (A1 as file A; A2, A3), after three data blocks (C1, C2, C3) of two data blocks (B1, B2) of file B, file C arrive document storage server at random, do not write hard disk separately at once; But when satisfying Writing condition by the time, eight data blocks of these three files are written among eight continuous hard disc physical piece Pa-Ph together.For example, write morely, when the data block of continuous physical address reaches predefined number threshold value, satisfy Writing condition after the ordering when concurrent; Perhaps, write lessly when concurrent, ordering back surpasses pre-set time interval, even then the data block of continuous physical address does not reach predefined number threshold values yet, also judges and satisfies Writing condition.
Said file access server is used to be responsible for providing access interface to the user, and the file service function is provided.In the time of user's file data, obtain file data blocks document storage server information to be deposited alternately with meta data server, send data block is accomplished data block to document storage server transmission then.For example, created file A, file B, file C on three file access servers respectively, all be submitted to meta data server simultaneously and handled.
In sum; The present invention is in distributed file system; Adopted the method for naked disk management and data block has been write the method that the buffer memory ordering has been adopted in formation for the file data blocks on the document storage server; Data block for different files on the meta data server are asked has simultaneously been carried out the method in hard disc physical address rank continuous dispensing; Reach distributed file, the centralized effect that writes hard disk of writing, fully adapted to the high characteristic of hard disk order write performance, made the concurrent performance that writes of distributed file system file be significantly improved.
As shown in Figure 2, the concurrent writing station in the distributed file system of the embodiment of the invention mainly comprises SRM module and the concurrent processing module that writes in the meta file server, wherein:
Said SRM module is used for, the storage resources information of maintenance documentation storage server;
The said concurrent processing module that writes is used for, and receives concurrent the writing when asking of a plurality of file access servers, and the said storage resources information of safeguarding according to said SRM module is the adjacent memory address of the said concurrent data allocations that writes.
Further, said apparatus also comprises the data block writing module in the document storage server,
The said concurrent processing module that writes also is used for, and will be included in for the adjacent memory address of the said concurrent data allocations that writes and send to said document storage server in the metadata information;
Said data block writing module is used for, and when receiving the data block that request that the file access server sends writes, is cached to data block and writes in the formation; And according to the said memory address that comprises in the said metadata information data block that said data block writes in the formation is sorted, when waiting to satisfy Writing condition, said data block is write the data block that memory address is adjacent in the formation write hard disk in regular turn.
Further, said SRM module is used for, and safeguards following storage resources information: document storage server information, the hard disk information of document storage server and hard disc physical block message;
The said concurrent processing module that writes is used for, and the said adjacent memory address for the said concurrent data allocations that writes comprises: the continuous hard disc physical block address in the same hard disk of same document storage server.
Further; Said data block writing module is used for; When said data block is write data block in the formation and sorts,, then judge and satisfy Writing condition if the adjacent data block of memory address reaches predefined number threshold value, or exceeds pre-set time interval.
Further, said SRM module is used for, in such a way the storage resources information of maintenance documentation storage server: to the idle hard disc physical piece on the same hard disk, and the arrangement in order that the address is continuous, calculated address bunch; And the address bunch number according to the hard disc physical piece that is comprised at interval carried out descending sort; And, when the hard disc physical piece on the hard disk discharges, to idle hard disc physical piece and address bunch upgrading;
The said concurrent processing module that writes is used for, and for the adjacent memory address of the said concurrent data allocations that writes the time, from the maximum address of the idle hard disc physical piece number that comprises bunch, chooses continuous hard disc physical block address.
More than be merely preferred case study on implementation of the present invention; Be not limited to the present invention; The present invention also can have other various embodiments; Under the situation that does not deviate from spirit of the present invention and essence thereof, those of ordinary skill in the art can make various corresponding changes and distortion according to the present invention, but these corresponding changes and distortion all should belong to the protection domain of the appended claim of the present invention.
Obviously, it is apparent to those skilled in the art that above-mentioned each module of the present invention or each step can realize with the general calculation device; They can concentrate on the single calculation element; Perhaps be distributed on the network that a plurality of calculation element forms, alternatively, they can be realized with the executable program code of calculation element; Thereby; Can they be stored in the memory storage and carry out, and in some cases, can carry out step shown or that describe with the order that is different from here by calculation element; Perhaps they are made into each integrated circuit modules respectively, perhaps a plurality of modules in them or step are made into the single integrated circuit module and realize.Like this, the present invention is not restricted to any specific hardware and software combination.
Claims (12)
1. the concurrent wiring method in the distributed file system is characterized in that,
The storage resources information of meta data server maintenance documentation storage server;
Said meta data server receives concurrent the writing when asking of a plurality of file access servers, is the adjacent memory address of the said concurrent data allocations that writes according to the said storage resources information of safeguarding.
2. the method for claim 1 is characterized in that, said method also comprises:
Said meta data server will be included in for the adjacent memory address of the said concurrent data allocations that writes and send to document storage server in the metadata information;
When said document storage server receives the data block that request that the file access server sends writes, be cached to data block and write in the formation; And according to the said memory address that comprises in the said metadata information data block that said data block writes in the formation is sorted, when waiting to satisfy Writing condition, said data block is write the data block that memory address is adjacent in the formation write hard disk in regular turn.
3. according to claim 1 or claim 2 method is characterized in that,
Said storage resources information comprises: document storage server information, the hard disk information of document storage server and hard disc physical block message;
Said meta data server is the said adjacent memory address of the said concurrent data allocations that writes, and is specially: the continuous hard disc physical block address in the same hard disk of same document storage server.
4. method as claimed in claim 2 is characterized in that,
Saidly satisfy Writing condition, specifically comprise:
When said data block is write data block in the formation and sorts,, then judge and satisfy Writing condition if the adjacent data block of memory address reaches predefined number threshold value, or exceeds pre-set time interval.
5. method as claimed in claim 3 is characterized in that,
The storage resources information of said meta data server maintenance documentation storage server specifically comprises:
To the idle hard disc physical piece on the same hard disk, the arrangement in order that the address is continuous, calculated address bunch;
And the address bunch number according to the idle hard disc physical piece that is comprised at interval carried out descending sort.
6. method as claimed in claim 5 is characterized in that,
Said meta data server is the adjacent memory address of the said concurrent data allocations that writes, and comprising: from the maximum address of the idle hard disc physical piece number that comprises bunch, choose continuous hard disc physical block address.
7. method as claimed in claim 5 is characterized in that,
The storage resources information of said meta data server maintenance documentation storage server also comprises:
When the hard disc physical piece on the hard disk discharged, said meta data server bunch upgraded idle hard disc physical piece and address.
8. the concurrent writing station in the distributed file system is characterized in that said device comprises SRM module and the concurrent processing module that writes in the meta file server, wherein:
Said SRM module is used for, the storage resources information of maintenance documentation storage server;
The said concurrent processing module that writes is used for, and receives concurrent the writing when asking of a plurality of file access servers, and the said storage resources information of safeguarding according to said SRM module is the adjacent memory address of the said concurrent data allocations that writes.
9. device as claimed in claim 8 is characterized in that, said device also comprises the data block writing module in the document storage server,
The said concurrent processing module that writes also is used for, and will be included in for the adjacent memory address of the said concurrent data allocations that writes and send to said document storage server in the metadata information;
Said data block writing module is used for, and when receiving the data block that request that the file access server sends writes, is cached to data block and writes in the formation; And according to the said memory address that comprises in the said metadata information data block that said data block writes in the formation is sorted, when waiting to satisfy Writing condition, said data block is write the data block that memory address is adjacent in the formation write hard disk in regular turn.
10. like claim 8 or 9 described devices, it is characterized in that,
Said SRM module is used for, and safeguards following storage resources information: document storage server information, the hard disk information of document storage server and hard disc physical block message;
The said concurrent processing module that writes is used for, and the said adjacent memory address for the said concurrent data allocations that writes comprises: the continuous hard disc physical block address in the same hard disk of same document storage server.
11. device as claimed in claim 9 is characterized in that,
Said data block writing module is used for, and when said data block is write data block in the formation and sorts, if the adjacent data block of memory address reaches predefined number threshold value, or exceeds pre-set time interval, then judges and satisfies Writing condition.
12. device as claimed in claim 10 is characterized in that,
Said SRM module is used for, in such a way the storage resources information of maintenance documentation storage server: to the idle hard disc physical piece on the same hard disk, and the arrangement in order that the address is continuous, calculated address bunch; And the address bunch number according to the hard disc physical piece that is comprised at interval carried out descending sort; And, when the hard disc physical piece on the hard disk discharges, to idle hard disc physical piece and address bunch upgrading;
The said concurrent processing module that writes is used for, and for the adjacent memory address of the said concurrent data allocations that writes the time, from the maximum address of the idle hard disc physical piece number that comprises bunch, chooses continuous hard disc physical block address.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100370141A CN102622412A (en) | 2011-11-28 | 2012-02-17 | Method and device of concurrent writes for distributed file system |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110384878 | 2011-11-28 | ||
| CN201110384878.6 | 2011-11-28 | ||
| CN2012100370141A CN102622412A (en) | 2011-11-28 | 2012-02-17 | Method and device of concurrent writes for distributed file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102622412A true CN102622412A (en) | 2012-08-01 |
Family
ID=46562331
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012100370141A Pending CN102622412A (en) | 2011-11-28 | 2012-02-17 | Method and device of concurrent writes for distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102622412A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102929915A (en) * | 2012-09-18 | 2013-02-13 | 浪潮(北京)电子信息产业有限公司 | Method and system for managing files in trunking system |
| CN104378400A (en) * | 2013-08-15 | 2015-02-25 | 腾讯科技（深圳）有限公司 | Data dispersion and concurrence method and device |
| CN104424292A (en) * | 2013-09-02 | 2015-03-18 | 联想(北京)有限公司 | Data processing method and electronic device |
| CN103647850B (en) * | 2013-12-25 | 2017-01-25 | 北京京东尚科信息技术有限公司 | Data processing method, device and system of distributed version control system |
| CN106980471A (en) * | 2017-03-27 | 2017-07-25 | 联想(北京)有限公司 | A kind of method and device for the hard disk write performance for improving smart machine |
| CN107967265A (en) * | 2016-10-18 | 2018-04-27 | 华为技术有限公司 | Access method, data server and the file access system of file |
| CN108491549A (en) * | 2018-04-09 | 2018-09-04 | 深圳市茁壮网络股份有限公司 | The creation method and device of file directory in a kind of distributed memory system |
| CN108701087A (en) * | 2016-04-01 | 2018-10-23 | 英特尔公司 | The method and apparatus being sequentially written in for the block group for handling the physical block into memory device |
| WO2019080015A1 (en) * | 2017-10-25 | 2019-05-02 | 华为技术有限公司 | Data reading and writing method and device, and storage server |
| CN112115113A (en) * | 2020-09-25 | 2020-12-22 | 北京百度网讯科技有限公司 | Data storage system, method, device, equipment and storage medium |
| CN114302085A (en) * | 2021-12-23 | 2022-04-08 | 云从科技集团股份有限公司 | Data storage method and device, electronic equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1320242A (en) * | 1999-07-28 | 2001-10-31 | 索尼公司 | Recording system, data recording device, memory device, and data recording method |
| US20030220974A1 (en) * | 2002-05-23 | 2003-11-27 | International Business Machines Corporation | Parallel metadata service in storage area network environment |
| CN1567237A (en) * | 2003-06-09 | 2005-01-19 | 浪潮电子信息产业股份有限公司 | Method for constructing high-available distributed memory system |
| CN1619522A (en) * | 2003-11-17 | 2005-05-25 | 联想(北京)有限公司 | Computer group file service system and its input output treatment method |
| CN101149755A (en) * | 2007-10-25 | 2008-03-26 | 中国科学院计算技术研究所 | Distributed file system file writing system and method |
| CN101382948A (en) * | 2008-10-14 | 2009-03-11 | 成都市华为赛门铁克科技有限公司 | File memory method, apparatus and system |
| CN101567001A (en) * | 2009-05-22 | 2009-10-28 | 清华大学 | Method for managing metadata file layout of parallel file system |
| CN101567003A (en) * | 2009-05-27 | 2009-10-28 | 清华大学 | Method for managing and allocating resource in parallel file system |
| US20100082774A1 (en) * | 2005-09-09 | 2010-04-01 | Pitts William M | Distributed File System Consistency Mechanism Extension for Enabling Internet Video Broadcasting |
| CN102073739A (en) * | 2011-01-25 | 2011-05-25 | 中国科学院计算技术研究所 | Method for reading and writing data in distributed file system with snapshot function |
-
2012
- 2012-02-17 CN CN2012100370141A patent/CN102622412A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1320242A (en) * | 1999-07-28 | 2001-10-31 | 索尼公司 | Recording system, data recording device, memory device, and data recording method |
| US20030220974A1 (en) * | 2002-05-23 | 2003-11-27 | International Business Machines Corporation | Parallel metadata service in storage area network environment |
| CN1567237A (en) * | 2003-06-09 | 2005-01-19 | 浪潮电子信息产业股份有限公司 | Method for constructing high-available distributed memory system |
| CN1619522A (en) * | 2003-11-17 | 2005-05-25 | 联想(北京)有限公司 | Computer group file service system and its input output treatment method |
| US20100082774A1 (en) * | 2005-09-09 | 2010-04-01 | Pitts William M | Distributed File System Consistency Mechanism Extension for Enabling Internet Video Broadcasting |
| CN101149755A (en) * | 2007-10-25 | 2008-03-26 | 中国科学院计算技术研究所 | Distributed file system file writing system and method |
| CN101382948A (en) * | 2008-10-14 | 2009-03-11 | 成都市华为赛门铁克科技有限公司 | File memory method, apparatus and system |
| CN101567001A (en) * | 2009-05-22 | 2009-10-28 | 清华大学 | Method for managing metadata file layout of parallel file system |
| CN101567003A (en) * | 2009-05-27 | 2009-10-28 | 清华大学 | Method for managing and allocating resource in parallel file system |
| CN102073739A (en) * | 2011-01-25 | 2011-05-25 | 中国科学院计算技术研究所 | Method for reading and writing data in distributed file system with snapshot function |
Non-Patent Citations (2)
| Title |
|---|
| 冯幼乐: ""分布式文件***元数据管理技术研究与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 耿亮: ""SAN并行文件***中文件布局的设计与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102929915A (en) * | 2012-09-18 | 2013-02-13 | 浪潮(北京)电子信息产业有限公司 | Method and system for managing files in trunking system |
| CN104378400A (en) * | 2013-08-15 | 2015-02-25 | 腾讯科技（深圳）有限公司 | Data dispersion and concurrence method and device |
| CN104424292A (en) * | 2013-09-02 | 2015-03-18 | 联想(北京)有限公司 | Data processing method and electronic device |
| CN104424292B (en) * | 2013-09-02 | 2020-04-24 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN103647850B (en) * | 2013-12-25 | 2017-01-25 | 北京京东尚科信息技术有限公司 | Data processing method, device and system of distributed version control system |
| CN108701087A (en) * | 2016-04-01 | 2018-10-23 | 英特尔公司 | The method and apparatus being sequentially written in for the block group for handling the physical block into memory device |
| CN108701087B (en) * | 2016-04-01 | 2024-01-26 | 英特尔公司 | Method and apparatus for handling sequential writes to a group of blocks of a physical block in a memory device |
| CN107967265A (en) * | 2016-10-18 | 2018-04-27 | 华为技术有限公司 | Access method, data server and the file access system of file |
| CN106980471A (en) * | 2017-03-27 | 2017-07-25 | 联想(北京)有限公司 | A kind of method and device for the hard disk write performance for improving smart machine |
| WO2018176811A1 (en) * | 2017-03-27 | 2018-10-04 | 联想(北京)有限公司 | Hard disk write-in method and apparatus |
| CN106980471B (en) * | 2017-03-27 | 2020-08-25 | 联想(北京)有限公司 | Method and device for improving hard disk writing performance of intelligent equipment |
| WO2019080015A1 (en) * | 2017-10-25 | 2019-05-02 | 华为技术有限公司 | Data reading and writing method and device, and storage server |
| US11397668B2 (en) | 2017-10-25 | 2022-07-26 | Huawei Technologies Co., Ltd. | Data read/write method and apparatus, and storage server |
| CN108491549B (en) * | 2018-04-09 | 2022-02-01 | 深圳市茁壮网络股份有限公司 | Method and device for creating file directory in distributed storage system |
| CN108491549A (en) * | 2018-04-09 | 2018-09-04 | 深圳市茁壮网络股份有限公司 | The creation method and device of file directory in a kind of distributed memory system |
| CN112115113A (en) * | 2020-09-25 | 2020-12-22 | 北京百度网讯科技有限公司 | Data storage system, method, device, equipment and storage medium |
| CN114302085A (en) * | 2021-12-23 | 2022-04-08 | 云从科技集团股份有限公司 | Data storage method and device, electronic equipment and storage medium |
| CN114302085B (en) * | 2021-12-23 | 2024-02-20 | 云从科技集团股份有限公司 | Data storage method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102622412A (en) | Method and device of concurrent writes for distributed file system | |
| US9792344B2 (en) | Asynchronous namespace maintenance | |
| TWI416323B (en) | Method,system and semiconductor device for management workload | |
| US8782368B2 (en) | Storing chunks in containers | |
| CN102934097B (en) | Data deduplication | |
| CN101741730B (en) | Method and equipment for downloading file and method and system for providing file downloading service | |
| US9189493B2 (en) | Object file system | |
| CN109710572B (en) | HBase-based file fragmentation method | |
| US9313270B2 (en) | Adaptive asynchronous data replication in a data storage system | |
| US10365845B1 (en) | Mapped raid restripe for improved drive utilization | |
| CN105787037B (en) | A kind of delet method and device of repeated data | |
| US8499131B2 (en) | Capping a number of locations referred to by chunk references | |
| US10235082B1 (en) | System and method for improving extent pool I/O performance by introducing disk level credits on mapped RAID | |
| US20220179585A1 (en) | Management of Idle Time Compute Tasks in Storage Systems | |
| CN103399823A (en) | Method, equipment and system for storing service data | |
| CN102142032A (en) | Method and system for reading and writing data of distributed file system | |
| CN110147203A (en) | A kind of file management method, device, electronic equipment and storage medium | |
| CN105653209A (en) | Object storage data transmitting method and device | |
| CN108958660B (en) | Distributed storage system and data processing method and device thereof | |
| CN103546580A (en) | File copy asynchronous writing method applied to distributed file system | |
| CN109840051A (en) | A kind of date storage method and device of storage system | |
| US10606501B2 (en) | Management of paging in compressed storage | |
| CN103064926A (en) | Data processing method and device | |
| US10063256B1 (en) | Writing copies of objects in enterprise object storage systems | |
| CN103186349B (en) | Block level distributed memory system and data read-write method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120801 |
|
| RJ01 | Rejection of invention patent application after publication |