CN102088484A - Write lock method of distributed file system and a system thereof - Google Patents
Write lock method of distributed file system and a system thereof Download PDFInfo
- Publication number
- CN102088484A CN102088484A CN2010105966195A CN201010596619A CN102088484A CN 102088484 A CN102088484 A CN 102088484A CN 2010105966195 A CN2010105966195 A CN 2010105966195A CN 201010596619 A CN201010596619 A CN 201010596619A CN 102088484 A CN102088484 A CN 102088484A
- Authority
- CN
- China
- Prior art keywords
- data block
- lock
- lower floor
- particular data
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/176—Support for shared access to files; File sharing support
- G06F16/1767—Concurrency control, e.g. optimistic or pessimistic approaches
- G06F16/1774—Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
- Storage Device Security (AREA)
Abstract
The present invention discloses a write lock method of distributed file system and a system thereof, wherein the method comprises the following steps: when an upper-layer data-location register server receives a request for opening a file sent from a file request client, the upper-layer data-location register server extracts the data block path information of the file and returns the same to the file request client; a lower-layer lock server returns the write lock authority information of the specific data portion of the specific data block to the file request client according to a lock request of the specific data portion of the specific data block in the file, wherein the lock request is sent from the file request client. The write lock method of distributed file system and a system thereof provided the present invention not only can improve throughput of a distributed file system lock write submission without any increase in hardware investment, but also can carry out synchronous capacity expansion with the capacity expansion of the distributed file system and possess an excellent expandability.
Description
Technical field
The present invention relates to field of data storage, specifically, what relate in particular to a kind of distributed file system writes locking means and system.
Background technology
On large-scale distributed file system, in order to accomplish the concurrency of magnanimity, guarantee simultaneously the consistency of file system data again, normally select a unified latch node that the data of storing in the distributed file system are concentrated lock management, this latch node can be single server or cluster, for ease of statement, hereinafter abbreviate latch node (lock server or cluster) as lock server.Yet along with the fast development of multimedia industry, for many-sided considerations such as cost, reliabilities, for large-scale distributed file system, increasing manufacturer or enterprise adopt the mode of cloud to store.The cloud storage is meant by functions such as cluster application or distributed file systems, various dissimilar memory devices a large amount of in the network is gathered collaborative work by application software, the common system that storage and Operational Visit function externally are provided that is to say that it is the cloud computing system of core with storage and management that the cloud storage is one.In the cloud storage, should consider fail safe, also to consider cost, therefore, it is crucial how disposing cloud.Especially present, the scale of cloud is increasing, and increasing cloud will be distributed in a plurality of places, and lock server how to dispose distributed file system afterwards will become complicated unusually.
In the cloud storage, if use single lock server to come the core of locked out distribution formula file system.When a plurality of programs are visited the different document misregistration amount of different files in the distributed system simultaneously, need visit this lock server successively to obtain that the specific document misregistration amount of specific file write the lock authority.At this moment, limit the visit handling capacity of distributed file system greatly, caused throughput performance to descend.And when a plurality of programs are visited the identical file side-play amount of same file in the distributed system simultaneously, earlier the program of visit obtained this specific file specific document misregistration amount write the lock authority.So this moment, other access program can only be waited in buffer memory.Have only in the ban the program of visit to discharge the writing when locking authority of specific document misregistration amount of this specific file, what other program could obtain the specific document misregistration amount of this specific file of data writes the lock authority.Therefore, single lock server becomes the bottleneck of writing for huge cloud.
Summary of the invention
The present invention is directed to the deficiencies in the prior art, what a kind of distributed file system was provided writes locking means and system, not only can guarantee under the situation that the hardware input does not increase, improve the handling capacity that the distributed file system locking is submitted to, and can also be along with the dilatation of distributed file system dilatation synchronously, have good expandability.
For solving the problems of the technologies described above, the invention provides following technical scheme:
A kind of distributed file system write locking means, may further comprise the steps:
When upper layer data location register server receive that the file request client sends write the request of opening certain file the time, upper layer data location register server extracts the data block routing information of described file, and returns to described file request client;
The locking request to given data piece particular data part in the described file that lower floor's lock server is sent according to described file request client is locked authority information with writing of described certain data block particular data part and is returned to described file request client.
In an embodiment of the present invention, when described upper layer data location register server extracts the data block routing information of described file and data block routing information that discovery does not have described file, described upper layer data location register server then at first is the unique complete data block path of described document creation, and write down the data block routing information of described file, thereby the lock authority of writing of the different pieces of information piece on the data block path of described file is distributed in the described lower floor lock server.
In an embodiment of the present invention, described file request client is before the locking request that sends to described lower floor lock server described certain data block particular data part, also judge the lock condition of described particular data part in described file request client terminal local buffer memory, if described particular data part is in described file request client terminal local buffer memory and locked, then described file request client does not send locking request to described lower floor lock server, if described particular data part is not locked, then described file request client sends locking request to described lower floor lock server.
In an embodiment of the present invention, described lower floor lock server is created data block lock state information table according to the locking request of sending to described certain data block particular data part of described file request client, is specially:
When described lower floor lock server receive that described file request client sends to the locking request of described certain data block particular data part the time, described lower floor lock server judges whether current be the locking request that receives for the first time described certain data block, if then in described data block lock state information table, add described certain data block and corresponding locking request information thereof; If not, then in described data block lock state information table, retrieve the locking request information of described certain data block particular data part.
In an embodiment of the present invention, described lower floor lock server according to the locking request of the described file request client that receives and return to described file request client described certain data block particular data part write the lock authority information, judge whether to notify the lower layer data block server that the needed described particular data of described file request client is partly sent to described file request client, be specially:
When described lower floor lock server receive that described file request client sends to the read-write locking request of described certain data block particular data part the time, described lower floor lock server is checked the lock condition of particular data part described in the current data block locking information table, if described particular data part is not locked, then in described data block lock state information table, described particular data lock-out state partly is labeled as locking, and notice lower layer data block server returns described particular data partly to described file request client; If described particular data part is locked, then carry out locking failure strategy;
When described lower floor lock server receive that described file request client sends to described certain data block particular data part write locking request the time, described lower floor lock server is then checked the lock condition of current described particular data part, if described particular data part is not locked, then in described data block status information table, the lock-out state of described particular data part is labeled as locking, and returns and lock successfully described file request client; If described particular data part is locked, then carry out locking failure strategy.
In an embodiment of the present invention, described file request client sends the read-write locking request of described certain data block particular data part or before writing locking request to described lower floor lock server, also check the hit situation of described particular data part in local cache, if do not have described particular data part in the described local cache, then judge and do not hit described particular data part, and send the read-write locking request to described lower floor lock server; If in the described local cache described certain data block and not locked is arranged, judge that then described particular data partly hits, and write locking request to the lock server transmission of described lower floor.
Simultaneously, the present invention also provides a kind of locking system of writing of distributed file system, comprises that being arranged at intrasystem upper layer data location register selects path module, lower floor's data interlock module and lower floor's data memory module; The upper layer data location register selects path module to be used to receive the request of opening certain file of writing that the file request client sends, and the data block routing information that extracts described file returns to described file request client; Lower floor's data interlock module is used for the locking request to described file certain data block particular data part sent according to described file request client, and the lock authority information of writing of described certain data block particular data part is returned to described file request client.
In an embodiment of the present invention, described upper layer data location register selects path module when data block routing information that extracts described file and data block routing information that discovery does not have described file, described upper layer data location register selects path module also to be used to the unique complete data block path of described document creation, and write down the data block routing information of described file, thereby the lock authority of writing of the different pieces of information piece on the data block path of described file is distributed in the described lower floor lock server.
In an embodiment of the present invention, described file request client is before the locking request that sends to described lower floor data interlock module described certain data block particular data part, also judge the lock condition of described particular data part in described file request client terminal local buffer memory, if described particular data part is in described file request client terminal local buffer memory and locked, then described file request client does not send locking request to described lower floor data interlock module, if described particular data part is not locked, then described file request client sends locking request to described lower floor data interlock module.
In an embodiment of the present invention, the locking request to described certain data block particular data part that described lower floor data interlock module is sent according to described file request client is created described data block lock state information table, is specially:
When described lower floor data interlock module receive that described file request client sends to the locking request of described certain data block particular data part the time, described lower floor data interlock module judges whether current be the locking request that receives for the first time described certain data block, if then in described data block lock state information table, add described certain data block and corresponding locking request information thereof; If not, then in described data block lock state information table, retrieve the locking request information of described particular data part.
In an embodiment of the present invention, described lower floor data interlock module according to the locking request of the described file request client that receives and return to described file request client described certain data block particular data part write the lock authority information, judge whether to notify the lower layer data block server that the needed described particular data of described file request client is partly sent to described file request client, be specially:
When described lower floor data interlock module receive that described file request client sends to the Read-Write Locks request of described certain data block particular data part the time, the lock condition of the part of particular data described in the described lower floor data interlock module check current data block lock state information table, if described particular data part is not locked, then in described data block lock state information table, described particular data lock-out state partly is labeled as locking, and notice lower floor data memory module returns described particular data partly to described file request client; If described particular data part is locked, then carry out locking failure strategy;
When described lower floor data interlock module receive that described file request client sends to described certain data block particular data part write locking request the time, described lower floor data interlock module is then checked the lock condition of the part of particular data described in the current described data block lock state information table, if described particular data part is not locked, then in described data block status information table, the lock-out state of described particular data part is labeled as locking, and returns and lock successfully described file request client; If described certain data block is locked, then carry out locking failure strategy.
In an embodiment of the present invention, described file request client sends the read-write locking request of described certain data block particular data part or before writing locking request to described lower floor data interlock module, also check the hit situation of described particular data part in local cache, if do not have described particular data part in the described local cache, then judge and do not hit described particular data part, and send the read-write locking request to described lower floor lock server; If described particular data part and not locked is arranged in the described local cache, then judge and hit described particular data part, and write locking request to the lock server transmission of described lower floor.
Owing to adopted above technical scheme, the beneficial effect that the present invention is possessed is: be the definite unique data block path in one or more file of file request client-requested by upper layer data location register server, thereby the lock authority of writing of the different pieces of information piece on the data block path of described file is distributed in lower floor's lock server, and the data block routing information of described file is returned to described file request client; The file request client is according to the described data block routing information that obtains from upper layer data location register server, initiate to write the solicited message of opening some certain data block particular data parts in the described file to the lower floor lock server, and obtain the locked authority information of writing of described certain data block in the corresponding lower floor lock server.So, upper layer data memory location storage server is distributed to the lock authority of writing of all data blocks of described file in one or more lower floor's lock server of lower floor, abandon the single lock server of traditional employing and come the locking means of locked out distribution formula file system, reduced the load of the lock server of upper layer data location register server on the one hand; On the other hand, also realize veritably corresponding lower layer data block server memory storage in lower floor's lock server managing distributing type system data write the lock authority, thereby under the situation that does not increase the distributed file system hardware resource, improve the handling capacity that the distributed file system locking is submitted to.Simultaneously, use locking means provided by the invention, can be used in the distributed file system of one or more upper layer data location register server, therefore, the ability of the raising throughput of system of this locking means can also be along with the dilatation of distributed file system dilatation synchronously, have good expandability.
Description of drawings
Fig. 1 is the falsework composition of an embodiment of the present invention;
Fig. 2 is the workflow diagram of the another kind of embodiment of the present invention.
Embodiment
In conjunction with the accompanying drawings the present invention is described in further detail below by embodiment.
At present, in order to solve the bottleneck problem that huge cloud that single lock server causes is write, as shown in Figure 1, what the invention provides a kind of distributed file system writes locking means and system, it is mainly conceived: in the lower floor of distributed file system, use the data of storing in the next common locked out distribution formula file system of lock server of a plurality of dispersions.Compare with traditional single lock server, the locked authority of the data of storing in the distributed file system that the present invention will lock is shared in a plurality of different file system data server of lower floor.Specifically, this document system data server mainly comprises lower floor's lock server and lower layer data block server, wherein, the lower layer data block server is mainly used to the data block information in the distributed storage file system, lower floor's lock server then mainly is to be used for locking to be stored in the lower layer data block data in server, and promptly the locked authority to the data in this data block server manages.What the locked authority of saying here mainly referred to data in the distributed file system writes the lock authority.Specifically, this writes locking means is that the file of one or more file of opening for the file request client-requested by upper layer data location register server is determined unique full block of data path, thereby the lock authority of writing of the different pieces of information piece on the data block path of this document is distributed in a plurality of lower floors lock server.In fact, in upper layer data location storage server, store a large amount of data block routing informations that are stored in the lower layer data block server, in internal system, the data block routing information that also will be stored in the lower layer data block server is called metadata.At upper layer data location register server place, adopt special lock server to lock metadata.And, then use a plurality of lower floors lock server that disperses to lock the data block that is stored in the lower layer data block server in lower floor.When lower floor's lock server receive that the file request client sends to this document in during the locking request of certain data block particular data part, lower floor's lock server just returns to the file request client with the lock authority information of writing of described certain data block particular data part.
The locking means of writing provided by the invention adopts a plurality of lower floors lock server that disperses to come the purpose of the data in the locked out distribution formula file system, mainly is the load for the lock server that reduces upper layer data location register server.Simultaneously, the lock authority of writing of the different pieces of information piece in the file of the file request client-requested being opened by upper layer data location register server is distributed in lower floor's lock server, make that the file system data server of lower floor no longer is a buffering of upper strata lock server, that has really realized data in the managing distributed file system of lower floor's lock server writes the lock authority.This is write locking means and can use widely in single or many upper layer datas location register server, if when using in a plurality of upper layer data location register servers, each upper layer data location register server does not have active and standby with relation.
In an embodiment of the present invention, when described upper layer data location register server extracts the data block routing information of certain specific file and data block routing information that discovery does not have this document, for example there is not to find to have the data block routing information of file A, show that then this moment, upper layer data location register server was to receive the request of opening of writing to this specific file A that the file request client is sent for the first time, so, upper layer data location register server then at first is that this specific file A creates a unique complete data block path, and the data block routing information of log file A.By the establishment in file data blocks path, thereby the lock authority of writing of the different pieces of information piece on the data block path of file A is distributed in the described lower floor lock server, thus the locking of realization the most fine-grained (byte level).Wherein the routing information of all data blocks in this document A can be described in this data block path, mainly comprises which data block and these data block store being arranged on this document data block path in which lower layer data block server.Need to prove that at this moment, what lower floor's lock server and not knowing oneself had been managed these data blocks writes the lock authority.
When the file request client being arranged again when the request of the A that opens file is write in the transmission of upper layer data location register server, because this moment, created in the unique complete data block path of file A, therefore, upper layer data location register server can extract the data block routing information of file A, and returns to the file request client.Should be noted that to send file request client that writing the A request that opens file for the first time and send the file request client of writing the A request that opens file the second time to be same file request client, also can be different file request clients.When these two file request clients need be write the same data block that locks among this document A, they can visit the same lower layer data block server of this data block of storage simultaneously.
In an embodiment of the present invention, the file request client was also judged the lock condition of this particular data part in file request client terminal local buffer memory before the locking request to certain data block particular data part in the lock server transmission file A of lower floor.At this,, abbreviate certain data block as data block B for explaining conveniently.If data block B particular data part is in file request client terminal local buffer memory and locked by this document requesting client, then this document requesting client does not need to send locking request to lower floor's lock server, can directly begin the operation of write data piece B particular data part; If data block B particular data part is not locked, have two kinds of situations this moment, a kind of is not have data block B particular data part in the local cache of this document requesting client, another kind of situation then is that data block B particular data part is arranged in the local cache of this document requesting client, it is locked that but this particular data does not partly have, for both of these case, this document requesting client all can send locking request to lower floor's lock server.
In an embodiment of the present invention, the locking request that lower floor's lock server is sent according to the file request client to this certain data block particular data part, create data block lock state information table, concrete mode is: when the lock server C of lower floor receive that the file request client sends to the locking request of this data block B particular data part the time, the lock server C of lower floor need judge whether current be to receive the locking request to this data block B that the file request client is sent for the first time, if, just know the lock server C of lower floor at this moment oneself managed this data block B write the lock authority.Also show simultaneously in the available data piece lock state information table that is stored in the lock server C of lower floor also there is not the relevant information of data block B.This moment, the lock server C of lower floor added clauses and subclauses and the corresponding locking request information of data block B thereof of this data block B in data block lock state information table; If not, then lower floor's lock server C only need the locking request information of retrieves data blocks B particular data part get final product in current data block lock state information table.
In an embodiment of the present invention, suppose that the lock server C of lower floor receives the locking request to data block B particular data part that the file request client is sent for the first time, that is to say, at this moment, the lock server C of lower floor can be according to the variable locking request retrieves data blocks B particular data relevant lock authority of writing partly of the file request client that receives, and return to this document requesting client data block B particular data part write the lock authority information, judge whether to notify the lower layer data block server of the lock server C of lower floor correspondence that the needed particular data of file request client is partly sent to this document requesting client simultaneously, specifically: when lower floor's lock server receive that the file request client sends to the read-write locking request of data block B particular data part D the time, wherein, for the user, this particular data part D is the side-play amount of file A.Lower floor's lock server is checked the lock condition of particular data part D in the current data block locking information table, if particular data part D is not locked, at this moment, show that particular data part D is not also locked by other programs, then in data block lock state information table, the lock-out state of particular data part D is labeled as locking, and notice lower layer data block server returns particular data part D and gives the file request client; Need to prove, for same data block, wherein can comprise a lot of data messages, may be a few page informations in numerous data messages and the file request client need lock.At this, these data messages are divided into a plurality of parts, the file request client need lock so then is some parts in a plurality of parts.If when two file request clients were write data divisions different among the same data block B simultaneously, then these two file request clients can be carried out write operation simultaneously.In other words, from user's angle, if two users need write locking is document misregistration amounts different in the same file, these two users just can carry out write operation simultaneously so.If particular data part D is locked, at this moment, show that particular data part D is locked by other programs, then use corresponding locking failure strategy;
When lower floor's lock server receive that the file request client sends to data block B particular data part write locking request the time, lower floor's lock server is checked the lock condition of particular data part in the current data block locking information table, if the particular data part is not locked, then in the data block status information table, the lock-out state of particular data part is labeled as locking, and returns and lock successfully described file request client; If the particular data part is locked, then use corresponding locking failure strategy, specifically, locking failure strategy comprises locking failure or other strategy.
Furtherly, when the file request client receives after data block B particular data partly locks successful message, return to and operate the user writing the successful message of locking, the user just can partly carry out data block B particular data and begin the operation writing or revise.If the file request client receives execution to the data data division when locking the failure strategy accordingly, the file request client can be selected to wait for and continue to lock, perhaps directly return and write the locking failure to the operation user, at this moment, the authority then data block B particular data partly not write of user.
In fact, the downward layer data storage server of file request client C sends the read-write locking request of data block B particular data part or writes before the locking request, also checks the hit situation of particular data part in local cache.If do not have the particular data part in the local cache, the data that do not have the storage file requesting client to lock in the local cache promptly are described, then judge and do not hit the certain data block part, and send the read-write locking request to the lock server C of lower floor; If in the local cache particular data part is arranged, before promptly showing the file request client data block B particular data was partly carried out read operation or write operation, just this moment, this particular data part was not also write locking by the file request client, then judge and hit this particular data part, and write locking request to the lock server C of lower floor transmission.That is to say,, when it asks this particular data part again, then only need write locking request and get final product to lower floor's lock server transmission if before a file client certain data block was partly carried out read operation or write operation.
As shown in Figure 2, be the method flow diagram of an embodiment of the present invention, in conjunction with Fig. 2, the embodiment of the invention is explained in detail.For explaining conveniently, need with the file request client that data block B particular data partly is that example is elaborated among the lock file A, wherein, Fac represents the file request client, Fas represents the file system server (comprising lower layer data block server and lower floor's lock server) of lower floor, Flr represents upper layer data location register server, and concrete steps are as follows:
S1: the solicited message of writing the A that opens file that file request client Fac sends to upper layer data location register server Flr;
S2: the solicited message of writing the A that opens file that upper layer data location register server Fl r reception file request client Fac sends and the data block routing information of extraction document A, be chunk information, if Flr extracts the chunk information less than file A, Flr judges that current is to receive the request of writing the A that opens file that the file request client is sent for the first time, and for file A creates intactly unique complete chunk of interior all data blocks of description document A, and write down this chunk information.If Flr can extract the chunk information of file A, the chunk information of the direct extraction document A of Flr then, and return to the file request client;
The S3:Fac judgment data piece B particular data part lock condition in local cache is if the certain data block part in the Fac local cache and locked, then forwards step S4 to; If it is locked that particular data does not partly have, then forward step S5 to;
S4: judged that the particular data part is locked, then Fac returns the Subscriber Locked success, can carry out the operation of write data piece B particular data part;
If the hit situation of S5:Fac judgment data piece B particular data part in local cache miss, then forwards S6 to, if hit, then forwards S9 to;
The lower floor data storage server of S6:Fac in Fas sends the request of Read-Write Locks given data piece B particular data part;
S7:Fas checks whether the particular data part is locked among the current data block lock state information list data block B, if it is not locked, lower floor's lock server is labeled as locking with the particular data lock-out state partly of data block B in data block lock state information table, and returns this particular data and partly give the file request client; If locked, then carry out locking failure strategy;
S8:Fac checks the state that fas returns, if return the particular data part that needs, and the then operation of returning the locking success and beginning to write to the user; If receiving to carry out locks the failure strategy, then can after returning the locking failure or wait for a period of time, the user continue the request locking.
The lower floor data storage server of S9:Fac in Fas sends the request of writing locking data piece B particular data part;
S10:Fas checks whether the particular data part is locked among the current data block lock state information list data block B, if it is not locked, lower floor's lock server is labeled as locking with the particular data lock-out state partly of data block B in data block lock state information table, and returns locking and successfully give the file request client; If locked, then carry out locking failure strategy;
S11:Fac checks the state that fas returns, and locks successfully if return, then the operation of returning the locking success and beginning to write to the user; If receiving to carry out locks the failure strategy, then can after returning the locking failure or wait for a period of time, the user continue the request locking.
Simultaneously, the invention allows for a kind of locking system of writing of distributed file system, comprise that being arranged at intrasystem upper layer data location register selects path module, lower floor's data interlock module and lower floor's data memory module; The upper layer data location register selects path module to be used to receive the request of opening certain file of writing that the file request client sends, and the data block routing information that extracts described file returns to described file request client; Lower floor's data interlock module is used for the locking request to described file certain data block particular data part sent according to described file request client, and the lock authority information of writing of described certain data block particular data part is returned to described file request client.
Need to prove that in the above-described embodiment, just in order to express easily, just needing locking data piece B particular data with the file request client partly is that example describes.In fact, in distributed file system, store a large amount of data in the file system data server of lower floor with the storage of data block form.Upper layer data location register server is divided into a plurality of data blocks with identical file, and be stored in same or different lower floor's file system data servers, normally being stored in the different file system data servers, specifically is to be stored in the data block server.That is to say, can store the data block that much comes from different files in the same lower floor file system data server.In case and upper layer data location register server has been determined a unique complete data block path of certain file, so, when all file request clients need lock certain particular data part of some data blocks in this document, all can only and carry out corresponding lock operation by this unique data block path access.That is to say this moment, the locked authority of this data block has been distributed in lower floor's file system data server.And upper layer data location register server is receiving writing of file request client when opening the locking request of certain file, only can select a copy data piece path determining, so itself does not improve the load of upper layer data location register server.Henceforth, all it doesn't matter with upper layer data location register server to the locking of writing of certain data block in the distributed file system.
And because lower floor's file system data server is more, so the data block of each file system data server admin is limited, therefore, the data block that each file system data server need be managed write the lock authority also be limited, so the load of lower floor's file system data server can not improve too much yet, therefore can not influence the operate as normal of lower floor's file system data server.Shared the locking load of single lock server in the existing distributed file system by adopting a plurality of file system data servers, under the situation that does not improve the hardware input of this distributed file system own, improved the throughput that the distributed file system locking is submitted to.In actual applications, can select a data location register server (single master), also can select a plurality of Data Position registrar server (many master).If use a plurality of Data Position registrar server on the upper strata, and the data in the distributed file system are write in the lower floor's file system data server that is distributed to magnanimity of lock authority by a plurality of Data Position registrar server, so along with the growth of lower floor's file system data server in the distributed file system, this locking system can be followed the dilatation of distributed file system and dilatation, thereby has favorable expansibility.
Writing under the prerequisite of opening same file, except for the first time to the file request client end of opening this document request is write in the transmission of upper layer data location register server, other each file request client all only need with the communication of upper layer data location register server once, need just can to obtain the data block routing information of the file at the data division place that locks.If what this document requesting client need obtain this data block once more writing lock during authority, this document requesting client only needs downward layer data storage server to send locking request, and what can obtain this data block in lower floor's lock server writes the lock authority.So just reduced the number of times of file request client and upper strata location register server communication.Wherein, this data block write that the lock authority comprises all data of storing in this data block write the lock authority.And for same file request client, only need carry out a communication with the data location register on upper strata gets final product, make upper layer data location register server can keep basically and do not support the file system of writing lock that same performance is arranged, thereby reduced unnecessary communication-cost.
Above content be in conjunction with concrete execution mode to further describing that the present invention did, can not assert that concrete enforcement of the present invention is confined to these explanations.For the general technical staff of the technical field of the invention, without departing from the inventive concept of the premise, can also make some simple deduction or replace, all should be considered as belonging to protection scope of the present invention.
Claims (12)
- A distributed file system write locking means, it is characterized in that, may further comprise the steps:When upper layer data location register server receive that the file request client sends write the request of opening certain file the time, upper layer data location register server extracts the data block routing information of described file, and returns to described file request client;The locking request to particular data part in the certain data block in the described file that lower floor's lock server is sent according to described file request client is locked authority information with writing of described certain data block particular data part and is returned to described file request client.
- 2. the method for claim 1, it is characterized in that, when described upper layer data location register server extracts the data block routing information of described file and data block routing information that discovery does not have described file, described upper layer data location register server then at first is the unique complete data block path of described document creation, and write down the data block routing information of described file, thereby the lock authority of writing of the different pieces of information piece on the data block path of described file is distributed in the described lower floor lock server.
- 3. the method for claim 1, it is characterized in that, described file request client is before the locking request that sends to described lower floor lock server described certain data block particular data part, also judge the lock condition of described particular data part in described file request client terminal local buffer memory, if described particular data part is in described file request client terminal local buffer memory and locked, then described file request client does not send locking request to described lower floor lock server, if described particular data part is not locked, then described file request client sends locking request to described lower floor lock server.
- 4. method as claimed in claim 3 is characterized in that, the locking request to described certain data block particular data part that described lower floor lock server is sent according to described file request client is created data block lock state information table, is specially:When described lower floor lock server receive that described file request client sends to the locking request of described certain data block particular data part the time, described lower floor lock server judges whether current be the locking request that receives for the first time described certain data block, if then in described data block lock state information table, add described certain data block and corresponding locking request information thereof; If not, then in described data block lock state information table, retrieve the locking request information of described certain data block particular data part.
- 5. method as claimed in claim 4, it is characterized in that, described lower floor lock server according to the locking request of the described file request client that receives and return to the described certain data block particular data part of described file request client write the lock authority information, judge whether to notify the lower layer data block server that the needed described particular data of described file request client is partly sent to described file request client, be specially:When described lower floor lock server receive that described file request client sends to the read-write locking request of described certain data block particular data part the time, described lower floor lock server is checked the lock condition of particular data part described in the current data block locking information table, if described particular data part is not locked, then in described data block lock state information table, described particular data lock-out state partly is labeled as locking, and notice lower layer data block server returns described particular data partly to described file request client; If described particular data part is locked, then carry out locking failure strategy;When described lower floor lock server receive that described file request client sends to described certain data block particular data part write locking request the time, described lower floor lock server is then checked the lock condition of the part of particular data described in the current described data block lock state information table, if described particular data part is not locked, then in described data block status information table, the lock-out state of described particular data part is labeled as locking, and returns and lock successfully described file request client; If described particular data part is locked, then carry out locking failure strategy.
- 6. method as claimed in claim 5, it is characterized in that, described file request client sends the read-write locking request of described certain data block particular data part or before writing locking request to described lower floor lock server, also check the hit situation of described particular data part in local cache, if do not have described particular data part in the described local cache, then judge and do not hit described particular data part, and send the read-write locking request to described lower floor lock server; If described particular data part and not locked is arranged in the described local cache, then judge and hit described particular data part, and write locking request to the lock server transmission of described lower floor.
- A distributed file system write locking system, it is characterized in that, comprise that being arranged at intrasystem upper layer data location register selects path module, lower floor's data interlock module and lower floor's data memory module; The upper layer data location register selects path module to be used to receive the request of opening certain file of writing that the file request client sends, and the data block routing information that extracts described file returns to described file request client; Lower floor's data interlock module is used for the locking request to described file certain data block particular data part sent according to described file request client, and the lock authority information of writing of described certain data block particular data part is returned to described file request client.
- 8. method as claimed in claim 7, it is characterized in that, described upper layer data location register selects path module when data block routing information that extracts described file and data block routing information that discovery does not have described file, described upper layer data location register selects path module also to be used to the unique complete data block path of described document creation, and write down the data block routing information of described file, thereby the lock authority of writing of the different pieces of information piece on the data block path of described file is distributed in the described lower floor lock server.
- 9. method as claimed in claim 7, it is characterized in that, described file request client is before the locking request that sends to described lower floor data interlock module described certain data block particular data part, also judge the lock condition of described particular data part in described file request client terminal local buffer memory, if described particular data part is in described file request client terminal local buffer memory and locked, then described file request client does not send locking request to described lower floor data interlock module, if described particular data part is not locked, then described file request client sends locking request to described lower floor data interlock module.
- 10. method as claimed in claim 9, it is characterized in that, the locking request to described certain data block particular data part that described lower floor data interlock module is sent according to described file request client is created described data block lock state information table, is specially:When described lower floor data interlock module receive that described file request client sends to the locking request of described certain data block particular data part the time, described lower floor data interlock module judges whether current be the locking request that receives for the first time described certain data block, if then in described data block lock state information table, add described certain data block and corresponding locking request information thereof; If not, then in described data block lock state information table, retrieve the locking request information of described particular data part.
- 11. method as claimed in claim 10, it is characterized in that, described lower floor data interlock module according to the locking request of the described file request client that receives and return to described file request client described certain data block particular data part write the lock authority information, judge whether to notify the lower layer data block server that the needed described particular data of described file request client is partly sent to described file request client:When described lower floor data interlock module receive that described file request client sends to the Read-Write Locks request of described certain data block particular data part the time, the lock condition of the part of particular data described in the described lower floor data interlock module check current data block lock state information table, if described particular data part is not locked, then in described data block lock state information table, described particular data lock-out state partly is labeled as locking, and notice lower floor data memory module returns described particular data partly to described file request client; If described particular data part is locked, then carry out locking failure strategy;When described lower floor data interlock module receive that described file request client sends to described certain data block particular data part write locking request the time, described lower floor data interlock module is then checked the lock condition of the part of particular data described in the current described data block lock state information table, if described particular data part is not locked, then in described data block status information table, the lock-out state of described particular data part is labeled as locking, and returns and lock successfully described file request client; If described certain data block is locked, then carry out locking failure strategy.
- 12. method as claimed in claim 11, it is characterized in that, described file request client sends the read-write locking request of described certain data block particular data part or before writing locking request to described lower floor data interlock module, also check the hit situation of described particular data part in local cache, if do not have described particular data part in the described local cache, then judge and do not hit described particular data part, and send the read-write locking request to described lower floor lock server; If described particular data part and not locked is arranged in the described local cache, then judge and hit described particular data part, and write locking request to the lock server transmission of described lower floor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010596619.5A CN102088484B (en) | 2010-12-20 | 2010-12-20 | Write lock method of distributed file system and a system thereof |
| PCT/CN2011/083694 WO2012083797A1 (en) | 2010-12-20 | 2011-12-08 | Write lock method and system for distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010596619.5A CN102088484B (en) | 2010-12-20 | 2010-12-20 | Write lock method of distributed file system and a system thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102088484A true CN102088484A (en) | 2011-06-08 |
| CN102088484B CN102088484B (en) | 2014-12-10 |
Family
ID=44100096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010596619.5A Active CN102088484B (en) | 2010-12-20 | 2010-12-20 | Write lock method of distributed file system and a system thereof |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102088484B (en) |
| WO (1) | WO2012083797A1 (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012083797A1 (en) * | 2010-12-20 | 2012-06-28 | 中兴通讯股份有限公司 | Write lock method and system for distributed file system |
| CN103095686A (en) * | 2012-12-19 | 2013-05-08 | 华为技术有限公司 | Hot metadata access control method and server |
| CN104281442A (en) * | 2013-07-12 | 2015-01-14 | 富泰华工业（深圳）有限公司 | Document processing system and document processing method |
| CN104471528A (en) * | 2012-04-23 | 2015-03-25 | 谷歌公司 | Associating a file type with an application in a network storage service |
| CN104573428A (en) * | 2013-10-12 | 2015-04-29 | 方正宽带网络服务股份有限公司 | Method and system for improving resource effectiveness of server cluster |
| CN105224255A (en) * | 2015-10-14 | 2016-01-06 | 浪潮(北京)电子信息产业有限公司 | A kind of storage file management method and device |
| CN105320512A (en) * | 2014-07-30 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Method and device for distributing trunk permission |
| CN105681367A (en) * | 2014-11-17 | 2016-06-15 | ***通信集团公司 | Method, server and client for sharing file contents |
| CN103647850B (en) * | 2013-12-25 | 2017-01-25 | 北京京东尚科信息技术有限公司 | Data processing method, device and system of distributed version control system |
| CN106855871A (en) * | 2015-12-09 | 2017-06-16 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of Data Migration |
| CN107146075A (en) * | 2016-03-01 | 2017-09-08 | 阿里巴巴集团控股有限公司 | Request processing method and device |
| CN108647285A (en) * | 2018-05-04 | 2018-10-12 | 北京小米移动软件有限公司 | Renaming method, apparatus and distributed file system in distributed file system |
| CN111274208A (en) * | 2018-12-05 | 2020-06-12 | 杭州海康威视***技术有限公司 | Method and device for locking file |
| CN116055565A (en) * | 2023-01-28 | 2023-05-02 | 北京蓝色星际科技股份有限公司 | Data transmission method, system, device, electronic equipment and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106559282A (en) * | 2015-09-29 | 2017-04-05 | 中兴通讯股份有限公司 | A kind of lock file management method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6925515B2 (en) * | 2001-05-07 | 2005-08-02 | International Business Machines Corporation | Producer/consumer locking system for efficient replication of file data |
| CN1771495A (en) * | 2003-05-07 | 2006-05-10 | 国际商业机器公司 | Distributed file serving architecture system |
| CN101706802A (en) * | 2009-11-24 | 2010-05-12 | 成都市华为赛门铁克科技有限公司 | Method, device and sever for writing, modifying and restoring data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088484B (en) * | 2010-12-20 | 2014-12-10 | 南京中兴新软件有限责任公司 | Write lock method of distributed file system and a system thereof |
-
2010
- 2010-12-20 CN CN201010596619.5A patent/CN102088484B/en active Active
-
2011
- 2011-12-08 WO PCT/CN2011/083694 patent/WO2012083797A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6925515B2 (en) * | 2001-05-07 | 2005-08-02 | International Business Machines Corporation | Producer/consumer locking system for efficient replication of file data |
| CN1771495A (en) * | 2003-05-07 | 2006-05-10 | 国际商业机器公司 | Distributed file serving architecture system |
| CN101706802A (en) * | 2009-11-24 | 2010-05-12 | 成都市华为赛门铁克科技有限公司 | Method, device and sever for writing, modifying and restoring data |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012083797A1 (en) * | 2010-12-20 | 2012-06-28 | 中兴通讯股份有限公司 | Write lock method and system for distributed file system |
| CN104471528A (en) * | 2012-04-23 | 2015-03-25 | 谷歌公司 | Associating a file type with an application in a network storage service |
| CN104471528B (en) * | 2012-04-23 | 2019-01-08 | 谷歌有限责任公司 | File type is associated with the application in network storage service |
| CN103095686A (en) * | 2012-12-19 | 2013-05-08 | 华为技术有限公司 | Hot metadata access control method and server |
| CN103095686B (en) * | 2012-12-19 | 2016-06-08 | 华为技术有限公司 | Focus metadata access control method and service device |
| CN104281442A (en) * | 2013-07-12 | 2015-01-14 | 富泰华工业（深圳）有限公司 | Document processing system and document processing method |
| CN104573428B (en) * | 2013-10-12 | 2018-02-13 | 方正宽带网络服务股份有限公司 | A kind of method and system for improving server cluster resource availability |
| CN104573428A (en) * | 2013-10-12 | 2015-04-29 | 方正宽带网络服务股份有限公司 | Method and system for improving resource effectiveness of server cluster |
| CN103647850B (en) * | 2013-12-25 | 2017-01-25 | 北京京东尚科信息技术有限公司 | Data processing method, device and system of distributed version control system |
| CN105320512A (en) * | 2014-07-30 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Method and device for distributing trunk permission |
| CN105320512B (en) * | 2014-07-30 | 2019-08-06 | 腾讯科技（深圳）有限公司 | Distribute the method and device of trunk permission |
| CN105681367A (en) * | 2014-11-17 | 2016-06-15 | ***通信集团公司 | Method, server and client for sharing file contents |
| CN105224255A (en) * | 2015-10-14 | 2016-01-06 | 浪潮(北京)电子信息产业有限公司 | A kind of storage file management method and device |
| CN105224255B (en) * | 2015-10-14 | 2018-10-30 | 浪潮(北京)电子信息产业有限公司 | A kind of storage file management method and device |
| CN106855871A (en) * | 2015-12-09 | 2017-06-16 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of Data Migration |
| CN106855871B (en) * | 2015-12-09 | 2020-04-07 | 阿里巴巴集团控股有限公司 | Data migration method and device |
| CN107146075A (en) * | 2016-03-01 | 2017-09-08 | 阿里巴巴集团控股有限公司 | Request processing method and device |
| CN107146075B (en) * | 2016-03-01 | 2020-11-10 | 创新先进技术有限公司 | Request processing method and device |
| CN108647285A (en) * | 2018-05-04 | 2018-10-12 | 北京小米移动软件有限公司 | Renaming method, apparatus and distributed file system in distributed file system |
| CN108647285B (en) * | 2018-05-04 | 2021-06-22 | 北京小米移动软件有限公司 | Renaming method and device in distributed file system and distributed file system |
| CN111274208A (en) * | 2018-12-05 | 2020-06-12 | 杭州海康威视***技术有限公司 | Method and device for locking file |
| CN116055565A (en) * | 2023-01-28 | 2023-05-02 | 北京蓝色星际科技股份有限公司 | Data transmission method, system, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102088484B (en) | 2014-12-10 |
| WO2012083797A1 (en) | 2012-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102088484B (en) | Write lock method of distributed file system and a system thereof | |
| CN102591970B (en) | Distributed key-value query method and query engine system | |
| US9405574B2 (en) | System and method for transmitting complex structures based on a shared memory queue | |
| US8566299B2 (en) | Method for managing lock resources in a distributed storage system | |
| US9069835B2 (en) | Organizing data in a distributed storage system | |
| CN103312624B (en) | A kind of Message Queuing Services system and method | |
| US8935560B2 (en) | System and method of file locking in a network file system federated namespace | |
| CN106936899B (en) | Configuration method of distributed statistical analysis system and distributed statistical analysis system | |
| US8832113B2 (en) | Data management apparatus and system | |
| WO2010072083A1 (en) | Web application based database system and data management method therof | |
| US9749445B2 (en) | System and method for updating service information for across-domain messaging in a transactional middleware machine environment | |
| CN105025053A (en) | Distributed file upload method based on cloud storage technology and system | |
| CN102314506B (en) | Based on the distributed buffering district management method of dynamic index | |
| WO2001084338A2 (en) | Cluster configuration repository | |
| CN111984696B (en) | Novel database and method | |
| WO2021057108A1 (en) | Data reading method, data writing method, and server | |
| CN103237046A (en) | Distributed file system supporting mixed cloud storage application and realization method thereof | |
| CN101576915A (en) | Distributed B+ tree index system and building method | |
| CN103310000A (en) | Metadata management method | |
| CN101771723A (en) | Data synchronization method | |
| CN103067461A (en) | Metadata management system of document and metadata management method thereof | |
| CN104750757B (en) | A kind of date storage method and equipment based on HBase | |
| CN104951475B (en) | Distributed file system and implementation method | |
| CN101778131A (en) | Data synchronization system | |
| CN100576848C (en) | Distribution-type data dynamic program agent method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |