CN103916465A - Data pre-reading device based on distributed file system and method thereof - Google Patents
Data pre-reading device based on distributed file system and method thereof Download PDFInfo
- Publication number
- CN103916465A CN103916465A CN201410108037.6A CN201410108037A CN103916465A CN 103916465 A CN103916465 A CN 103916465A CN 201410108037 A CN201410108037 A CN 201410108037A CN 103916465 A CN103916465 A CN 103916465A
- Authority
- CN
- China
- Prior art keywords
- data
- read
- module
- page
- anonymous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a data pre-reading device based on a distributed file system. The data pre-reading device comprises a client-side module, a metadata server module and a data memory module. The client-side module obtains the catalogue read extension authorization and a small file layout by accessing the metadata server module. Small file data and large granularity data of space continuity of the small file data are pre-read to a cache of the client-side module at the same time from the data memory module according to the small file layout. The invention further discloses a data pre-reading method based on the distributed file system.
Description
Technical field
The present invention relates to the interaction technique of distributed type file system client side and server end, particularly a kind of distributed file system is carried out the method and system of data pre-head between small documents in client.
Background technology
Along with information technology develop rapidly, global metadata informational capacity rapid growth, unstructured data is more and more.Add up according to Gartner, 2010 global metadata informational capacities reach 1.2ZB, and estimate to grow continuously and fast with annual at least 50% growth rate, wherein 85% is made up of various unstructured datas, and unstructured data is stored in distributed file system mainly with document form.In the emerging application such as web2.0, social networks, data message mainly exists with the form of small documents, and document size is less.Along with small documents quantity is more and more, effectively manage mass small documents in the urgent need to distributed file system, the small documents access services of high-performance, low delay is provided.
In large-scale distributed file system, the structure of metadata and data separating has become a kind of trend.Meta data server is responsible for maintaining file system global name space, and the mapping relations of management document logical place and disk physical location are file allocation physical store resource, and metadata record is in proprietary metadata store equipment.Client file data access needn't pass through meta data server, but adopts out-band method directly to access proprietary data storage device, thereby obtains higher data access performance.Said structure is very effective to large file access, large file is in proprietary data memory device, it is good that data are distributed in spatial continuity on memory device, and the situation of large file coarsegrain continuous data access is conventionally more, can give full play to the performance advantage of the coarsegrain sequential access of data storage device.
But in mass small documents data read access, due to data, to read data access spatial continuity between the little and different small documents of granularity poor, be difficult to the performance advantage of the coarsegrain sequential access of bringing into play data storage device, cause the access performance of mass small documents well below the access performance of large file, performance gap reaches more than 10 times.
Pre-reading is a main method that promotes small documents read access performance, by the data in advance that will access is read to client-cache, when subsequent client access, can in buffer memory, obtain data, save the synchronous expense from disk reading out data, reach lower access delay.The current algorithm that pre-reads carries out mainly for single large file, according to the logical place of historical access in Single document, predicts the follow-up logical place that will access, and pre-reads the data of the logical place that will access.But small documents once reads all files data conventionally, there is no historical visit information, therefore above-mentioned to pre-read algorithm invalid to small documents.In addition, pre-read for logical place in file, do not consider the spatial continuity that physical location reads, for the poor file of physical space continuity, can not bring into play the connected reference performance advantage of data storage device.
Summary of the invention
Technical problem to be solved by this invention is to provide a kind of pre-reader and method thereof based on distributed file system, cannot effectively reduce to overcome distributed file system in the mass small documents application scenarios existing in prior art the problem that small documents data read access postpones, cannot give full play to data storage device coarsegrain sequential access performance advantage.
For reaching above-mentioned purpose, the invention provides a kind of pre-reader based on distributed file system, it is characterized in that, described pre-reader comprises: client modules, meta data server module and data memory module:
Described client modules obtains catalogue and reads expansion and authorize and small documents layout by accessing described meta data server module, reads in advance in the buffer memory of described client modules by small documents data with the continuous coarsegrain data of the described small documents data space while according to described small documents layout from described data memory module.
The above-mentioned pre-reader based on distributed file system, is characterized in that,
Described meta data server module for carry out catalogue read expansion authorize management and the maintenance of small documents layout information, reception is obtained catalogue and is read the request of expansion mandate and small documents layout, and returns to described catalogue and read expansion mandate and described small documents layout;
Described data memory module is used for providing data storage and inquire interface, receives the access request that reads specific physical location, and return data reading of content;
Described client modules is for sending the request that expansion is authorized of reading of obtaining catalogue to described meta data server module, reading expansion according to the described catalogue getting authorizes, described client modules sends the request of obtaining small documents layout to described meta data server module, and send data read request according to data memory module described in the described small documents cloth office direction getting, complete described small documents data and described coarsegrain data content reads.
The above-mentioned pre-reader based on distributed file system, is characterized in that, described meta data server module also comprises:
Catalogue reads expansion entitlement management module: read for obtaining catalogue described in receiving that described client modules sends the request that expansion is authorized, and authorize described catalogue to described client and read expansion and authorize;
File layout maintenance module: for receiving the request of the small documents layout that described client sends, and reply to described client the described small documents layout that record is safeguarded.
The above-mentioned pre-reader based on distributed file system, is characterized in that, described client modules also comprises:
Catalogue read expansion authorize acquisition module: for read to described catalogue expansion entitlement management module send obtain catalogue read expansion authorize request, and obtain catalogue read authorize laggard line item;
File layout acquisition module: for sending and obtain the request of small documents layout to described file layout maintenance module, to obtain small documents layout, obtain the laggard line item of described small documents layout;
File read module: for according to the described small documents layout of obtaining, obtain the physical location of described small documents data, carry out data read access;
Anonymous caching of page module: the data page pre-reading from described data storage device module for buffer memory, described file read module is searched desired data page in described anonymous caching of page module, if searched successfully, directly from described anonymous caching of page module, take desired data away;
Data storage device access driver module: for accessing described data storage device module, from the physical location of data storage device, obtain data, if described file read module is searched data failure in described anonymous caching of page module, build the continuous data access request of coarsegrain, obtain described coarsegrain continuous data by described data storage device access driver module.
The above-mentioned pre-reader based on distributed file system, it is characterized in that, described catalogue read expansion authorize for guarantee the data content of described catalogue include file can be awarded described catalogue read expansion authorize described client read, can not revised by other clients simultaneously.
The above-mentioned pre-reader based on distributed file system, is characterized in that, described in read expansion authorization packets containing version number, described version number is used for distinguishing same client and reads expansion described in repeatedly same catalogue being obtained and authorize.
The present invention also provides a kind of data pre-head method based on distributed file system, is applied to described pre-reader, and described pre-reader comprises: client modules, meta data server module and data memory module, and the method comprises:
Obtain catalogue and read expansion authorisation step: described client modules obtains catalogue to described meta data server module transmission and reads expansion authorization requests, described meta data server module is authorized catalogue to described client modules and is read expansion mandate;
File data read step: read expansion according to the described catalogue of obtaining and authorize, described client modules sends the request of small documents layout to described meta data server module, and send data read request according to data memory module described in the described small documents cloth office direction getting, complete described small documents data and described coarsegrain content reads.
The above-mentioned data pre-head method based on distributed file system, is characterized in that, described in obtain catalogue read expansion authorisation step also comprise:
Authorize determining step: read expansion and authorize if the described client modules of described client modules judgement has had described catalogue that described meta data server module authorizes, the direct described file data read step of execution;
Authorization requests forwarding step: read expansion mandate if described client modules does not have described catalogue, described client modules obtains described catalogue to described meta data server module transmission and reads the request that expansion is authorized;
Grant authorization step: do not have the modification mandate of conflict if described catalogue reads expansion authorization module inspection, authorizing described in described client modules catalogue reads expansion and authorizes, otherwise, wait is recalled after the mandate of conflict, described client modules is carried out to described catalogue and read authorizing of expansion mandate;
Authority record step: the described catalogue that described client records gets reads expansion and authorizes.
The above-mentioned data pre-head method based on distributed file system, is characterized in that, described file data read step also comprises:
File layout acquisition of information step: send the request of small documents layout for described client modules to described meta data server module, obtain small documents layout information;
Data content read step: the described small documents layout information getting for basis, carrying out data by described client modules to described anonymous caching of page module reads, if read failure, carry out data by described data storage device access driver module to described data storage device module and read.
The above-mentioned data pre-head method based on distributed file system, is characterized in that, described file layout acquisition of information step also comprises:
Layout determining step: if the described client modules of described client modules judgement has had the described small documents layout information that described meta data server module is replied, directly carry out described data content read step;
Layout request forwarding step: if described client modules does not have described small documents layout information, described file layout acquisition module sends the request of small documents layout to described file layout maintenance module;
Layout is returned to step: described file layout maintenance module is replied the request of described small documents layout to described file layout acquisition module, sends described small documents layout;
Layout recording step: described client modules records the described small documents layout getting in buffer memory.
The above-mentioned data pre-head method based on distributed file system, is characterized in that, described data content read step also comprises:
Anonymous caching of page read step: described file read module sends data read request to described anonymous caching of page module, after described anonymous caching of page module searches success, returns to desired data content;
Data storage device read step: if described anonymous caching of page module is searched unsuccessfully according to described data read request, described data storage device access driver module sends data access request to described data storage device, obtains desired data content.
The above-mentioned data pre-head method based on distributed file system, is characterized in that, described anonymous caching of page read step also comprises:
Anonymous page request step: described file read module, according to the data storage device physical location in described file layout information, sends data read request to described anonymous caching of page module;
Anonymous page finding step: described anonymous caching of page module is carried out anonymity page according to described data read request and searched, and described anonymous page is the data page that does not belong to any small documents; If exist described anonymous page identical with the data storage device physical location in described file layout information, search anonymous page success;
Data message returns to step: if it is unsuccessful to search anonymous page in described anonymous page finding step, returns to anonymous caching of page to described file read module and read failure; If search anonymous page success in described anonymous page finding step, and described anonymous page has the identical expansion of reading with the catalogue at described small documents place and authorizes version number, returns to described file read module by described anonymous page; If search anonymous page success in described anonymous page finding step, but having the different expansions of reading from the catalogue at described small documents place, described anonymous page authorizes version number, return to anonymous caching of page to described file read module and read failure;
Anonymous page removes step: if search anonymous page success in described anonymous page finding step, from described anonymous caching of page module, remove described anonymous page.
The above-mentioned data pre-head method based on distributed file system, is characterized in that, described data storage device read step also comprises:
Coarsegrain request of data step: read failure if return to anonymous caching of page in described anonymous caching of page read step, described file read module builds coarsegrain continuous data access request;
Data storage device data read request step: the coarsegrain continuous data access request building according to described coarsegrain request of data step, described data storage device access driver module sends described coarsegrain continuous data access request to described data storage device, and obtains coarsegrain data;
Data are returned to step: described data storage device access driver module returns to the described coarsegrain data that get after described file read module, described file read module directly obtains small documents and accesses required data page, the described anonymous page obtaining after expansion granularity is recorded in anonymous caching of page, and the version number that described anonymous page is set is the version number that expansion is authorized that reads of described small documents place catalogue simultaneously.
Compared with the present invention reads technology with existing small documents, beneficial effect of the present invention is: the present invention is by carrying out the data pre-head between small documents in client, effectively reduce small documents data access delay, brought into play the performance advantage of data storage device coarsegrain sequential access; Simultaneously, the present invention proposes data pre-head method and system between a kind of distributed file system small documents, can effectively pre-read the data between small documents, and bring into play the performance advantage of data storage device sequential access, effectively reduce mass small documents access delay.
Accompanying drawing explanation
Fig. 1 is the structural representation of pre-reader of the present invention;
Fig. 2 is data pre-head method schematic flow sheet of the present invention;
Fig. 3 is file layout acquisition of information of the present invention and data content read step schematic diagram;
Fig. 4 is that the anonymous caching of page of the present invention reads and data storage device read step schematic flow sheet;
Fig. 5 A～Fig. 5 D is the inventive method specific embodiment step schematic diagram;
Fig. 6 is one embodiment of the invention system configuration schematic diagram;
Fig. 7 is the inventive method specific embodiment detailed step schematic diagram.
Wherein, Reference numeral:
40 meta data server module 50 data memory modules
60 client modules
401 catalogues read expansion entitlement management module 402 file layout maintenance modules
601 catalogues read expansion and authorize acquisition module 602 file layout acquisition modules
The anonymous caching of page module of 603 file read module 604
605 data storage device access driver modules
S1～S2, S11～S14, S21～S22, S211～S214, S221～S222, S2211～S2222: the administration step of various embodiments of the present invention
Embodiment
Provide the specific embodiment of the present invention below, with concrete embodiment, the present invention is described in detail by reference to the accompanying drawings.
The present invention proposes the data pre-head method between a kind of small documents.Because the data between the different small documents under same catalogue have better spatial continuity, therefore in the time reading single small documents data, not only obtain the required small grain size data of this small documents, and expansion is reading in advance buffer memory with the continuous coarsegrain data of this small documents data space from data storage device.For data storage device, read coarsegrain alphabetic data and only read small grain size accessing cost for data very approaching, therefore in the time that follow-up other small documents need data access, if desired data has been read in buffer memory in advance, avoid the delay expense synchronously reading from data storage device, brought into play the performance advantage of data storage device coarsegrain sequential access.The present invention also provides a kind of system of carrying out the data pre-head between small documents in client.
The invention provides a kind of pre-reader based on distributed file system, comprising: meta data server module 40 and data memory module 50, client modules 60:
Client modules 60 obtains catalogue by accesses meta-data server module 40 and reads expansion and authorize and small documents layout, reads in advance in the buffer memory of client modules 60 by small documents data with the continuous coarsegrain data of the small documents data space while according to small documents layout from data memory module 50.
Fig. 1 is the structural representation of pre-reader of the present invention, and as shown in Figure 1, this pre-reader comprises:
Meta data server module 40, for reading the management of expansion mandate and the maintenance of file layout information, receives and reads the request of expanding mandate and file layout, and returns and read expansion mandate and small documents layout;
Wherein, meta data server module 40 also comprises:
Catalogue reads expansion entitlement management module 401: described in receiving that described client modules sends, read the request that expansion is authorized, and described in authorizing to described client, read expansion and authorize;
File layout maintenance module 402: for receiving the request of the small documents layout that described client sends, and reply to described client the described small documents layout that record is safeguarded.
Catalogue reads expansion entitlement management module 401 and is responsible for receiving client and obtains particular category and read the request that expansion is authorized, and in the situation that not there is not share and access conflict, authorizes the expansion of reading of catalogue described in client and authorizes.The expansion of reading of catalogue authorizes the data of the catalogue institute include file that has guaranteed that client holds can not revised by other clients simultaneously, hold reading expansion mandate of catalogue in client, meta data server guarantees that do not have other clients revises any file under catalogue simultaneously.In the time that meta data server receives other clients and revises the request of include file of catalogues, need to recall catalogue that client obtained and read after expansion authorizes, the modification mandate that just can authorize other client directorys; Obtain catalogue and read expansion when authorization requests when meta data server receives other clients, if check the modification mandate that catalogue has not been authorized, represent not have share and access conflict, authorize the expansion of reading of other client directorys and authorize.
Wherein, catalogue read expansion authorize for guarantee the data content of described catalogue include file can be awarded described catalogue read expansion authorize described client read, can not revised by other clients simultaneously.
File layout maintenance module 402 is responsible for receiving client and is obtained the request of small documents layout, and returns to the file layout that meta data server record is safeguarded.Wherein, small documents layout is used for the physical location of the data storage device that records small documents data place.
Data memory module 50, for data storage and inquire interface is provided, receives the access request that reads specific physical location, and return data reading of content.
Client modules 60 is for sending the request that expansion is authorized of reading of obtaining catalogue to meta data server module, getting the described catalogue of authorizing reads after expansion mandate, reading expansion according to the catalogue of obtaining authorizes, client modules is to the request of meta data server module Transmit message layout, and send data read request according to the file layout getting to data memory module, complete data content and read.
Wherein, client modules 60 also comprises:
Catalogue read expansion authorize acquisition module 601: for read to catalogue expansion entitlement management module send obtain catalogue read expansion authorize request, and obtain described catalogue read expansion authorize laggard line item;
Catalogue reads expansion and authorizes acquisition module 601, is responsible for reading the expansion of reading of catalogue described in expansion entitlement management module request to meta data server catalogue and authorizes.Described catalogue read expansion authorize obtain after, record described catalogue got catalogue read expansion authorize.The expansion of reading of described catalogue is authorized and is had a version number, same catalogue is obtained to catalogue reads expansion and authorizes to distinguish repeatedly, the catalogue of repeatedly obtaining of same catalogue is read expansion and is authorized, described version number difference.
File layout acquisition module 602: for sending and obtain the request of small documents layout to file layout maintenance module, to obtain small documents layout, obtain the laggard line item of small documents layout;
File layout acquisition module 602, is responsible for to small documents layout described in meta data server small documents layout maintenance module acquisition request.After obtaining described small documents layout, caching record is in client.
File read module 603: for according to the small documents layout of obtaining, obtain the physical location of small documents data, carry out data read access;
File read module 603, according to the small documents layout of obtaining small documents layout modules and having obtained, obtains the physical location of the data storage device at small documents data place, then carries out data read access.First access anonymous caching of page module, if the data page of the physical location of the data storage device of access has been read in anonymous caching of page in advance, directly from anonymous caching of page, take this data page away; Otherwise, expand described data access granularity, build the continuous data access of coarsegrain, request msg storage device access driver module obtains coarsegrain continuous data.
Anonymous caching of page module 604: the data page pre-reading from data storage device module for buffer memory, file read module is searched desired data page in anonymous caching of page module, if searched successfully, directly from anonymous caching of page module, takes desired data away;
Anonymous caching of page module 604 is responsible for the data page that buffer memory pre-reads from data storage device, and data page does not temporarily also belong to any small documents, is therefore called anonymous page.Anonymous page take data storage device physical location as indexed cache in anonymous caching of page.Anonymous page has version number, in the time that small documents is accessed, authorizes version number to compare, thereby determine whether described anonymous page is effective with the expansion of reading of place catalogue.
Data storage device access driver module 605: for accessing data storage devices module, from the physical location of data storage device, obtain data, file read module is searched data failure in anonymous caching of page module, build the continuous data access request of coarsegrain, access driver module by data storage device and obtain coarsegrain continuous data.
Data storage device access driver module 605, responsible accessing data storage devices obtains the data of described physical location granularity.After obtaining data, small documents comprises data and is directly returned to small documents read module, and the anonymous page that expansion granularity reads is recorded in anonymous caching of page.Anonymous page has version number, is set to the version number that expansion is authorized that reads of small documents place catalogue.
The present invention also provides a kind of data pre-head method based on distributed file system, and Fig. 2 is data pre-head method schematic flow sheet of the present invention, and as shown in Figure 2, the method comprises:
Obtain catalogue and read expansion authorisation step S1: client modules 3 obtains catalogue to 1 transmission of meta data server module and reads expansion authorization requests, meta data server module 1 is authorized described catalogue to client modules 3 and is read expansion mandate;
File data read step S2: read expansion according to the catalogue of obtaining and authorize, client modules 3 sends the request of obtaining small documents layout to meta data server module 1, and send data read request according to the described small documents cloth office direction data memory module getting, complete small documents data and coarsegrain data content reads.
Wherein, obtaining catalogue reads expansion authorisation step S1 and also comprises:
Authorize determining step S11: read expansion and authorize if client modules judgement client modules has had catalogue that meta data server module authorizes, direct execute file data read step;
Authorization requests forwarding step S12: read expansion mandate if client modules does not have catalogue, client modules obtains catalogue to the transmission of meta data server module and reads the request that expansion is authorized;
Grant authorization step S13: do not have the modification mandate of conflict if catalogue reads expansion authorization module inspection, authorize client modules catalogue and read expansion mandate, otherwise, wait for and recalling after the mandate of conflict, client modules is carried out to catalogue and read authorizing of expansion mandate;
Authority record step S14: the catalogue that client records gets reads expansion and authorizes.
Wherein, file data read step S2 also comprises:
File layout acquisition of information step S21: send the request of small documents layout for client modules to meta data server module, obtain small documents layout information;
Data content read step S22: the small documents layout information getting for basis, carrying out data by client modules to anonymous caching of page module reads, if read failure, access driver module by data storage device and carry out data to data storage device module and read.
Wherein, Fig. 3 is file layout acquisition of information of the present invention and data content read step schematic diagram, and as shown in Figure 3, file layout acquisition of information step S21 also comprises:
Layout determining step S211: if client modules judgement client modules has had the small documents layout information that meta data server module is replied, direct executing data content read step;
Layout request forwarding step S212: if client modules does not have small documents layout information, file layout acquisition module sends the request of small documents layout to the file layout maintenance module of meta data server;
Layout is returned to step S213: the file layout maintenance module of meta data server is replied the request of small documents layout to the file layout acquisition module of client, sends small documents layout;
Layout recording step S214: client modules records the small documents layout getting in buffer memory.
Wherein, data content read step S22 also comprises:
Anonymous caching of page read step S221: file read module sends data read request to anonymous caching of page module, after anonymous caching of page module searches success, returns to desired data content;
Data storage device read step S222: if anonymous caching of page module is searched unsuccessfully according to data read request, described data storage device access driver module sends data access request to data storage device, obtains desired data content.
Fig. 4 is that the anonymous caching of page of the present invention reads and data storage device read step schematic flow sheet, and as shown in Figure 4, anonymous caching of page read step S221 also comprises:
Anonymous page request step S2211: file read module, according to the data storage device physical location in file layout information, sends data read request to anonymous caching of page module;
Anonymous page finding step S2212: anonymous caching of page module is carried out anonymity page according to data read request and searched, and anonymous page is the data page that does not belong to any small documents; If exist anonymous page identical with the data storage device physical location in file layout information, search anonymous page success;
Data message returns to step S2213: if it is unsuccessful to search anonymous page in anonymous page finding step, returns to anonymous caching of page to file read module and read failure; If search anonymous page success in anonymous page finding step, and anonymous page has the identical expansion of reading with the catalogue at small documents place and authorizes version number, anonymity page returned to file read module; If search anonymous page success in anonymous page finding step, but having the different expansions of reading from the catalogue at small documents place, anonymous page authorizes version number, return to anonymous caching of page to file read module and read failure;
Anonymous page removes step S2214: if search anonymous page success in anonymous page finding step, from anonymous caching of page module, remove anonymous page.
Wherein, data storage device read step S222 also comprises:
Coarsegrain request of data step S2221: read failure if return to anonymous caching of page in anonymous caching of page read step, file read module builds coarsegrain continuous data access request;
Data storage device data read request step S2222: the coarsegrain continuous data access request building according to coarsegrain request of data step, data storage device access driver module sends coarsegrain continuous data access request to data storage device, and obtains coarsegrain data;
Data are returned to step S2223: data storage device access driver module returns to the coarsegrain data that get after file read module, file read module directly obtains small documents and accesses required data page, the anonymous page obtaining after expansion granularity is recorded in anonymous caching of page, and the version number that anonymous page is set is the version number that expansion is authorized that reads of small documents place catalogue simultaneously.
Below in conjunction with the step of the specific embodiment of the invention, data pre-head process is described.
A kind of method of carrying out data pre-head between small documents in client provided by the invention, Fig. 5 A～Fig. 5 D is the inventive method specific embodiment step schematic diagram, as shown in Fig. 5 A～Fig. 5 D, method comprises the steps:
1, client directory reads expansion and authorizes obtaining step;
2, client is obtained small documents layout step;
3, client small documents read step from anonymous caching of page;
4, client small documents drives read step from data storage device access;
In technique scheme, further, described step 1 specifically comprises the steps:
Read expansion mandate if 11 hold described catalogue, needn't carry out subsequent step;
12, client obtain catalogue read expansion authorization module by RPC agreement to meta data server send obtain described catalogue read expansion authorize request;
13, meta data server catalogue reads expansion entitlement management module, if check that described catalogue does not exist the modification mandate of conflict, authorizes described in described client catalogue and reads expansion and authorize, and record described catalogue and read expansion licensing status; Otherwise, wait for and recalling after the modification mandate of conflict, just can authorize described in described client catalogue and read expansion and authorize.
14, the described catalogue that client records is obtained reads expansion and authorizes;
In technique scheme, further, described step 2 specifically comprises the steps:
If 21 clients have been held the layout buffer memory of described small documents, needn't carry out subsequent step;
22, client is obtained small documents layout modules and is sent the request of obtaining described small documents layout by RPC agreement to meta data server;
23, the small documents layout maintenance module of meta data server returns to the file layout of described small documents;
24, client records the layout of the described small documents obtaining in layout buffer memory.
In technique scheme, further, described step 3 specifically comprises the steps:
31, client, according to data storage device physical location in small documents layout, sends read requests to anonymous caching of page module;
32, the if there is no anonymous page of same, physical of anonymous caching of page module searches, returns and does not exist to small documents read module, jumps to (35) step;
If having the identical expansion of reading with the parent directory at described small documents place, the anonymous page that 33, anonymous caching of page module check is obtained authorizes version number, return to small documents read module, from anonymous caching of page module, remove described anonymous page, finish subsequent step simultaneously;
Authorize version number if the 34 anonymous pages that obtain have the different expansions of reading from the parent directory at described small documents place, from anonymous caching of page module, remove described anonymous page, return and do not exist to small documents read module;
35, small documents read module growth data reads granularity, builds the coarsegrain continuous data access request that comprises described small documents data place data storage device physical location, transfers to step 4.
In technique scheme, further, described step 4 specifically comprises the steps:
41, client data storage device access driver module sends data read access request to data storage device, obtains coarsegrain data;
42, client data storage device access driver module is directly accessed required data page small documents and is returned to small documents access modules; The anonymous page that expansion is obtained after granularity is recorded in anonymous caching of page, and the version number that anonymous page is set is the version number that expansion is authorized that reads of described small documents place catalogue.
Below in conjunction with accompanying drawing and concrete embodiment, the present invention is described in further detail.
For reaching above object, the present invention is based on pNFS, provide the concrete embodiment of data pre-head method between a kind of small documents that reads expansion mandate based on catalogue, Fig. 6 is one embodiment of the invention system configuration schematic diagram, as shown in Figure 6, disk block memory device is as data memory module 2, and be divided into the piece of fixing granularity, and piece is carried out to serial number, number the physical location of continuous data block on data storage device also continuous, such as continuous piece 1001,1002,1003,1004,1005 physical location on memory device is also continuous.Client Client1 is as client modules 3, expect the file f 1～f4 under access directory A, the file size of f1～f4 is only all a data block on data storage device, and wherein file f 1～f4 is distributed in respectively the physical block position of the piece 1001,1002,1003,1005 of data storage device.Anonymous page is organized in anonymous caching of page module 34 in chained list anoy_page_list mode, and anonymous page is the not associated data page that arrives any file also.Blocklayoutdriver module, as data storage device access driver module 35, is responsible for obtaining the data of specifying physical location from data storage device.
Fig. 7 is the inventive method specific embodiment detailed step schematic diagram, as shown in Figure 7, specifically comprises the steps:
100, client is obtained particular category and is read the step that expansion is authorized
101, client inspection whether have particular category read expansion authorize.
(1) client Client1 is receiving after the request of file f1～f4 under application program access directory A, first obtains the inode(index node of parent directory A).If the inode of parent directory A exists in client terminal local buffer memory, directly obtain; If the inode of parent directory A is not at client terminal local buffer memory, to meta data server Server(as meta data server module 1) send request and obtain directory A metadata information, then client is set up the inode of directory A in buffer memory according to the directory A metadata information that obtains.
(2) then client Client1 checks that whether holding reading of directory A expands mandate.Read expansion mandate if held, the expansion of reading that directly use has been held is authorized;
102, client Client1 obtains directory A to meta data server transmission and reads expansion authorization requests
Client1 sends and obtains the OPEN_RPC request that directory A expansion is authorized to meta data server, comprises need to obtain catalogue to read the mark need_Dir_Delegation_RD_Extent that expansion authorizes be 1 in request.
103, server end is authorized Client1 directory A and is read expansion mandate
(1) Server termination is received after the OPEN_RPC request of Client1 transmission, analysis request obtains the value of need_Dir_Delegation_RD_Extent, determine that client Client1 need to obtain catalogue and read expansion mandate, in catalogue mandate chained list Dir_delegation_list, whether retrieval exists directory A conflict to license to other clients.
(2) license to other clients if do not retrieve directory A conflict in Dir_delegation_list, Server end reads expansion directory A and licenses to Client1, and in Dir_delegation_list, adds record.
(3) license to other clients if retrieve the conflict of directory A in Dir_delegation_list, first Server recalls conflict from other clients and authorizes, then Server reads expansion directory A and licenses to Client1, and in Dir_delegation_list, adds directory A and read expansion and license to the record of Client1.
104, client Client1 records the directory A of obtaining and reads expansion mandate
(1) client Client1 receives that the directory A that Server returns reads after expansion authorization response, and in the inode of directory A, setting is held catalogue and read the flag bit Dir_Delegation_RD_Extent that expansion is authorized.
(2) client Client1 arranges the expansion of reading of directory A to authorize the Delegation_RD_Extent_version of version number is the unique value of the client Client1 overall situation, repeatedly obtains the expansion of reading of directory A authorize in order to distinguish.As a kind of specific embodiment, a global variable RD_generation is set in Client1, with the value of RD_generation, Delegation_RD_Extent_version value is set at every turn, and arranges rear to RD_generation increase by 1 at every turn.
200, client file reading data step
201, client is obtained the layout of file
Client Client1 sends the RPC request of obtaining file layout to meta data server, first obtain the layout of directory A institute include file, be the information of description document data place physical location, for example: the layout of file f 1～f4 is respectively piece 1001,1002,1003,1005.
202, client file reading content
(1) client Client1, obtaining after the layout piece 1001 of f1, first searches the page that whether has piece 1001 correspondences in anonymous caching of page anoy_page_cache.Anonymous caching of page traversal anoy_page_list, whether retrieval there is the page of piece 1001 correspondences.The page of if block 1001 correspondences does not exist, and returns to non-existent response results; The page of if block 1001 correspondences exists, if check that the version of described page is identical with the Delegation_RD_Extent_version of directory A, represent that described page holds reading expansion mandate of directory A in client to obtain, guarantee the validity of described page, return to described page1001, and remove described page from the anoy_page_list chained list of anonymous caching of page; If the version of described page is different from the Delegation_RD_Extent_version of directory A, represent that described page holds reading expansion mandate of directory A in client to obtain, can not guarantee the validity of described page, therefore return to non-existent response results, and remove described page from the anoy_page_list chained list of anonymous caching of page.
(2) if client Client1 gets the page 1001 at file data place from anonymous caching of page, directly return to application program, be associated with in file f 1.Otherwise, client Client1 drives Blocklayoutdriver to send data read request to data storage device access, the original position of request msg is piece 1001, reading length from essential 1, expand to comprise and pre-read 5 of piece 1002～1005, once piece 1001 start, length is that the data of 5 read from data storage device.
(3) data storage device access drives Blocklayoutdriver from data storage device reads data content, first the page 1001 reading is directly returned to application program, and is associated with file f 1; Page 1002～1005 is added in the anoy_page_list chained list of anonymous caching of page, and the Delegation_RD_Extent_version value that the version value of page 1002～1005 is directory A is set.
(4) in the time that client Client1 accesses follow-up file f 2～f4, can directly in anonymous caching of page, obtain corresponding page 1002, page 1003, page 1005, directly return to application program, and be associated with respectively in file f 2～f4.
203, anonymous caching of page is removed step
(1), in the time of client Client1 unloading of file system, remove anonymous pages whole in anonymous caching of page.
In sum, a kind of pre-reader and method thereof based on distributed system provided by the invention, effectively reduces small documents data access delay, and has brought into play the performance advantage of data storage device coarsegrain sequential access.
Certainly; the present invention also can have other various embodiments; in the situation that not deviating from spirit of the present invention and essence thereof; those of ordinary skill in the art are when making according to the present invention various corresponding changes and distortion, but these corresponding changes and distortion all should belong to the protection range of the appended claim of the present invention.
Claims (13)
1. the pre-reader based on distributed file system, is characterized in that, described pre-reader comprises: client modules, meta data server module and data memory module:
Described client modules obtains catalogue and reads expansion and authorize and small documents layout by accessing described meta data server module, reads in advance in the buffer memory of described client modules by small documents data with the continuous coarsegrain data of the described small documents data space while according to described small documents layout from described data memory module.
2. the pre-reader based on distributed file system according to claim 1, is characterized in that,
Described meta data server module: for carry out catalogue read expansion authorize management and the maintenance of small documents layout information, reception is obtained catalogue and is read the request of expansion mandate and small documents layout, and returns to described catalogue and read expansion mandate and described small documents layout;
Described data memory module: for data storage and inquire interface is provided, receive the access request that reads specific physical location, and return data reading of content;
Described client modules: read the request that expansion is authorized for obtaining catalogue to described meta data server module transmission, reading expansion according to the described catalogue getting authorizes, described client modules sends the request of obtaining small documents layout to described meta data server module, and send data read request according to data memory module described in the described small documents cloth office direction getting, complete described small documents data and described coarsegrain data content reads.
3. the pre-reader based on distributed file system according to claim 2, is characterized in that, described meta data server module also comprises:
Catalogue reads expansion entitlement management module: read for obtaining catalogue described in receiving that described client modules sends the request that expansion is authorized, and authorize described catalogue to described client and read expansion and authorize;
File layout maintenance module: for receiving the request of the small documents layout that described client sends, and reply to described client the described small documents layout that record is safeguarded.
4. the pre-reader based on distributed file system according to claim 2, is characterized in that, described client modules also comprises:
Catalogue read expansion authorize acquisition module: for read to described catalogue expansion entitlement management module send obtain catalogue read expansion authorize request, and obtaining authorize laggard line item;
File layout acquisition module: for sending and obtain the request of small documents layout to described file layout maintenance module, to obtain small documents layout, obtain the laggard line item of described small documents layout;
File read module: for according to the described small documents layout of obtaining, obtain the physical location of described small documents data, carry out data read access;
Anonymous caching of page module: the data page pre-reading from described data storage device module for buffer memory, described file read module is searched desired data page in described anonymous caching of page module, if searched successfully, directly from described anonymous caching of page module, take desired data away;
Data storage device access driver module: for accessing described data storage device module, from the physical location of data storage device, obtain data, if described file read module is searched data failure in described anonymous caching of page module, build the continuous data access request of coarsegrain, obtain described coarsegrain continuous data by described data storage device access driver module.
5. according to the pre-reader based on distributed file system described in claim 1 or 2, it is characterized in that, described catalogue read expansion authorize for guarantee the data content of described catalogue include file can be awarded described catalogue read expansion authorize described client read, can not revised by other clients simultaneously.
6. the pre-reader based on distributed file system according to claim 5, is characterized in that, described in read expansion authorization packets containing version number, described version number is used for distinguishing same client and reads expansion described in repeatedly same catalogue being obtained and authorize.
7. the data pre-head method based on distributed file system, be applied to the pre-reader as described in any one in claim 1-6, described pre-reader comprises: client modules, meta data server module and data memory module, and the method comprises:
Obtain catalogue and read expansion authorisation step: described client modules obtains catalogue to described meta data server module transmission and reads expansion authorization requests, described meta data server module is authorized catalogue to described client modules and is read expansion mandate;
File data read step: read expansion according to the described catalogue of obtaining and authorize, described client modules sends the request of obtaining small documents layout to described meta data server module, and send data read request according to data memory module described in the described small documents cloth office direction getting, complete described small documents data and described coarsegrain data content reads.
8. the data pre-head method based on distributed file system according to claim 7, is characterized in that, described in obtain catalogue and read expansion authorisation step and also comprise:
Authorize determining step: read expansion and authorize if the described client modules of described client modules judgement has had described catalogue that described meta data server module authorizes, the direct described file data read step of execution;
Authorization requests forwarding step: read expansion mandate if described client modules does not have described catalogue, described client modules reads to obtaining catalogue described in described meta data server module transmission the request that expansion is authorized;
Grant authorization step: do not have the modification mandate of conflict if described catalogue reads expansion authorization module inspection, authorizing described in described client modules catalogue reads expansion and authorizes, otherwise, wait is recalled after the mandate of conflict, described client modules is carried out to described catalogue and read authorizing of expansion mandate;
Authority record step: the described catalogue that described client records gets reads expansion and authorizes.
9. the data pre-head method based on distributed file system according to claim 7, is characterized in that, described file data read step also comprises:
File layout acquisition of information step: send and obtain the request of small documents layout to described meta data server module for described client modules, obtain small documents layout information;
Data content read step: the described small documents layout information getting for basis, carrying out data by described client modules to described anonymous caching of page module reads, if read failure, carry out data by described data storage device access driver module to described data storage device module and read.
10. the data pre-head method based on distributed file system according to claim 9, is characterized in that, described file layout acquisition of information step also comprises:
Layout determining step: if the described client modules of described client modules judgement has had the described small documents layout information that described meta data server module is replied, directly carry out described data content read step;
Layout request forwarding step: if described client modules does not have described small documents layout information, described file layout acquisition module sends the request of small documents layout to described file layout maintenance module;
Layout is returned to step: described file layout maintenance module is replied the request of described small documents layout to described file layout acquisition module, sends described small documents layout;
Layout recording step: described client modules records the described small documents layout getting in buffer memory.
11. data pre-head methods based on distributed file system according to claim 9, is characterized in that, described data content read step also comprises:
Anonymous caching of page read step: described file read module sends data read request to described anonymous caching of page module, after described anonymous caching of page module searches success, returns to desired data content;
Data storage device read step: if described anonymous caching of page module is searched unsuccessfully according to described data read request, described data storage device access driver module sends data access request to described data storage device, obtains desired data content.
12. according to the data pre-head method based on distributed file system described in claim 11, it is characterized in that, described anonymous caching of page read step also comprises:
Anonymous page request step: described file read module, according to the data storage device physical location in described file layout information, sends data read request to described anonymous caching of page module;
Anonymous page finding step: described anonymous caching of page module is carried out anonymity page according to described data read request and searched, and described anonymous page is the data page that does not belong to any small documents; If exist described anonymous page identical with the data storage device physical location in described file layout information, search anonymous page success;
Data message returns to step: if it is unsuccessful to search anonymous page in described anonymous page finding step, returns to anonymous caching of page to described file read module and read failure; If search anonymous page success in described anonymous page finding step, and described anonymous page has the identical expansion of reading with the catalogue at described small documents place and authorizes version number, returns to described file read module by described anonymous page; If search anonymous page success in described anonymous page finding step, but having the different expansions of reading from the catalogue at described small documents place, described anonymous page authorizes version number, return to anonymous caching of page to described file read module and read failure;
Anonymous page removes step: if search anonymous page success in described anonymous page finding step, from described anonymous caching of page module, remove described anonymous page.
13. according to the data pre-head method based on distributed file system described in claim 11, it is characterized in that, described data storage device read step also comprises:
Coarsegrain request of data step: read failure if return to anonymous caching of page in described anonymous caching of page read step, described file read module builds coarsegrain continuous data access request;
Data storage device data read request step: the coarsegrain continuous data access request building according to described coarsegrain request of data step, described data storage device access driver module sends described coarsegrain continuous data access request to described data storage device, and obtains coarsegrain data;
Data are returned to step: described data storage device access driver module returns to the described coarsegrain data that get after described file read module, described file read module directly obtains small documents and accesses required data page, the described anonymous page obtaining after expansion granularity is recorded in anonymous caching of page, and the version number that described anonymous page is set is the version number that expansion is authorized that reads of described small documents place catalogue simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410108037.6A CN103916465A (en) | 2014-03-21 | 2014-03-21 | Data pre-reading device based on distributed file system and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410108037.6A CN103916465A (en) | 2014-03-21 | 2014-03-21 | Data pre-reading device based on distributed file system and method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103916465A true CN103916465A (en) | 2014-07-09 |
Family
ID=51041863
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410108037.6A Pending CN103916465A (en) | 2014-03-21 | 2014-03-21 | Data pre-reading device based on distributed file system and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103916465A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104519132A (en) * | 2014-12-19 | 2015-04-15 | 西安理邦科学仪器有限公司 | Client application handling method and client application handling system of distributed monitoring network |
| CN105138545A (en) * | 2015-07-09 | 2015-12-09 | 中国科学院计算技术研究所 | Method and system for asynchronously pre-reading directory entries in distributed file system |
| CN105141666A (en) * | 2015-07-29 | 2015-12-09 | 江苏天联信息科技发展有限公司 | Information data storing method and device |
| CN105491103A (en) * | 2015-11-18 | 2016-04-13 | 临沂大学 | Logistics storage indoor environment monitoring system |
| CN105701158A (en) * | 2015-12-30 | 2016-06-22 | 浪潮(北京)电子信息产业有限公司 | File system read-write optimization method and framework |
| CN105740166A (en) * | 2014-12-11 | 2016-07-06 | 中兴通讯股份有限公司 | Cache reading and reading processing method and device |
| CN106776759A (en) * | 2016-11-17 | 2017-05-31 | 郑州云海信息技术有限公司 | The small documents pre-head method and system of distributed file system |
| CN107291870A (en) * | 2017-06-15 | 2017-10-24 | 郑州云海信息技术有限公司 | Files in batch read method in a kind of distributed storage |
| CN107491545A (en) * | 2017-08-25 | 2017-12-19 | 郑州云海信息技术有限公司 | The catalogue read method and client of a kind of distributed memory system |
| CN108717420A (en) * | 2018-04-20 | 2018-10-30 | 北京鲸鲨软件科技有限公司 | A kind of method that small documents reading efficiency is improved in distributed file system |
| CN110008016A (en) * | 2019-04-15 | 2019-07-12 | 深圳市万普拉斯科技有限公司 | Anonymous page management method, device, terminal device and readable storage medium storing program for executing |
| CN110352410A (en) * | 2016-09-29 | 2019-10-18 | 华睿泰科技有限责任公司 | Track the access module and preextraction index node of index node |
| CN110502498A (en) * | 2019-08-16 | 2019-11-26 | 济南浪潮数据技术有限公司 | A kind of distributed file system pre-reading method of files and system |
| CN110765086A (en) * | 2019-10-25 | 2020-02-07 | 浪潮电子信息产业股份有限公司 | Directory reading method and system for small files, electronic equipment and storage medium |
| CN111680014A (en) * | 2020-05-29 | 2020-09-18 | 北京五八信息技术有限公司 | Shared file acquisition method and device, electronic equipment and storage medium |
| CN112559574A (en) * | 2020-12-25 | 2021-03-26 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and readable storage medium |
| CN115277145A (en) * | 2022-07-20 | 2022-11-01 | 北京志凌海纳科技有限公司 | Distributed storage access authorization management method, system, device and readable medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103176754A (en) * | 2013-04-02 | 2013-06-26 | 浪潮电子信息产业股份有限公司 | Reading and storing method for massive amounts of small files |
-
2014
- 2014-03-21 CN CN201410108037.6A patent/CN103916465A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103176754A (en) * | 2013-04-02 | 2013-06-26 | 浪潮电子信息产业股份有限公司 | Reading and storing method for massive amounts of small files |
Non-Patent Citations (4)
| Title |
|---|
| 刘高军;王帝澳: "基于Redis的海量小文件分布式存储方法研究", 《计算机工程与科学》 * |
| 曹风华: "一种基于授权机制的分布式文件***小文件访问优化策略", 《计算机***应用》 * |
| 李洪昌;舒继武: "一种基于pNFS协议的对象文件***的设计与实现", 《小型微型计算机***》 * |
| 百度文库: "pNFS-(并行NFS)-详解", 《百度文库》 * |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105740166A (en) * | 2014-12-11 | 2016-07-06 | 中兴通讯股份有限公司 | Cache reading and reading processing method and device |
| CN105740166B (en) * | 2014-12-11 | 2020-05-19 | 中兴通讯股份有限公司 | Cache reading and reading processing method and device |
| CN104519132B (en) * | 2014-12-19 | 2019-04-12 | 西安理邦科学仪器有限公司 | The client computer application processing method and system of distributed monitoring network |
| CN104519132A (en) * | 2014-12-19 | 2015-04-15 | 西安理邦科学仪器有限公司 | Client application handling method and client application handling system of distributed monitoring network |
| CN105138545A (en) * | 2015-07-09 | 2015-12-09 | 中国科学院计算技术研究所 | Method and system for asynchronously pre-reading directory entries in distributed file system |
| CN105138545B (en) * | 2015-07-09 | 2018-10-09 | 中国科学院计算技术研究所 | The asynchronous method and system pre-read of directory entry in a kind of distributed file system |
| CN105141666A (en) * | 2015-07-29 | 2015-12-09 | 江苏天联信息科技发展有限公司 | Information data storing method and device |
| CN105141666B (en) * | 2015-07-29 | 2018-12-14 | 江苏天联信息科技发展有限公司 | Information data storing method and device |
| CN105491103A (en) * | 2015-11-18 | 2016-04-13 | 临沂大学 | Logistics storage indoor environment monitoring system |
| CN105701158A (en) * | 2015-12-30 | 2016-06-22 | 浪潮(北京)电子信息产业有限公司 | File system read-write optimization method and framework |
| CN110352410B (en) * | 2016-09-29 | 2023-01-20 | 华睿泰科技有限责任公司 | Tracking access patterns of index nodes and pre-fetching index nodes |
| CN110352410A (en) * | 2016-09-29 | 2019-10-18 | 华睿泰科技有限责任公司 | Track the access module and preextraction index node of index node |
| CN106776759A (en) * | 2016-11-17 | 2017-05-31 | 郑州云海信息技术有限公司 | The small documents pre-head method and system of distributed file system |
| CN107291870A (en) * | 2017-06-15 | 2017-10-24 | 郑州云海信息技术有限公司 | Files in batch read method in a kind of distributed storage |
| CN107291870B (en) * | 2017-06-15 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Method for reading files in distributed storage in batch |
| CN107491545A (en) * | 2017-08-25 | 2017-12-19 | 郑州云海信息技术有限公司 | The catalogue read method and client of a kind of distributed memory system |
| CN108717420B (en) * | 2018-04-20 | 2022-04-15 | 北京鲸鲨软件科技有限公司 | Method for improving small file reading efficiency in distributed file system |
| CN108717420A (en) * | 2018-04-20 | 2018-10-30 | 北京鲸鲨软件科技有限公司 | A kind of method that small documents reading efficiency is improved in distributed file system |
| CN110008016A (en) * | 2019-04-15 | 2019-07-12 | 深圳市万普拉斯科技有限公司 | Anonymous page management method, device, terminal device and readable storage medium storing program for executing |
| CN110502498A (en) * | 2019-08-16 | 2019-11-26 | 济南浪潮数据技术有限公司 | A kind of distributed file system pre-reading method of files and system |
| CN110765086A (en) * | 2019-10-25 | 2020-02-07 | 浪潮电子信息产业股份有限公司 | Directory reading method and system for small files, electronic equipment and storage medium |
| CN110765086B (en) * | 2019-10-25 | 2022-08-02 | 浪潮电子信息产业股份有限公司 | Directory reading method and system for small files, electronic equipment and storage medium |
| CN111680014A (en) * | 2020-05-29 | 2020-09-18 | 北京五八信息技术有限公司 | Shared file acquisition method and device, electronic equipment and storage medium |
| CN111680014B (en) * | 2020-05-29 | 2023-06-02 | 北京五八信息技术有限公司 | Shared file acquisition method and device, electronic equipment and storage medium |
| CN112559574A (en) * | 2020-12-25 | 2021-03-26 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and readable storage medium |
| CN112559574B (en) * | 2020-12-25 | 2023-10-13 | 北京百度网讯科技有限公司 | Data processing method, device, electronic equipment and readable storage medium |
| CN115277145A (en) * | 2022-07-20 | 2022-11-01 | 北京志凌海纳科技有限公司 | Distributed storage access authorization management method, system, device and readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103916465A (en) | Data pre-reading device based on distributed file system and method thereof | |
| CN103179185B (en) | Method and system for creating files in cache of distributed file system client | |
| CN101510219B (en) | File data accessing method, apparatus and system | |
| CN104753966A (en) | Resource file inquiry method and system based on server and client caches | |
| US9195611B2 (en) | Efficiently updating and deleting data in a data storage system | |
| CN107153644B (en) | Data synchronization method and device | |
| KR102256890B1 (en) | Access method and device | |
| CN108614837B (en) | File storage and retrieval method and device | |
| CN107818111B (en) | Method for caching file data, server and terminal | |
| CN104331453A (en) | Distributed file system and constructing method thereof | |
| CN103631623A (en) | Method and device for allocating application software in trunking system | |
| CN102984256B (en) | Processing method and system for metadata based on authorization manner | |
| CN103902660A (en) | System and method for prefetching file layout through readdir++ in cluster file system | |
| CN103559319A (en) | Cache synchronization method and equipment for distributed cluster file system | |
| CN103631820A (en) | Metadata management method and device of distributed file system | |
| CN108540510B (en) | Cloud host creation method and device and cloud service system | |
| CN103761256A (en) | Maintenance system and method of branch points of remote directories | |
| CN111177159A (en) | Data processing system and method and data updating equipment | |
| CN106919696B (en) | SEO site construction method and SEO request response method | |
| WO2016101758A1 (en) | Cross-cluster data synchronization method and device | |
| CN104021137A (en) | Method and system for opening and closing file locally through client side based on catalogue authorization | |
| CN109783462B (en) | Data access method and device based on distributed file system | |
| CN102572011A (en) | Method, device and system for processing data | |
| CN112866406B (en) | Data storage method, system, device, equipment and storage medium | |
| CN112148745B (en) | Multi-HBase cluster access method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140709 |