CN109144951A - A kind of catalogue update method and meta data server based on distributed file system - Google Patents
A kind of catalogue update method and meta data server based on distributed file system Download PDFInfo
- Publication number
- CN109144951A CN109144951A CN201810864713.0A CN201810864713A CN109144951A CN 109144951 A CN109144951 A CN 109144951A CN 201810864713 A CN201810864713 A CN 201810864713A CN 109144951 A CN109144951 A CN 109144951A
- Authority
- CN
- China
- Prior art keywords
- catalogue
- file
- meta data
- data server
- new file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present application discloses a kind of catalogue update method and meta data server based on distributed file system.This method obtains the request of the new file of creation of client transmission first, thereafter, creates new file according to request.The new file directory chained list built-up according to the multistage catalogue of local cache by client is carried in request, this method updates in meta data server also according to the catalogue chained list to the statistical information of the multistage catalogue of new file.The statistical information that this method is capable of more than the parent directory and parent directory to new file each hierarchical directory is updated, compared with the existing technology, the accuracy for having ensured catalogue statistical information avoids the file Premium Features of distributed file system from exception occur because catalogue is not statistical uncertainty true.In addition, this method saves the time of the catalogue updated needed for searching to other metadata server, therefore the timely quick update of catalogue can be realized while quickly creation file.
Description
Technical field
This application involves distributed file system fields, more more particularly to a kind of catalogue based on distributed file system
New method and meta data server.
Background technique
Currently, the mass data that internet generates has been difficult to the server system processing single by tradition.Tradition is concentrated
The extended capability of formula system is limited.Demand with storage, analysis and processing to ultra-large data constantly increases, distributed
System is come into being.
Distributed file system is one kind of distributed system, for local file system, distributed field system
System is mainly stored in file on multiple data servers.It is broadly divided into: meta data server, storage server, client
End composition.User can access meta data server and storage server by client, realize the storage of file data.
What is mainly stored on meta data server is metadata information, for file system, is needed new in creation
The quantity of subdirectory and subfile below catalogue is counted when file, it is therefore desirable to which timely update file directory.
Currently, only updating created file in more new directory to guarantee that the speed of document creation was unlikely to slowly
Parent directory, but for the higher levels such as parent directory of ancestor directories and ancestor directories catalogue then without update.But it is this
There are some problems for update mode, for example, if not updating higher levels' catalogue such as parent directory of ancestor directories and ancestor directories
Statistical information will lead to some file Premium Features such as file system directories quota and need that catalogue file number is used to unite
When evaluation, file number information can not be promptly and accurately got, and then it is abnormal to cause these Premium Features to occur.
Therefore, how while guaranteeing document creation speed, father's mesh of timely update ancestor directories and ancestor directories
The statistical information of higher levels' catalogues such as record, becomes distributed file system field urgent problem.
Summary of the invention
Based on the above issues, this application provides a kind of catalogue update method and metadata based on distributed file system
Server, with while guaranteeing the fast speed of creation file, more than the file parent directory that timely updates each hierarchical directory
Statistical information.
The embodiment of the present application discloses following technical solution:
The application first aspect provides a kind of catalogue update method based on distributed file system, and the method is applied to
Meta data server, which comprises
The request for the new file of creation that client is sent is obtained, the catalogue chained list of the new file is carried in the request；
The catalogue chained list, to be built-up according to the multistage catalogue of local cache by the client, the multistage catalogue includes: institute
State each hierarchical directory of the parent directory and the parent directory of new file or more；
The new file is created according to the request, and, according to the catalogue chained list, update the meta data server
In to the statistical information of the multistage catalogue of the new file.
Optionally, described to update in the meta data server to the statistical information of the multistage catalogue of the new file, tool
Body includes:
It updates in the meta data server, file and file number under the parent directory of the new file of statistics, and,
It updates in the meta data server, file and file number under each hierarchical directory more than parent directory of statistics.
The application second aspect provides a kind of meta data server, which includes: acquisition module, creates mould
Block and update module；
The acquisition module, for obtaining the request for creating new file of client transmission, in the request described in carrying
The catalogue chained list of new file；The catalogue chained list, for by the client according to the multistage catalogue of local cache built-up, institute
State the parent directory and each hierarchical directory more than parent directory that multistage catalogue includes: the new file；
The creation module, for creating the new file according to the request；
The update module, for updating and believing the statistics of the multistage catalogue of the new file according to the catalogue chained list
Breath.
Optionally, the update module, specifically includes: first, which updates submodule and second, updates submodule；
Described first updates submodule, file and file number under the parent directory of the new file for updating statistics；
Described second updates submodule, file and text under each hierarchical directory more than parent directory for updating statistics
Part number.
Compared to the prior art, the application has the advantages that
Catalogue update method provided by the present application based on distributed file system is applied to meta data server, obtains first
The request for the new file of creation for taking client to send creates new file according to request thereafter.Due to carrying in request by client
The new file directory chained list built-up according to the multistage catalogue of local cache is held, this method is updated also according to the catalogue chained list
To the statistical information of the multistage catalogue of new file in meta data server.
Since catalogue chained list is client according to each hierarchical directory building more than parent directory and parent directory of new file
At, therefore this method is capable of the statistical information of more than parent directory and parent directory to new file each hierarchical directory and is carried out more
Newly, the accuracy for having ensured the statistical information of catalogue avoids the file Premium Features of distributed file system from counting not because of catalogue
Accurately occur abnormal.
In addition, catalogue update method provided by the present application, utilizes the catalogue chained list constructed by client according to local cache
Update the catalogue of new file in meta data server, without expend the time searched to other metadata server needed for update
Catalogue, therefore the timely quick update of catalogue can be realized while quickly creation file.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of application without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is the catalogue update method flow chart provided by the embodiments of the present application based on distributed file system；
Fig. 2 is the structural schematic diagram of meta data server provided by the embodiments of the present application.
Specific embodiment
In order to make the above objects, features, and advantages of the present application more apparent, with reference to the accompanying drawing and it is specific real
Mode is applied to be described in further detail the embodiment of the present application.Firstly, the research background to the application is introduced.
Currently, when creating new file under a catalogue, the system of meta data server meeting its parent directory of statistical updating
Information is counted, but for higher levels' catalogue such as parent directory of ancestor directories and ancestor directories, if catalogue level is more, member
Data server need to spend the time go to search the parent directory of update in need and each hierarchical directory more than parent directory.
But for meta data server, the information such as parent directory and ancestor directories of new file are likely to not exist
It is stored on the same meta data server, therefore, each layer more than parent directory and parent directory to search update in need
Grade catalogue, needs constantly to communicate between different meta data servers, can just find these information, and catalogue is so caused to update speed
It spends slow.The considerations of for the speed for creating file, distributed file system not will be updated ancestor directories and ancestor directories
Higher levels' catalogue such as parent directory statistical information, this results in updating higher levels' mesh such as the parent directory of ancestor directories and ancestor directories
The statistics of record uses catalogue file number to count not in time, in some file Premium Features needs such as file system directories quota
When value, file number information can not be promptly and accurately got, therefore these Premium Features often will appear exception.
To solve the above problems, inventor after study, proposes a kind of catalogue update based on distributed file system
Method and meta data server.Method provided by the present application and meta data server are described in detail with reference to the accompanying drawing.
Referring to Fig. 1, which is the catalogue update method process provided by the embodiments of the present application based on distributed file system
Figure.
As shown in Figure 1, the catalogue update method provided by the embodiments of the present application based on distributed file system, comprising:
Step 101: meta data server obtains the request for the new file of creation that client is sent, and carries new file in request
Catalogue chained list.
Due to each level mesh more than parent directory and parent directory of the new file of client local cache creation in need
Record, such as ancestor directories and the parent directory of ancestor directories etc., inventor has found that the multistage catalogue of these cachings is available
In the new file directory for updating meta data server end, the time is expended to other metadata service without meta data server
The directory information of device inquiry missing.
In the present embodiment, catalogue chained list can be constructed according to the above-mentioned multistage catalogue of local cache by client.As one
Optional embodiment is planted, the parent directory of new file and each hierarchical directory more than parent directory in client query caching, it
Afterwards, these catalogues are connected according to hierarchical relationship, is configured to catalogue chained list.In catalogue chained list, there is father in catalogues at different levels
There is directive property between the catalogue of subrelation.
As an example, parent directory is directed toward ancestor directories (i.e. the parent directory of parent directory), and ancestor directories are directed toward ancestor directories
Parent directory.The number for being directed toward the catalogue of ancestor directories is file number under ancestor directories, and so on.
As another example, ancestor directories are directed toward parent directory, and the parent directory of ancestor directories is directed toward ancestor directories.Ancestor directories
The number of pointed catalogue is file number under ancestor directories, and so on.
In the present embodiment, when can send the request for creating new file by client, the catalogue chained list of new file is carried
Wherein, when and then meta data server obtains the request for the new file of creation that client is sent, it can get and wherein carry
Catalogue chained list.
Step 102: meta data server creates new file according to request, is updated in meta data server according to catalogue chained list
To the statistical information of the multistage catalogue of new file.
In this step, meta data server is able to respond the request of the new file of creation of client transmission, completes new file
Creation.Simultaneously as containing the information of each hierarchical directory of parent directory and parent directory or more in catalogue chained list, such as multiple
Set membership etc. under hierarchical directory between file number and catalogue, therefore, meta data server can be directly according to catalogues
Chained list updates the statistical information to the multistage catalogue of new file.
Specifically, meta data server can update in meta data server according to the catalogue chained list of new file, statistics
File and file number under the parent directory of new file, and, it updates in meta data server, each layer more than parent directory of statistics
File and file number under grade catalogue.
The catalogue update method based on distributed file system that the above are provided by the embodiments of the present application, this method are applied to
The request for the new file of creation that meta data server, first acquisition client are sent creates new file according to request thereafter.By
The new file directory chained list built-up according to the multistage catalogue of local cache by client is carried in request, this method is also
According to the catalogue chained list, update in meta data server to the statistical information of the multistage catalogue of new file.
Since catalogue chained list is client according to each hierarchical directory building more than parent directory and parent directory of new file
At, therefore this method is capable of the statistical information of more than parent directory and parent directory to new file each hierarchical directory and is carried out more
Newly, the accuracy for having ensured the statistical information of catalogue avoids the file Premium Features of distributed file system from counting not because of catalogue
Accurately occur abnormal.
In addition, catalogue update method provided by the present application, utilizes the catalogue chained list constructed by client according to local cache
Update the catalogue of new file in meta data server, without expend the time searched to other metadata server needed for update
Catalogue, therefore the timely quick update of catalogue can be realized while quickly creation file.
The catalogue update method provided based on the above embodiment, present invention also provides a kind of with catalogue more new function
Meta data server.The meta data server is introduced with reference to the accompanying drawing.
Referring to fig. 2, which is the structural schematic diagram of meta data server provided by the embodiments of the present application.
As shown in Fig. 2, meta data server provided by the embodiments of the present application, comprising: obtain module 201, creation module 202
With update module 203.
Wherein, module 201 is obtained, for obtaining the request for creating new file of client transmission, new text is carried in request
The catalogue chained list of part；
Creation module 202, for creating new file according to request；
Update module 203, for updating the statistical information to the multistage catalogue of new file according to catalogue chained list.
It should be noted that in the present embodiment, catalogue chained list, to be constructed by client according to the multistage catalogue of local cache
It forms.The multistage catalogue for constructing catalogue chained list includes: the parent directory of new file and each hierarchical directory more than parent directory.
The above are meta data servers provided by the embodiments of the present application.Since catalogue chained list is client according to new file
Each hierarchical directory more than parent directory and parent directory is built-up, therefore the meta data server can be to the father of new file
The statistical information of each hierarchical directory more than catalogue and parent directory is updated, and has ensured the accuracy of the statistical information of catalogue,
The file Premium Features of distributed file system are avoided exception occur because catalogue is not statistical uncertainty true.
In addition, meta data server provided by the present application, utilizes the catalogue chained list constructed by client according to local cache
Update the catalogue of new file, without expend the time searched to other metadata server needed for the catalogue that updates, therefore can
The timely quick update of catalogue is realized while quickly creation file.
Optionally, in meta data server provided by the above embodiment, update module 203 can also be specifically included: the
One, which updates submodule and second, updates submodule.
Wherein, first updates submodule, file and file number under the parent directory of the new file for updating statistics；
Second updates submodule, file and file number under each hierarchical directory more than parent directory for updating statistics.
It should be noted that all the embodiments in this specification are described in a progressive manner, each embodiment it
Between same and similar part may refer to each other, each embodiment focuses on the differences from other embodiments.
For equipment and system embodiment, since it is substantially similar to the method embodiment, so describe fairly simple,
The relevent part can refer to the partial explaination of embodiments of method.Equipment and system embodiment described above is only schematic
, wherein unit may or may not be physically separated as illustrated by the separation member, as unit prompt
Component may or may not be physical unit, it can and it is in one place, or may be distributed over multiple networks
On unit.Some or all of the modules therein can be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
Those of ordinary skill in the art can understand and implement without creative efforts.
The above, only a kind of specific embodiment of the application, but the protection scope of the application is not limited thereto,
Within the technical scope of the present application, any changes or substitutions that can be easily thought of by anyone skilled in the art,
Should all it cover within the scope of protection of this application.Therefore, the protection scope of the application should be with scope of protection of the claims
Subject to.
Claims (4)
1. a kind of catalogue update method based on distributed file system, which is characterized in that the method takes applied to metadata
Business device, which comprises
The request for the new file of creation that client is sent is obtained, the catalogue chained list of the new file is carried in the request；It is described
Catalogue chained list, to be built-up according to the multistage catalogue of local cache by the client, the multistage catalogue includes: described new
Each hierarchical directory more than parent directory of file and the parent directory；
The new file is created according to the request, and, according to the catalogue chained list, it is right in the meta data server to update
The statistical information of the multistage catalogue of the new file.
2. the method according to claim 1, wherein described update in the meta data server to the new text
The statistical information of the multistage catalogue of part, specifically includes:
It updates in the meta data server, file and file number under the parent directory of the new file of statistics, and, it updates
In the meta data server, file and file number under each hierarchical directory more than parent directory of statistics.
3. a kind of meta data server characterized by comprising obtain module, creation module and update module；
The acquisition module carries the new text in the request for obtaining the request for creating new file of client transmission
The catalogue chained list of part；The catalogue chained list is described more to be built-up according to the multistage catalogue of local cache by the client
The parent directory and each hierarchical directory more than parent directory that grade catalogue includes: the new file；
The creation module, for creating the new file according to the request；
The update module, for updating the statistical information to the multistage catalogue of the new file according to the catalogue chained list.
4. meta data server according to claim 3, which is characterized in that the update module specifically includes: first more
New submodule and second updates submodule；
Described first updates submodule, file and file number under the parent directory of the new file for updating statistics；
Described second updates submodule, file and file under each hierarchical directory more than parent directory for updating statistics
Number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810864713.0A CN109144951A (en) | 2018-08-01 | 2018-08-01 | A kind of catalogue update method and meta data server based on distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810864713.0A CN109144951A (en) | 2018-08-01 | 2018-08-01 | A kind of catalogue update method and meta data server based on distributed file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109144951A true CN109144951A (en) | 2019-01-04 |
Family
ID=64798626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810864713.0A Pending CN109144951A (en) | 2018-08-01 | 2018-08-01 | A kind of catalogue update method and meta data server based on distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109144951A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110944043A (en) * | 2019-11-14 | 2020-03-31 | 北京无限光场科技有限公司 | Method, device, equipment and medium for displaying medium information |
| CN111258957A (en) * | 2020-01-10 | 2020-06-09 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for updating directory of distributed file system |
| CN113626404A (en) * | 2021-06-30 | 2021-11-09 | 济南浪潮数据技术有限公司 | Method, device and medium for controlling nested directory quota of distributed file system |
| CN118034611A (en) * | 2024-04-09 | 2024-05-14 | 联想凌拓科技有限公司 | Method, device, equipment and medium for managing quota of file |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207778A (en) * | 2006-12-19 | 2008-06-25 | 中兴通讯股份有限公司 | Method for updating electronic service guidebooks information |
| US7752226B1 (en) * | 2002-12-20 | 2010-07-06 | Symantec Operating Corporation | Reverse pathname lookup by inode identifier |
| CN103179185A (en) * | 2012-12-25 | 2013-06-26 | 中国科学院计算技术研究所 | Method and system for creating files in cache of distributed file system client |
| CN104484460A (en) * | 2014-12-29 | 2015-04-01 | 浪潮(北京)电子信息产业有限公司 | Metadata heat degree statistical method of distributed file system |
| CN105302817A (en) * | 2014-06-25 | 2016-02-03 | 苏宁云商集团股份有限公司 | Distributed file system management method and apparatus |
-
2018
- 2018-08-01 CN CN201810864713.0A patent/CN109144951A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7752226B1 (en) * | 2002-12-20 | 2010-07-06 | Symantec Operating Corporation | Reverse pathname lookup by inode identifier |
| CN101207778A (en) * | 2006-12-19 | 2008-06-25 | 中兴通讯股份有限公司 | Method for updating electronic service guidebooks information |
| CN103179185A (en) * | 2012-12-25 | 2013-06-26 | 中国科学院计算技术研究所 | Method and system for creating files in cache of distributed file system client |
| CN105302817A (en) * | 2014-06-25 | 2016-02-03 | 苏宁云商集团股份有限公司 | Distributed file system management method and apparatus |
| CN104484460A (en) * | 2014-12-29 | 2015-04-01 | 浪潮(北京)电子信息产业有限公司 | Metadata heat degree statistical method of distributed file system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110944043A (en) * | 2019-11-14 | 2020-03-31 | 北京无限光场科技有限公司 | Method, device, equipment and medium for displaying medium information |
| CN111258957A (en) * | 2020-01-10 | 2020-06-09 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for updating directory of distributed file system |
| CN111258957B (en) * | 2020-01-10 | 2023-06-02 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for updating distributed file system catalog |
| CN113626404A (en) * | 2021-06-30 | 2021-11-09 | 济南浪潮数据技术有限公司 | Method, device and medium for controlling nested directory quota of distributed file system |
| CN113626404B (en) * | 2021-06-30 | 2024-02-13 | 济南浪潮数据技术有限公司 | Control method, device and medium for nested directory quota of distributed file system |
| CN118034611A (en) * | 2024-04-09 | 2024-05-14 | 联想凌拓科技有限公司 | Method, device, equipment and medium for managing quota of file |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109144951A (en) | A kind of catalogue update method and meta data server based on distributed file system | |
| CN102164186B (en) | Method and system for realizing cloud search service | |
| US9940381B1 (en) | Managing information about entities using observations | |
| CN103544261B (en) | A kind of magnanimity structuring daily record data global index's management method and device | |
| US20190073362A1 (en) | Method and system for providing a federated wide area motion imagery collection service | |
| CN107451208B (en) | Data searching method and device | |
| Zhu et al. | Distributed skyline retrieval with low bandwidth consumption | |
| CN104216957A (en) | Query system and query method for video metadata | |
| CN105760380A (en) | Database query method, device and system | |
| CN110413845B (en) | Resource storage method and device based on Internet of things operating system | |
| CN111258978B (en) | Data storage method | |
| CN105512320A (en) | User ranking obtaining method and device and server | |
| CN106611000A (en) | Method, device and system for searching resource object | |
| US20230409527A1 (en) | Method And System For Deleting Obsolete Files From A File System | |
| CN103823846A (en) | Method for storing and querying big data on basis of graph theories | |
| Ramesh et al. | Data modelling for discrete time series data using Cassandra and MongoDB | |
| CN103034650B (en) | A kind of data handling system and method | |
| CN109062500B (en) | Metadata management server, data storage system and data storage method | |
| CN113810234B (en) | Method and device for processing micro-service link topology and readable storage medium | |
| CN104408084A (en) | Method and device for screening big data | |
| CN105589910A (en) | HBase (Hadoop Database)-based mass transaction data retrieving method and system | |
| CN103136294A (en) | File operating method and device | |
| WO2017146348A1 (en) | Location-based big data system | |
| EP2420945A1 (en) | Foldering by stable query | |
| Antunes et al. | Semantic-based publish/subscribe for M2M |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190104 |