CN109857703A - A kind of space bucket accounting statistical method and its device based on minio file system - Google Patents
A kind of space bucket accounting statistical method and its device based on minio file system Download PDFInfo
- Publication number
- CN109857703A CN109857703A CN201910034312.7A CN201910034312A CN109857703A CN 109857703 A CN109857703 A CN 109857703A CN 201910034312 A CN201910034312 A CN 201910034312A CN 109857703 A CN109857703 A CN 109857703A
- Authority
- CN
- China
- Prior art keywords
- bucket
- level
- space
- module
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The space the bucket accounting statistical method based on minio file system that the present invention relates to a kind of, including connection telefile control system；It identifies and generates bucket list；Extract bucket unit information；Traversal bucket unit simultaneously generates level list；Extract hierarchical information；Traversal level simultaneously generates listed files；Extraction document information；Traversal file simultaneously generates configuration information；Generate level-one report；Analysis level-one report simultaneously generates reference listing；Reference listing is uploaded to server.A kind of space bucket accounting statistic device based on minio file system, comprising: insmod, total spider module, Bucket calling module, Bucket spider module, level calling module, level spider module, file calling module, file traversal module, preprocessing module, analysis module and uploading module.The present invention can carry out effective space accounting statistics for the space bucket in the memory space of minio file system, greatly facilitate management of the staff to file system, improve work efficiency.
Description
Technical field
The present invention relates to a kind of space bucket accounting statistical method and its device based on minio file system, belong to
Server-side testing field.
Background technique
Under existing big business scenario, Minio file system using more and more extensive.As a kind of the multi-platform of open source
File system, Minio can provide all multiplex roles for uploading for file, downloading and modifying etc., realize the file of lightweight
Operation；And the object storage service based on Apache License v2.0 open source protocol, Amazon S3 cloud storage clothes can be compatible with
Be engaged in interface, be very suitable for storing large capacity and non-structured data, such as picture, video, journal file, Backup Data with
And virtual machine image etc.；The size of obj ect file is not also limited clearly, from several kb to maximum 5T etc..Meanwhile Minio
As the service of a unusual light weight, can easily realize and the combination of NodeJS, Redis or MySQL etc..
But it when backstage presses survey platform to use minio file system as file storage medium, can have a problem that, i.e.,
How space hold statistical information is effectively obtained；If wanting to count the occupied space of all bucket units, source system does not provide phase
Pass means.Technological means at this stage is the root into file in machine, checks that space uses by inputting du-sh order
Situation.
However as file system, data scrubbing is necessarily referred to；And to realize effective data scrubbing, it at least needs to obtain
Take each file occupied space and newest modification time, in this way can targetedly defrag space, and it is how real
Existing this effect is problem to be solved instantly.
Summary of the invention
In view of the deficiencies of the prior art, the space the bucket accounting based on minio file system that the present invention provides a kind of
Statistical method, comprising the following steps:
S1, connection telefile control system；
S2, memory space is traversed, identify bucket unit and generates bucket list；
S3, it calls bucket list and extracts bucket unit information；
S4, different bucket units are traversed respectively based on bucket unit information and generate corresponding level list；
S5, it calls different levels list and extracts corresponding hierarchical information；
S6, different levels are traversed respectively based on hierarchical information and generate corresponding document list；
S7, different listed files are called and extract corresponding document information；
S8, different files are traversed respectively based on the file information and generate corresponding configuration information；
S9, all configuration informations are pre-processed and generate level-one report；
S10, further data processing is carried out to level-one report based on presupposed information and generates reference listing；
S11, reference listing is uploaded to server.
Further, the memory space includes at least a bucket unit.
Further, the bucket unit information include at least bucket, the address bucket, shared amount of memory with
And newest modification time.
Further, the bucket unit includes at least a level.
Further, the hierarchical information includes at least level name, level address and shared amount of memory.
Further, the level includes at least a file.
Further, the file information includes at least filename, file address and shared amount of memory.
Further, the generation of level-one list is referred specifically to by traversal time to bucket unit all in memory space, no
With the sequence of the progress of the corresponding document information of the corresponding level and different levels of bucket unit.
Further, the presupposed information includes at least the sequence of storage time, occupies the sequence of amount of memory and account for
With the accounting of amount of memory.
A kind of space bucket accounting statistic device based on minio file system, comprising:
It insmods, for connecting telefile control system；
Total spider module, for being traversed to memory space, identifying bucket unit and generating bucket list；
Bucket calling module, for calling bucket list and extracting bucket unit information；
Bucket spider module, for being traversed respectively to different bucket units and generating corresponding level list；
Level calling module, for calling different levels list and extracting corresponding hierarchical information；
Level spider module, for being traversed respectively to different levels and generating corresponding document list；
File calling module, for calling different listed files and extracting corresponding document information；
File traversal module, for being traversed respectively to different files and generating corresponding configuration information；
Preprocessing module, for being pre-processed to all configuration informations and generating level-one report；
Analysis module, for carrying out further data processing to level-one report and generating reference listing；And
Uploading module, for uploading reference listing to server.
The invention has the benefit that can be carried out in the memory space of minio file system for the space bucket
Effective space accounting statistics, greatly facilitates management of the staff to file system, improves work efficiency.
Detailed description of the invention
Fig. 1 show overview flow chart according to the present invention；
Fig. 2 show apparatus according to the invention connection figure；
Fig. 3 show the schematic diagram of A according to embodiments of the present invention；
Fig. 4 show the schematic diagram of B according to embodiments of the present invention.
Specific embodiment
It should be appreciated that the embodiment of the present invention can be by computer hardware, the combination of hardware and software or by depositing
The computer instruction in non-transitory computer-readable memory is stored up to be effected or carried out.Standard volume can be used in the method
Journey technology-includes that the non-transitory computer-readable storage media configured with computer program is realized in computer program,
In configured in this way storage medium computer is operated in a manner of specific and is predefined --- according in a particular embodiment
The method and attached drawing of description.Each program can with the programming language of level process or object-oriented come realize with department of computer science
System communication.However, if desired, the program can be realized with compilation or machine language.Under any circumstance, which can be volume
The language translated or explained.In addition, the program can be run on the specific integrated circuit of programming for this purpose.
In addition, the operation of process described herein can be performed in any suitable order, unless herein in addition instruction or
Otherwise significantly with contradicted by context.Process described herein (or modification and/or combination thereof) can be held being configured with
It executes, and is can be used as jointly on the one or more processors under the control of one or more computer systems of row instruction
The code (for example, executable instruction, one or more computer program or one or more application) of execution, by hardware or its group
It closes to realize.The computer program includes the multiple instruction that can be performed by one or more processors.
Further, the method can be realized in being operably coupled to suitable any kind of computing platform, wrap
Include but be not limited to PC, mini-computer, main frame, work station, network or distributed computing environment, individual or integrated
Computer platform or communicated with charged particle tool or other imaging devices etc..Each aspect of the present invention can be to deposit
The machine readable code on non-transitory storage medium or equipment is stored up to realize no matter be moveable or be integrated to calculating
Platform, such as hard disk, optical reading and/or write-in storage medium, RAM, ROM, so that it can be read by programmable calculator, when
Storage medium or equipment can be used for configuration and operation computer to execute process described herein when being read by computer.This
Outside, machine readable code, or part thereof can be transmitted by wired or wireless network.When such media include combining microprocessor
Or other data processors realize steps described above instruction or program when, invention as described herein including these and other not
The non-transitory computer-readable storage media of same type.When methods and techniques according to the present invention programming, the present invention
It further include computer itself.
Computer program can be applied to input data to execute function as described herein, to convert input data with life
At storing to the output data of nonvolatile memory.Output information can also be applied to one or more output equipments as shown
Device.In the preferred embodiment of the invention, the data of conversion indicate physics and tangible object, including the object generated on display
Reason and the particular visual of physical objects are described.
It should be noted that such as without Special Statement, in the disclosure used in the "an" of singular, " described " and
"the" is also intended to including most forms, unless the context clearly indicates other meaning.In addition, unless otherwise defined, this paper institute
All technical and scientific terms used are identical as the normally understood meaning of those skilled in the art.This paper specification
Used in term be intended merely to description specific embodiment, be not intended to be limiting of the invention.Term as used herein
"and/or" includes the arbitrary combination of one or more listed items.
(" such as ", " such as ") makes it should be appreciated that provided in this article any and all example or exemplary language
With being intended merely to that the embodiment of the present invention is better described, and unless the context requires otherwise, otherwise the scope of the present invention will not be applied
Limitation.
Next specific embodiments of the present invention are described further in conjunction with attached drawing:
It show overview flow chart according to the present invention referring to Fig.1, specifically includes the following steps:
S1, connection telefile control system；
S2, memory space is traversed, identify bucket unit and generates bucket list；The memory space is at least
Including a bucket unit；
S3, it calls bucket list and extracts bucket unit information；The bucket unit information includes at least
Bucket, the address bucket, shared amount of memory and newest modification time；
S4, different bucket units are traversed respectively based on bucket unit information and generate corresponding level list；
The bucket unit includes at least a level；
S5, it calls different levels list and extracts corresponding hierarchical information；The hierarchical information includes at least level name, level
Address and shared amount of memory；
S6, different levels are traversed respectively based on hierarchical information and generate corresponding document list；
S7, different listed files are called and extract corresponding document information；The level includes at least a file；
S8, different files are traversed respectively based on the file information and generate corresponding configuration information；The file information
Including at least filename, file address and shared amount of memory；
S9, all configuration informations are pre-processed and generate level-one report；The generation of level-one list is referred specifically to by traversal
Time is to the corresponding level of bucket unit all in memory space, difference bucket unit and the corresponding text of different levels
The sequence of the progress of part information；
S10, further data processing is carried out to level-one report based on presupposed information and generates reference listing；The default letter
Breath includes at least the sequence of storage time, occupies the sequence of amount of memory and occupies the accounting of amount of memory；
S11, reference listing is uploaded to server.
It is shown apparatus according to the invention connection figure referring to Fig. 2, specifically includes following part:
It insmods, realization interaction is connect with total spider module, for connecting telefile control system；
Total spider module connect realization interaction with Bucket calling module, for traversing to memory space, identifies
Bucket unit simultaneously generates bucket list；
Bucket calling module connect realization interaction with Bucket spider module, for calling bucket list and extracting
Bucket unit information；
Bucket spider module, connect with level calling module realization interaction, for different bucket units respectively into
Row traverses and generates corresponding level list；
Level calling module connect realization interaction with level spider module, for calling different levels list and extracting phase
Answer hierarchical information；
Level spider module connect realization interaction with file calling module, for being traversed respectively simultaneously to different levels
Generate corresponding document list；
File calling module connect realization interaction with file traversal module, for calling different listed files and extracting phase
Answer the file information；
File traversal module connect realization interaction with preprocessing module, for different files to be traversed and given birth to respectively
At corresponding configuration information；
Preprocessing module connect realization interaction with analysis module, for all configuration informations to be pre-processed and generated
Level-one report；
Analysis module connect realization interaction with uploading module, for carrying out further data processing and life to level-one report
At reference listing；And
Uploading module, for uploading reference listing to server.
The schematic diagram of A according to embodiments of the present invention is shown referring to Fig. 3, embodiment A shows in minio file system
The list of different bucket units can be ranked up according to its occupied space size or newest modification time, pass through this column
The form of table can the open-and-shut memory state for showing memory space；In the actual use environment, program can be with certain
Period or number carry out data acquisition, the long term maintenance suitable for server.
The schematic diagram of B according to embodiments of the present invention is shown referring to Fig. 4, embodiment B shows based in embodiment A
The table that list is further beautified, in the actual use environment, staff can be periodically by this form to O&Ms
Or other related practitioners send the disk occupancy situation of bucket.
The above, only presently preferred embodiments of the present invention, the invention is not limited to above embodiment, as long as
It reaches technical effect of the invention with identical means, all should belong to protection scope of the present invention.In protection model of the invention
Its technical solution and/or embodiment can have a variety of different modifications and variations in enclosing.
Claims (10)
1. a kind of space bucket accounting statistical method based on minio file system, comprising the following steps:
S1, connection telefile control system；
S2, memory space is traversed, identify bucket unit and generates bucket list；
S3, it calls bucket list and extracts bucket unit information；
S4, different bucket units are traversed respectively based on bucket unit information and generate corresponding level list；
S5, it calls different levels list and extracts corresponding hierarchical information；
S6, different levels are traversed respectively based on hierarchical information and generate corresponding document list；
S7, different listed files are called and extract corresponding document information；
S8, different files are traversed respectively based on the file information and generate corresponding configuration information；
S9, all configuration informations are pre-processed and generate level-one report；
S10, further data processing is carried out to level-one report based on presupposed information and generates reference listing；
S11, reference listing is uploaded to server.
2. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the memory space includes at least a bucket unit.
3. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In when the bucket unit information is including at least bucket, the address bucket, shared amount of memory and newest modification
Between.
4. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the bucket unit includes at least a level.
5. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the hierarchical information includes at least level name, level address and shared amount of memory.
6. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the level includes at least a file.
7. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the file information includes at least filename, file address and shared amount of memory.
8. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the generation of level-one list is referred specifically to by traversal time to bucket unit all in memory space, difference bucket unit
Corresponding level and different levels corresponding document information progress sequence.
9. the space the bucket accounting statistical method according to claim 1 based on minio file system, feature exist
In the presupposed information includes at least the sequence of storage time, occupies the sequence of amount of memory and occupies amount of memory
Accounting.
10. a kind of space bucket accounting statistic device based on minio file system, comprising:
It insmods, for connecting telefile control system；
Total spider module, for being traversed to memory space, identifying bucket unit and generating bucket list；
Bucket calling module, for calling bucket list and extracting bucket unit information；
Bucket spider module, for being traversed respectively to different bucket units and generating corresponding level list；
Level calling module, for calling different levels list and extracting corresponding hierarchical information；
Level spider module, for being traversed respectively to different levels and generating corresponding document list；
File calling module, for calling different listed files and extracting corresponding document information；
File traversal module, for being traversed respectively to different files and generating corresponding configuration information；
Preprocessing module, for being pre-processed to all configuration informations and generating level-one report；
Analysis module, for carrying out further data processing to level-one report and generating reference listing；And
Uploading module, for uploading reference listing to server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910034312.7A CN109857703B (en) | 2019-01-15 | 2019-01-15 | Pocket space duty ratio statistical method and device based on minium file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910034312.7A CN109857703B (en) | 2019-01-15 | 2019-01-15 | Pocket space duty ratio statistical method and device based on minium file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109857703A true CN109857703A (en) | 2019-06-07 |
| CN109857703B CN109857703B (en) | 2023-05-12 |
Family
ID=66894615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910034312.7A Active CN109857703B (en) | 2019-01-15 | 2019-01-15 | Pocket space duty ratio statistical method and device based on minium file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109857703B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380163A (en) * | 2020-10-20 | 2021-02-19 | 广州西山居世游网络科技有限公司 | S3 file system space occupation monitoring method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014201942A1 (en) * | 2013-06-19 | 2014-12-24 | Tencent Technology (Shenzhen) Company Limited | Method, server and system for managing content in content delivery network |
| CN105046162A (en) * | 2014-03-12 | 2015-11-11 | 阿普赛尔有限公司 | Maintaining and using buffer memory of sun-to-father map in content addressing storage system |
| CN106156289A (en) * | 2016-06-28 | 2016-11-23 | 北京百迈客云科技有限公司 | The method of the data in a kind of read-write object storage system and device |
| CN107844271A (en) * | 2017-10-27 | 2018-03-27 | 郑州云海信息技术有限公司 | A kind of method, apparatus and computer-readable recording medium for being classified storage |
-
2019
- 2019-01-15 CN CN201910034312.7A patent/CN109857703B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014201942A1 (en) * | 2013-06-19 | 2014-12-24 | Tencent Technology (Shenzhen) Company Limited | Method, server and system for managing content in content delivery network |
| CN105046162A (en) * | 2014-03-12 | 2015-11-11 | 阿普赛尔有限公司 | Maintaining and using buffer memory of sun-to-father map in content addressing storage system |
| CN106156289A (en) * | 2016-06-28 | 2016-11-23 | 北京百迈客云科技有限公司 | The method of the data in a kind of read-write object storage system and device |
| CN107844271A (en) * | 2017-10-27 | 2018-03-27 | 郑州云海信息技术有限公司 | A kind of method, apparatus and computer-readable recording medium for being classified storage |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380163A (en) * | 2020-10-20 | 2021-02-19 | 广州西山居世游网络科技有限公司 | S3 file system space occupation monitoring method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109857703B (en) | 2023-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108123994B (en) | Industrial-field-oriented cloud platform architecture | |
| CN107493310B (en) | Cloud resource processing method and cloud management platform | |
| CN106250987B (en) | A kind of machine learning method, device and big data platform | |
| CN109582301A (en) | Method for processing business, device, equipment and the medium of task based access control scheduling system | |
| CN109074377A (en) | Managed function for real-time processing data stream executes | |
| CN105574082A (en) | Storm based stream processing method and system | |
| CN106201757A (en) | A kind of abnormality eliminating method and device | |
| CN106557470A (en) | data extraction method and device | |
| CN112651126A (en) | Building engineering construction optimization system, method, terminal and storage medium based on BIM | |
| CN101308553A (en) | Use of a three-dimensional (3D) data center to share service operations | |
| CN106886535A (en) | A kind of data pick-up method and apparatus for being adapted to multiple data sources | |
| CN104023072A (en) | Device and method for pushing data | |
| CN110489343A (en) | A kind of conversational services test method, system, device and storage medium | |
| CN110249312A (en) | Data integration operation changing | |
| CN109893854A (en) | A kind of server data management method and system | |
| CN109446255A (en) | A kind of storage method and system that can be adaptive | |
| CN109857703A (en) | A kind of space bucket accounting statistical method and its device based on minio file system | |
| CN110298007A (en) | User behavior statistical method, device, electronic equipment and computer readable storage medium | |
| CN110134381A (en) | A kind of system and method intelligently generating source code configuration for alarm host machine | |
| CN108769094A (en) | Acquisition method, device, PC equipment and the mobile terminal of user information | |
| CN106021270B (en) | The method and system of synergistic data intelligence between data warehouse and big data storage | |
| CN112541688A (en) | Service data checking method and device, electronic equipment and computer storage medium | |
| CN109947466A (en) | A kind of reverse method and device thereof for obtaining UE4 global object table | |
| CN109032628A (en) | A kind of method and apparatus of the page of application program and rear end decoupling | |
| KR20160072851A (en) | System and method for analyzing large-scale high resolution satellite image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 519080 Room 102, 202, 302, 402 Room 327 Room 102, 202 Room 329 Room 302, 325 Qiandao Ring Road, Tangjiawan Town, High-tech Zone, Zhuhai City, Guangdong Province Applicant after: Zhuhai Jinshan Digital Network Technology Co.,Ltd. Applicant after: CHENGDU XISHANJU INTERACTIVE ENTERTAINMENT TECHNOLOGY Co.,Ltd. Address before: Building 3, Jinshan Software Park, 325 Qiandao Ring Road, Tangjiawan Town, high tech Zone, Zhuhai, Guangdong 519000 Applicant before: ZHUHAI KINGSOFT ONLINE GAME TECHNOLOGY Co.,Ltd. Applicant before: CHENGDU XISHANJU INTERACTIVE ENTERTAINMENT TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |