CN103902669B - A kind of separate type file system based on different storage mediums - Google Patents
A kind of separate type file system based on different storage mediums Download PDFInfo
- Publication number
- CN103902669B CN103902669B CN201410097343.4A CN201410097343A CN103902669B CN 103902669 B CN103902669 B CN 103902669B CN 201410097343 A CN201410097343 A CN 201410097343A CN 103902669 B CN103902669 B CN 103902669B
- Authority
- CN
- China
- Prior art keywords
- write
- read
- medium
- volatile
- file system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
Abstract
The invention discloses a kind of separate type file system based on different storage mediums, different storage mediums include the medium of the medium of non-volatile small grain size read-write and non-volatile big granularity read-write, the medium of non-volatile small grain size read-write is connected with rambus, and the medium of non-volatile big granularity read-write is connected with I/O buses；The medium of non-volatile small grain size read-write is used for the metadata of storage file system, and the medium of non-volatile big granularity read-write is used for the data of storage file system；On the medium of non-volatile small grain size read-write, the read-write that operates with small grain size of the file system to metadata, on the medium that non-volatile big granularity is read and write, the read-write that operates with big granularity of the file system to data.Be placed on the metadata of file system on the medium of non-volatile small grain size read-write by the present invention, and the medium of non-volatile small grain size read-write is connected with rambus, the speed of metadata operation is accelerated, can effectively improve file system consistency and persistent problem.
Description
Technical field
The invention belongs to computer memory system technical field, more particularly, to a kind of based on different storage mediums
Separate type file system.
Background technology
The capacity for being commonly stored medium accesses time delay and is directly proportional to it, and the bigger access time delay of capacity is more long.Therefore calculate
Multi-level storage hierarchy is usually provided in machine architecture.At present, Large Copacity is usually realized using magnetic storage technology
Storage, most commonly hard disk（Hard Disk Drive, HDD）, long, read-write granularity is big for its read-write delay.File system is being visited
When asking the HDD as peripheral hardware, many times can be consumed, reduce overall performance.In addition, the uniformity to keep file system
And persistence, file system can utilize some mechanism, such as daily record, copy-on-write etc., the performance overall to system bring
Adverse effect.
The performance overall in order to improve computer, has done many researchs.The appearance of solid-state disk is even more there is provided a contract
Machine.A series of mixing storage scheme is suggested.The scheme of wherein main flow has two kinds：As shown in Fig. 1 (a), a kind of scheme is used
Used as the caching for reading and writing slow medium, such as using solid-state disk as caching, i.e., solid-state disk is used as disk for the fast medium of read-write
Caching, by the frequent data buffer storage for accessing in solid-state disk, to improve the hit rate of solid-state disk in this storage scheme, goes out in succession
The strategy of series of optimum caching is showed；As shown in Fig. 1 (b), another scheme is using the fast medium of read-write Jie slow with read-write
Matter is stored as mixing, and by taking solid-state disk and hard disk as an example, data are uniquely stored in solid-state disk or hard disk, and data have solid-state
Selection algorithm in disk or hard disk lays particular stress on point difference according to performance also various.Due to having introduced solid-state disk in all schemes,
Read file data speed on improve to some extent, but for write and frequently change access performance optimization it is not high, particularly
Updated for irregular small documents, in order to the uniformity and persistence that ensure file system will devote a tremendous amount of time to update
Metadata is write back, due also to device factor in itself has and substantial amounts of additionally writes.
The content of the invention
For the disadvantages described above of prior art, the invention provides a kind of new mixing storage scheme and based on this mixing
The separate type file system of storage, the performance brought its object is to be reduced to holding file system consistency and persistence disappears
Consumption, improves the speed of query metadata and more new metadata, thus solves and additionally writes, the technical problem of hydraulic performance decline.
To achieve the above object, it is described the invention provides a kind of separate type file system based on different storage mediums
Different storage mediums include the medium of the medium of non-volatile small grain size read-write and non-volatile big granularity read-write, wherein, it is described
The medium of non-volatile small grain size read-write is connected with rambus, the medium and I/O buses of the non-volatile big granularity read-write
It is connected；The medium of the non-volatile small grain size read-write is used for the metadata of storage file system, the non-volatile big granularity
The medium of read-write is used for the data of storage file system；On the medium that non-volatile small grain size is read and write, file system is to first number
According to the read-write for operating with small grain size, on the medium that non-volatile big granularity is read and write, operation of the file system to data makes
With the read-write of big granularity.
As a further improvement on the present invention, the file system periodically will be stored in the non-volatile small grain size reading
On the medium that the copies of metadata on medium write is read and write to the non-volatile big granularity.
As present invention further optimization, if medium and internal memory that file system can read and write non-volatile small grain size
Stored as mixing, i.e., used as internal memory, then file system is directly non-easy for the medium of the non-volatile small grain size read-write of system identification
Operating file system on the medium of the small grain size read-write of mistake, to the direct establishment of metadata, renewal and deletion.
As present invention further optimization, if medium and internal memory that file system can not read and write non-volatile small grain size
Mixing, i.e. system can not recognize Jie that the medium of non-volatile small grain size read-write is read and write as internal memory, then non-volatile small grain size
Metadata in matter is operated in needing to call in internal memory, the medium that metadata is read and write through rambus in non-volatile small grain size
Called in and internal memory between and write back.
Used as present invention further optimization, the medium of the non-volatile small grain size read-write is stored for magnetic random is accessed
Device, resistive random access storage device or phase transition storage.
Used as present invention further optimization, the medium of the non-volatile big granularity read-write is solid-state disk or hard disk.
Specifically, the metadata of the file system refer to management space data structure and file system in file unit
Data and directory tree structure, or the only data structure of management space distribution.
In general, by the contemplated above technical scheme of the present invention compared with prior art, due to by file system
Metadata be placed on the medium of non-volatile small grain size read-write, and the non-volatile small grain size read-write medium and internal memory
Bus is connected, and accelerates the speed of metadata operation, can effectively improve file system consistency and persistent problem；By
Can be solved the problems, such as additionally to be write in current storage system with the renewal of small grain size in metadata, improve the effect of whole system
Rate；And periodically the metadata of file system is backuped to due to file system on the medium of non-volatile big granularity read-write,
When metadata frequent operation causes system fault, data will not lose, and can easily shift, with high reliability.
Brief description of the drawings
Fig. 1 is typically to mix storage architecture at present；
Fig. 2 is a kind of typically mixed architecture based on different storage mediums；
Fig. 3 is a kind of typical file system physical layout；
Fig. 4 is the treatment to file system metadata in a kind of typical architecture.
Specific embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as additionally, technical characteristic involved in invention described below each implementation method
Not constituting conflict each other can just be mutually combined.
First, some nouns and term in the present invention are explained：
Big granularity and small grain size：Big granularity is generally the size of page or block, the non-volatile big granularity read-write of current main flow
Medium is solid-state disk and hard disk；Small grain size is usually to be read and write by byte, and more non-volatile small grain size read-writes are discussed at present
Medium be phase transition storage, magnetic random access memory and resistive random access storage device；
File system metadata：According to different demands, the scope of file system metadata is changeable, can refer to pipe
Manage the metadata and directory tree structure of file in the data structure and file system in space, it is also possible to only management space distribution
Data structure, by taking EXT2 as an example, the metadata of file system can refer to superblock, data bitmap block, inode bitmaps, inode
The data block of table, catalogue, if the dielectric capacity ratio that the medium of non-volatile small grain size read-write is read and write with non-volatile big granularity
Very little, or the medium of non-volatile small grain size read-write capacity very little, or system can not compatible enough capacity it is non-volatile
The medium of small grain size read-write, the metadata scope of file system can reduce, can refer to superblock, block group descriptor, inode
Figure, data bitmap block, inode table；
The data of file system：Refer to the part in addition to file system metadata, scope is according to file system metadata
Scope it is changeable, equally by taking EXT2 as an example, if non-volatile small grain size read-write dielectric capacity enough, the number of file system
According to the content that can only include file, if the dielectric capacity of non-volatile small grain size read-write is not enough, the data of file system may include
The content of data block, i.e. catalogue or file.
The invention provides a kind of separate type file system based on different storage mediums, the different storage mediums include
The medium of non-volatile small grain size read-write and the medium of non-volatile big granularity read-write, wherein：
The medium of non-volatile small grain size read-write is connected with rambus；
The medium of non-volatile big granularity read-write is connected with I/O buses.
Wherein, the medium read-write of non-volatile small grain size read-write postpones small, such as magnetic random access memory（Magnetic
Random Access Memory, MRAM）, resistive random access storage device（ResistiveRandomAccessMemory,
RRAM）, phase transition storage（Phase Change Memory, PCM）Deng；The medium read-write of non-volatile big granularity read-write postpones
Greatly, such as solid-state disk, hard disk.
In the file system：
The metadata of file system is placed on the medium of non-volatile small grain size read-write；
The data of file system are placed on the medium of non-volatile big granularity read-write；
On the medium that non-volatile small grain size is read and write, the read-write that operates with small grain size of the file system to metadata,
On the medium that non-volatile big granularity is read and write, the read-write that operates with big granularity of the file system to data.
Wherein, if system can recognize the medium of non-volatile small grain size read-write as a part for internal memory, file system
The metadata that system is pointed on non-volatile small grain size read-write medium uses local operation.
If system can not recognize the medium of non-volatile small grain size read-write as a part for internal memory, file system is by position
Internal memory is called in the metadata on non-volatile small grain size read-write medium.
Read operation to metadata is divided into two kinds, and one is read for big granularity, such as file system initialization, and replaces text
During the caching of part system, using the reading of big granularity, general, such as the page identical with the unit of memory management of reading of big granularity；Secondly
It is the reading of small grain size, such as when file system reads inode in search procedure, using the reading of small grain size, small grain size is read general
It is byte level.To the write operation of metadata, using write through method, and writing for small grain size, unit and small grain size that small grain size is write
The unit of reading is identical.
In the file system, metadata of the storage on the medium of the non-volatile small grain size read-write can be periodically
Backup on the medium of non-volatile big granularity read-write.
The medium for setting non-volatile small grain size read-write below is magnetic random access memory（Magnetic Random
Access Memory, MRAM）, the medium of non-volatile big granularity read-write is solid-state disk（Solid State Disk, SSD）, with
EXT2 is source, is provided with two block groups, and the metadata of file system refers to superblock, data bitmap block, inode bitmaps, inode table
With the content of catalogue, the data of file system refer to the content of file.The structure for then mixing storage is shown in Fig. 2, wherein, MRAM is non-easy
The medium of the small grain size read-write of mistake, is connected with rambus, and SSD is the medium of non-volatile big granularity read-write, with I/O bus phases
Even.The layout of file system is shown in Fig. 3, and the medium that MRAM reads and writes as non-volatile small grain size deposits the metadata of file system,
Metadata including block group 0 and block group 1, the metadata of each block group includes superblock, group descriptor, data bitmap block, inode
Bitmap, inode table, the data block of storing directory content；The medium that SSD reads and writes as non-volatile big granularity, the He of storage block group 0
The data of block group 1, data are the content for being stored in file in block group.
Assuming that system can recognize MRAM, using MRAM as internal memory a part, with reference to above-mentioned example to the present invention
It is specifically described.
In the example illustrated, new mixing storage organization includes：
MRAM is connected with rambus；
SSD is connected with I/O buses.
MRAM and SSD set up new file system as mixing storage thereon.
In new file system：
The metadata of file system is placed on MRAM；
The data of file system are placed on SSD；
As shown in figure 4, operation of the file system to metadata directly runs on MRAM；
Establishment, renewal, deletion, lookup to metadata etc. are operated, direct local operation, without transfer medium；
In the operation such as establishment, renewal, deletion to metadata, the granularity write is identical with the granularity for updating DRAM.
File system periodically by the data copy on MRAM to SSD when MRAM is damaged, or data can be shifted
When, directly SSD can be transferred on another machine.
Above-mentioned scheme, can efficiently solve or improve file system consistency and persistent problem, reduce volume
The outer amount write, improves the efficiency of whole system, and with high reliability.
As it will be easily appreciated by one skilled in the art that the foregoing is only presently preferred embodiments of the present invention, it is not used to
The limitation present invention, all any modification, equivalent and improvement made within the spirit and principles in the present invention etc., all should include
Within protection scope of the present invention.
Claims (7)
1. a kind of separate type file system based on different storage mediums, it is characterised in that the different storage mediums include non-
The medium of volatile small grain size read-write and the medium of non-volatile big granularity read-write, wherein, the non-volatile small grain size read-write
Medium be connected with rambus, the medium of the non-volatile big granularity read-write is connected with I/O buses；It is described non-volatile small
The medium of granularity read-write is used for the metadata of storage file system, and the medium of the non-volatile big granularity read-write is used to store text
The data of part system；On the medium that non-volatile small grain size is read and write, operate with small grain size of the file system to metadata
Read-write, on the medium that non-volatile big granularity is read and write, the read-write that operates with big granularity of the file system to data；
The big granularity is generally the size of page or block, and small grain size is usually to be read and write by byte.
2. the system as claimed in claim 1, it is characterised in that the file system periodically will be stored in described non-volatile
On the medium that copies of metadata on the medium of small grain size read-write is read and write to the non-volatile big granularity.
3. system as claimed in claim 1 or 2, it is characterised in that if file system can read and write non-volatile small grain size
Medium and internal memory as medium that the non-volatile small grain size of storage, i.e. system identification reads and writes is mixed as internal memory, then file system
The direct operating file system on the medium that non-volatile small grain size is read and write of system, to the direct establishment of metadata, renewal and deletion.
4. system as claimed in claim 1 or 2, it is characterised in that if file system can not read and write non-volatile small grain size
Medium mix with internal memory, i.e., system can not recognize the medium of non-volatile small grain size read-write as internal memory, then non-volatile small
Metadata in the medium of granularity read-write is operated in needing to call in internal memory, and metadata is through rambus in non-volatile granule
Called in and write back between the medium and internal memory of spending read-write.
5. the system as claimed in claim 1, it is characterised in that the medium of the non-volatile small grain size read-write is magnetic random
Access memory, resistive random access storage device or phase transition storage.
6. the system as claimed in claim 1, it is characterised in that the medium of the non-volatile big granularity read-write is solid-state disk.
7. the system as claimed in claim 1, it is characterised in that the metadata of the file system refers to the data of management space
The metadata and directory tree structure of file in structure and file system, or the only data structure of management space distribution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410097343.4A CN103902669B (en) | 2014-03-17 | 2014-03-17 | A kind of separate type file system based on different storage mediums |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410097343.4A CN103902669B (en) | 2014-03-17 | 2014-03-17 | A kind of separate type file system based on different storage mediums |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103902669A CN103902669A (en) | 2014-07-02 |
| CN103902669B true CN103902669B (en) | 2017-06-16 |
Family
ID=50993991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410097343.4A Active CN103902669B (en) | 2014-03-17 | 2014-03-17 | A kind of separate type file system based on different storage mediums |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103902669B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104536903B (en) * | 2014-12-25 | 2018-02-23 | 华中科技大学 | A kind of mixing storage method and system stored classifiedly by data attribute |
| CN105630692A (en) * | 2015-05-22 | 2016-06-01 | 上海磁宇信息科技有限公司 | File storage system storing file directory by using MRAM |
| CN105718530B (en) * | 2016-01-15 | 2020-02-04 | 上海磁宇信息科技有限公司 | File storage system and file storage control method thereof |
| CN107526537B (en) * | 2016-06-22 | 2020-03-20 | 伊姆西Ip控股有限责任公司 | Method and system for locking storage area in storage system |
| CN106599236A (en) * | 2016-12-20 | 2017-04-26 | 北海市云盛科技有限公司 | Metadata storage method and apparatus for file system |
| CN107479827A (en) * | 2017-07-24 | 2017-12-15 | 上海德拓信息技术股份有限公司 | A kind of mixing storage system implementation method based on IO and separated from meta-data |
| CN107590285A (en) * | 2017-09-30 | 2018-01-16 | 郑州云海信息技术有限公司 | A kind of method of heterogeneous system data consistency |
| CN107766002A (en) * | 2017-10-20 | 2018-03-06 | 华中科技大学 | A kind of virtual mixed file system based on mixing storage device |
| CN108196795B (en) * | 2017-12-30 | 2020-09-04 | 惠龙易通国际物流股份有限公司 | Data storage method and device and computer storage medium |
| CN109669640B (en) | 2018-12-24 | 2023-05-23 | 浙江大华技术股份有限公司 | Data storage method, device, electronic equipment and medium |
| CN116069263B (en) * | 2023-03-07 | 2023-07-14 | 苏州浪潮智能科技有限公司 | File system optimization method, device, server, equipment and storage medium |
| CN117453632B (en) * | 2023-12-25 | 2024-04-12 | 杭州计算机外部设备研究所(中国电子科技集团公司第五十二研究所) | Data storage method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102075581A (en) * | 2011-01-25 | 2011-05-25 | 中国科学院计算技术研究所 | Data transmission method and device oriented to distributed file system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6024776A (en) * | 1997-08-27 | 2000-02-15 | Kennametal Inc. | Cermet having a binder with improved plasticity |
| WO2010028279A1 (en) * | 2008-09-05 | 2010-03-11 | Arcsight, Inc. | Storing log data efficiently while supporting querying |
| CN102364474B (en) * | 2011-11-17 | 2014-08-20 | 中国科学院计算技术研究所 | Metadata storage system for cluster file system and metadata management method |
-
2014
- 2014-03-17 CN CN201410097343.4A patent/CN103902669B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102075581A (en) * | 2011-01-25 | 2011-05-25 | 中国科学院计算技术研究所 | Data transmission method and device oriented to distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103902669A (en) | 2014-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103902669B (en) | A kind of separate type file system based on different storage mediums | |
| CN103838853B (en) | Mixed file system based on different storage media | |
| CN111033477B (en) | Logical to physical mapping | |
| US9665305B1 (en) | Tiering data between two deduplication devices | |
| US8838903B2 (en) | Priority ordered multi-medium solid-state storage system and methods for use | |
| US9652386B2 (en) | Management of memory array with magnetic random access memory (MRAM) | |
| DE102017104150B4 (en) | Wear compensation in memory devices | |
| US20120158674A1 (en) | Indexing for deduplication | |
| US20130151759A1 (en) | Storage device and operating method eliminating duplicate data storage | |
| CN107784121B (en) | Lowercase optimization method of log file system based on nonvolatile memory | |
| US20180024919A1 (en) | Mapping tables for storage devices | |
| US8650379B2 (en) | Data processing method for nonvolatile memory system | |
| CN109416666A (en) | Caching with compressed data and label | |
| CN104395904A (en) | Efficient data object storage and retrieval | |
| US20170131943A1 (en) | Management of memory array with magnetic random access memory (mram) | |
| CN102201005A (en) | Method for distributing streaming media enhanced file of file system | |
| WO2017213908A1 (en) | Method and apparatus for improving performance of sequential logging in a storage device | |
| US20170285961A1 (en) | 3d stackable hybrid phase change memory with improved endurance and non-volatility | |
| CN102520885B (en) | Data management system for hybrid hard disk | |
| CN101996228A (en) | Method for sharing file data block and method for de-sharing file data block | |
| WO2020057479A1 (en) | Address mapping table item page management | |
| KR102589609B1 (en) | Snapshot management in partitioned storage | |
| US10140029B2 (en) | Method and apparatus for adaptively managing data in a memory based file system | |
| US20190129627A1 (en) | Method and system for wear-leveling using a multi-gap progress field | |
| CN112083883B (en) | Storage system and method for use in storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |