CN102521147B - Management method by using rapid non-volatile medium as cache - Google Patents
Management method by using rapid non-volatile medium as cache Download PDFInfo
- Publication number
- CN102521147B CN102521147B CN201110364311.2A CN201110364311A CN102521147B CN 102521147 B CN102521147 B CN 102521147B CN 201110364311 A CN201110364311 A CN 201110364311A CN 102521147 B CN102521147 B CN 102521147B
- Authority
- CN
- China
- Prior art keywords
- buffer memory
- state hard
- solid state
- hard disc
- memory device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides a management method by using a rapid non-volatile medium as a cache. According to the method, a solid-state hard disk is used as a caching device, each caching device is shared by multiple disks, and each disk can only use one caching device; the caching device is divided into a region structure, and mapping connection is adopted between the caching devices and the disks; the caching device is divided into two parts, the front part is used for storing the region data structure in a memory, and the back part is a cache. The method provided by the invention adopts a policy that metadata can be written in the solid-state hard disk in real time to solve the problem, and synchronizes the metadata for managing the dirty data in an assigned position of the solid-state hard disk while the dirty data are written in the solid-state hard disk, so that all the data cached in the solid-sate hard disk without being written in the conventional disk can be read only by reading the metadata when the system is restarted, and further all the data can be prevented from being lost when the system breaks down and is in power failure.
Description
Technical field
The present invention relates to disk buffering administrative skill, specifically, relate to a kind of management strategy of data buffer storage in high speed non-volatile media of disk at a slow speed.
Background technology
Disk buffering is divided into be read buffer memory and writes buffer memory.Read buffer memory to refer to, operating system is the file data read, and stays in memory headroom when internal memory is more idle, just need not again read from disk when next software or user read identical file again, thus improves speed.Writing data that buffer memory will write disk in fact exactly, to be first stored in system be write in the memory headroom of Cache sharing, when being saved in the data in memory pool and reaching a degree, just data is saved in hard disk.Can reduce actual disk operating like this, effective protection disk avoids the read-write operation of repetition and the damage that causes, also can reduce the time needed for write.According to the difference of writing mode, write general formula and rewriting two kinds.Write general formula when reading hard disc data, system first checks request instruction, looks at whether in the buffer desired data, words just sent the data of response by buffer memory, this process is called hit.Such system just need not data in access hard disk, because the speed ratio magnetizing mediums of SDRAM is a lot of soon, therefore also just accelerate the speed of data transmission.Rewriting write hard disc data time also look in the buffer, find just by buffer memory with regard in data write dish, most hard disk be all employing write-back cache, so just substantially increase performance.
But adopt volatile devices as internal memory, after system crash or power down, the data be buffered on high-speed equipment will be lost, this is insufferable to being similar to the such application program very high concerning data security requirement of database.
Summary of the invention
The present invention uses high speed non-volatile media to substitute volatibility high-speed equipment to carry out buffer memory data in magnetic disk at a slow speed, even if when system crash or power down, the data of buffer memory also can not be lost, after restart can by the data reading of buffer memory by the form of agreement.
Use high speed non-volatile media to do a management method for buffer memory, use solid state hard disc as buffer memory device, each buffer memory device can by multiple magnetic disc shares, and each disk can only use a buffer memory device; Described buffer memory device divides region structure, and between buffer memory device with disk, employing group maps and is connected; Buffer memory device is divided into two parts, the region data structure in previous section stored memory, and aft section is buffer memory.
Preferably, described solid state hard disc can form raid disk array expansion buffer memory capacity and improve performance.
Preferably, described region size is 64KB ~ 1MB, and is the power of 2, and described region can not be modified after equipment runs.
Preferably, when described buffer memory device and employing group between disk map and be connected, acquiescence often organizes 16 region, can by grand change
Preferably, described buffer memory device, when read operation, finds corresponding region data structure according to the address information recorded in bio, then checks whether data buffer storage is in solid state hard disc by region data structure, if so, then directly to read from solid state hard disc; Otherwise, read from disk and be cached to solid state hard disc.
Preferably, described buffer memory device, when write operation, finds corresponding region data structure according to the address information recorded in bio, if available free solid state hard disc space, then directly data is write toward solid state hard disc; Otherwise, write toward conventional hard; If write toward solid state hard disc, upgrade corresponding region data structure metadata information to solid state hard disc simultaneously.
Preferably, described buffer memory device when power down, when after system failure recovery, can scan the metadata information left in solid state hard disc after system crash, and according to all environment before these Information recovering, the dirty data be buffered in solid state hard disc of staying alone.
The present invention takes strategy metadata being write in real time solid state hard disc to solve this problem, while dirty data writes into solid state hard disc, the metadata managing these data is also synchronized to the assigned address of solid state hard disc, as long as read these metadata when restarting can be buffered in solid state hard disc by all and do not write the data reading of traditional magnetic disk, thus all data also can ensure when system crash and power down not lose.
Accompanying drawing explanation
Fig. 1 is overall structure schematic diagram of the present invention
Fig. 2 is mapping relations schematic diagram of the present invention
Detailed description of the invention
The present invention utilizes the Device Mapper mechanism of Linux to manage multiple block device, sets up a two-level memory system using high-speed equipment wherein as the buffer memory of low-speed device, and obtain higher memory property with lower cost, overall structure as shown in Figure 1.
Managed equipment is divided into buffer memory device and disk unit by the present invention, the solid state hard disc that wherein buffer memory device serviceability is higher, and in actual use, available polylith solid state hard disc composition RAID is to improve the performance of buffer memory.Buffer memory device and disk unit are all sightless to user, but the present invention can provide identical with disk unit number, that characteristic is identical pseudo-device to user.The present invention can manage multiple buffer memory device and disk unit, and buffer memory device and disk unit are the relations of one-to-many, and namely a buffer memory device can be shared by multiple disk unit, but any one disk unit uses a buffer memory device only.As for equipment, specifically then correspond to disk partition, the block device file/dev/sda1 etc. namely under Linux.
Why selecting the equipment more at a high speed such as solid state hard disc instead of direct use internal memory, be because the data on solid state hard disc can not be lost after system crash or power down, and the capacity of solid state hard disc is larger, can the more data of buffer memory.But, although the data of buffer memory can be kept in solid state hard disc, certain measure must be taked to obtain the management of metadata of these data, although otherwise data exist but we but cannot analyze the real meaning of these data.
The present invention makes buffer memory to disk, and the thinking of itself is relatively more directly perceived, and be similar to processor cache, buffer memory device is divided into many region by fixed size, usually, the size of region is 64K ~ 1M, and is the power of 2, can not dynamically change once run again.Region is the base unit of buffer memory device management, can cache line in analogy processor.Each region safeguards corresponding state, the necessary information such as lock, tag.Region is larger relative to the request of file system, the least unit of file system IO is generally 512 bytes (being referred to as sector), therefore an IO of file system is more may be the subregion that cover only certain region, Given this, each region also needs to safeguard some bitmaps, is used for marking the state of wherein each sector.The mapping of buffer memory device and disk unit adopts set associative, gives tacit consent to one group of 16 region, can to grand adjustment of being correlated with.
The data structure (struct region) of management data mapping relations is arranged in internal memory, and internal memory is volatile devices, will lose these data structures after system crash or power down.Although in fact the data on solid state hard disc do not lose, because the struct region of these data of management loses, so in fact data also just lost.In order to address this problem, the present invention adopts structure as shown in Figure 2.Solid state hard disc is divided into two parts, and front portion preserves the struct region data structure in internal memory, and rear portion is only the data of real cache.Like this, when finding and leave latter part of data in by reading the struct region leaving solid state hard disc front portion in after system crash or power down, and rebuild whole file system environment, thus ensure that the data of buffer memory are not lost.Any amendment to struct region all will not be synchronized to solid state hard disc during practical application, such expense is larger.Only can revise struct region again when data change, particularly will record those and be buffered in solid state hard disc and there is no the situation of change in the data of traditional magnetic disk.
Read operation.Find corresponding struct region according to the address information recorded in bio, then check whether data buffer storage is in solid state hard disc by struct region, if so, then directly to read from solid state hard disc; Otherwise, read from disk and be cached to solid state hard disc.
Write operation.Find corresponding struct region according to the address information recorded in bio, if available free solid state hard disc space, then directly data are write toward solid state hard disc; Otherwise, write toward conventional hard.If write toward solid state hard disc, upgrade corresponding struct region metadata information to solid state hard disc simultaneously.
Recovery operation.After system crash when power down, when after system failure recovery, the present invention can scan the metadata information left in solid state hard disc, and according to all environment before these Information recovering, the dirty data be now buffered in solid state hard disc can intactly read.
Claims (4)
1. use high speed non-volatile media to do a management method for buffer memory, it is characterized in that: use solid state hard disc as buffer memory device, each buffer memory device can by multiple magnetic disc shares, and each disk can only use a buffer memory device; Described buffer memory device divides region structure, and between buffer memory device with disk, employing group maps and is connected; Buffer memory device is divided into two parts, the region data structure in previous section stored memory, and aft section is buffer memory;
Described region size is 64KB ~ 1MB, and is the power of 2, and described region can not be modified after equipment runs;
When described buffer memory device and employing group between disk map and be connected, acquiescence often organizes 16 region, by grand change;
Described solid state hard disc composition raid disk array expands buffer memory capacity and improves performance.
2. the method for claim 1, it is characterized in that: described buffer memory device is when read operation, corresponding region data structure is found according to the address information recorded in bio, then check whether data are buffered in solid state hard disc by region data structure, if so, then directly to read from solid state hard disc; Otherwise, read from disk and be cached to solid state hard disc.
3. the method for claim 1, is characterized in that: described buffer memory device, when write operation, finds corresponding region data structure according to the address information recorded in bio, if available free solid state hard disc space, then directly data is write toward solid state hard disc; Otherwise, write toward conventional hard; If write toward solid state hard disc, upgrade corresponding region data structure metadata information to solid state hard disc simultaneously.
4. the method for claim 1, it is characterized in that: described buffer memory device is after system crash when power down, when after system failure recovery, the metadata information left in solid state hard disc can be scanned, and according to all environment before these Information recovering, read the dirty data be buffered in solid state hard disc.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110364311.2A CN102521147B (en) | 2011-11-17 | 2011-11-17 | Management method by using rapid non-volatile medium as cache |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110364311.2A CN102521147B (en) | 2011-11-17 | 2011-11-17 | Management method by using rapid non-volatile medium as cache |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102521147A CN102521147A (en) | 2012-06-27 |
| CN102521147B true CN102521147B (en) | 2015-03-25 |
Family
ID=46292077
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110364311.2A Active CN102521147B (en) | 2011-11-17 | 2011-11-17 | Management method by using rapid non-volatile medium as cache |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102521147B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103827837B (en) * | 2012-08-22 | 2016-08-03 | 华为技术有限公司 | Access the method in solid state hard disc space, Apparatus and system |
| CN102945207A (en) * | 2012-10-26 | 2013-02-27 | 浪潮(北京)电子信息产业有限公司 | Cache management method and system for block-level data |
| CN103345450B (en) * | 2013-06-26 | 2016-11-02 | 浙江宇视科技有限公司 | The apparatus and method that a kind of dynamic adjustment redundancy RAID array is rebuild |
| CN103678166A (en) * | 2013-08-16 | 2014-03-26 | 记忆科技(深圳)有限公司 | Method and system for using solid-state disk as cache of computer |
| CN103744796A (en) * | 2013-09-29 | 2014-04-23 | 记忆科技(深圳)有限公司 | Caching method and system by means of uSSD |
| CN103744611A (en) * | 2013-12-17 | 2014-04-23 | 记忆科技(深圳)有限公司 | Computer system based on solid state disc as cache and cache accelerating method |
| CN104793892B (en) * | 2014-01-20 | 2019-04-19 | 优刻得科技股份有限公司 | A method of accelerating disk stochastic inputs output (IO) read-write |
| CN104866438B (en) * | 2014-02-20 | 2019-02-05 | 联想(北京)有限公司 | Storage device, storage apparatus control method and electronic equipment |
| CN104050014B (en) * | 2014-05-23 | 2017-12-29 | 上海爱数信息技术股份有限公司 | Efficient storage management method based on virtual platform |
| CN105094711B (en) * | 2015-09-22 | 2018-05-18 | 浪潮(北京)电子信息产业有限公司 | A kind of method and device for realizing copy-on-write file system |
| CN105302656B (en) * | 2015-10-21 | 2018-09-04 | 浪潮(北京)电子信息产业有限公司 | A kind of data test method and device |
| CN106443740A (en) * | 2016-09-13 | 2017-02-22 | 上海顺砾智能科技有限公司 | Rapid positioning system and method applied to unmanned aerial vehicle |
| CN106528001B (en) * | 2016-12-05 | 2019-08-23 | 北京航空航天大学 | A kind of caching system based on nonvolatile memory and software RAID |
| CN107229573B (en) * | 2017-05-22 | 2020-04-28 | 上海天玑数据技术有限公司 | Elastic high-availability caching method based on solid state disk |
| CN107402724B (en) * | 2017-07-31 | 2020-07-24 | 苏州浪潮智能科技有限公司 | Method and system for storing Journal metadata in SSD |
| CN107402819A (en) * | 2017-08-04 | 2017-11-28 | 郑州云海信息技术有限公司 | The management method and system of a kind of client-cache |
| CN107480074B (en) * | 2017-08-31 | 2020-02-07 | 郑州云海信息技术有限公司 | Caching method and device and electronic equipment |
| CN107704400A (en) * | 2017-10-12 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of caching of page method towards non-volatile memory |
| CN108829835A (en) * | 2018-06-19 | 2018-11-16 | 深圳市瑞驰信息技术有限公司 | A kind of novel high speed cache file system architecture |
| CN113312300B (en) * | 2021-06-17 | 2024-05-03 | 上海天玑科技股份有限公司 | Nonvolatile memory caching method integrating data transmission and storage |
| CN115437853B (en) * | 2022-11-07 | 2023-03-10 | 四川省华存智谷科技有限责任公司 | Cache data protection method and system under system abnormal restart |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5860083A (en) * | 1996-11-26 | 1999-01-12 | Kabushiki Kaisha Toshiba | Data storage system having flash memory and disk drive |
| CN1465009A (en) * | 2000-06-23 | 2003-12-31 | 英特尔公司 | Non-volatile cache integrated with mass storage device |
| CN1527973A (en) * | 2000-06-23 | 2004-09-08 | 英特尔公司 | Non-volatile cache |
| CN201698255U (en) * | 2010-05-14 | 2011-01-05 | 上海浙大网新易得科技发展有限公司 | Server capable of accessing disc at high speed |
-
2011
- 2011-11-17 CN CN201110364311.2A patent/CN102521147B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5860083A (en) * | 1996-11-26 | 1999-01-12 | Kabushiki Kaisha Toshiba | Data storage system having flash memory and disk drive |
| CN1465009A (en) * | 2000-06-23 | 2003-12-31 | 英特尔公司 | Non-volatile cache integrated with mass storage device |
| CN1527973A (en) * | 2000-06-23 | 2004-09-08 | 英特尔公司 | Non-volatile cache |
| CN201698255U (en) * | 2010-05-14 | 2011-01-05 | 上海浙大网新易得科技发展有限公司 | Server capable of accessing disc at high speed |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102521147A (en) | 2012-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102521147B (en) | Management method by using rapid non-volatile medium as cache | |
| US10108545B2 (en) | Operating shingled magnetic recording device | |
| US10521131B2 (en) | Storage apparatus and storage control apparatus | |
| US8521949B2 (en) | Data deleting method and apparatus | |
| US7979631B2 (en) | Method of prefetching data in hard disk drive, recording medium including program to execute the method, and apparatus to perform the method | |
| US20170322888A1 (en) | Zoning of logical to physical data address translation tables with parallelized log list replay | |
| US9323687B2 (en) | Use of differing granularity heat maps for caching and migration | |
| US20090327608A1 (en) | Accelerated resume from hibernation in a cached disk system | |
| US9798623B2 (en) | Using cache to manage errors in primary storage | |
| CN105718206B (en) | The flash translation layer (FTL) and its implementation of RAID can be perceived | |
| US9086991B2 (en) | Solid state drive cache recovery in a clustered storage system | |
| CN104903872A (en) | Systems, methods, and interfaces for adaptive persistence | |
| JP6713934B2 (en) | Storage device and its operating method and system | |
| WO2018090255A1 (en) | Memory access technique | |
| CN109739696B (en) | Double-control storage array solid state disk caching acceleration method | |
| US11144224B2 (en) | Locality-aware, memory-efficient, time-efficient hot data identification using count-min-sketch for flash or streaming applications | |
| US8977816B2 (en) | Cache and disk management method, and a controller using the method | |
| CN104750433A (en) | Cache design method based on SCST | |
| CN111488125A (en) | Cache Tier Cache optimization method based on Ceph cluster | |
| US20040039875A1 (en) | Disk array device and virtual volume management method in disk array device | |
| CN105786721A (en) | Memory address mapping management method and processor | |
| CN111443874B (en) | Solid-state disk memory cache management method and device based on content awareness and solid-state disk | |
| CN101807212B (en) | Caching method for embedded file system and embedded file system | |
| JP4734432B2 (en) | Data storage system | |
| Yang et al. | A shingle-aware persistent cache management scheme for DM-SMR disks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220725 Address after: 100193 No. 36 Building, No. 8 Hospital, Wangxi Road, Haidian District, Beijing Patentee after: Dawning Information Industry (Beijing) Co.,Ltd. Patentee after: DAWNING INFORMATION INDUSTRY Co.,Ltd. Address before: 100084 Beijing Haidian District City Mill Street No. 64 Patentee before: Dawning Information Industry (Beijing) Co.,Ltd. |