WO2016181481A1 - ストレージシステムおよび記憶制御方法 - Google Patents
ストレージシステムおよび記憶制御方法 Download PDFInfo
- Publication number
- WO2016181481A1 WO2016181481A1 PCT/JP2015/063563 JP2015063563W WO2016181481A1 WO 2016181481 A1 WO2016181481 A1 WO 2016181481A1 JP 2015063563 W JP2015063563 W JP 2015063563W WO 2016181481 A1 WO2016181481 A1 WO 2016181481A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- physical
- data
- area
- write
- address space
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/16—Protection against loss of memory contents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/70—Details relating to dynamic memory management
- G06F2212/702—Conservative garbage collection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
- G06F3/0641—De-duplication techniques
Definitions
- the present invention generally relates to garbage collection in a storage system.
- update writing that updates at least a part of already stored data
- deduplication that eliminates at least one of a plurality of overlapping data
- already stored data Old data in arithmetic compression to be compressed (post-process arithmetic compression) (for example, data before update by update writing, data excluded by deduplication, or data before arithmetic compression)
- the data storage area becomes invalid. Since the invalidated area becomes a free area, such invalidation causes fragmentation of the free area. Therefore, a storage system that employs a log structuring method requires garbage collection (GC) for collecting fragmented free areas (fragmented invalid areas).
- GC garbage collection
- Patent Document 1 relates to a GC control method.
- the technology of Patent Document 1 uses a logical / physical table for deriving a physical address from a logical address by identifying valid data (non-invalid survival data) using a physical table for deriving a logical address from a physical address.
- the logical address of the valid data is specified, the valid data is copied to another area, and the physical theory table and the logical / physical table are updated so as to associate the physical address of the copy destination with the logical address of the valid data.
- the technique of Non-Patent Document 1 can divide a physical address space into segments having a certain size, and select a segment with high GC efficiency as a copy source.
- the technique of Non-Patent Document 1 formalizes the obtained GC efficiency by Equation 1 for each segment, and selects a segment with high GC efficiency as a copy source.
- the number 1 is an expression that means the idea of GC efficiency.
- the live data space (live data space) that is the amount of data to be copied in the segment is the cost u
- the age of the segment is The segment free capacity (1-u) (free space) multiplied by the indicated coefficient a (age) is defined as segment gain (benefit), and the gain relative to cost is defined as GC efficiency (benefit / cost).
- the age of the segment is an average of time stamps of data in the segment.
- a RAID level that requires at least one parity per stripe (for example, RAID5, RAID6) can reduce the GC efficiency for reading the parity at such a RAID level.
- the controller in the storage system uses the old parity (parity already stored) and the old data. Must be read from each drive device. In this case, new data is generated by updating a part of the old data, and new parity is generated by an arithmetic operation using the new data and the old parity.
- Such a problem may also exist for other storage systems that employ a log structuring method regardless of whether or not at least one of arithmetic compression and deduplication is employed.
- the storage system has a plurality of physical storage devices including one or more RAID groups, and a controller connected to the plurality of physical storage devices and including a processor and a storage unit.
- Each area of one or more RAID groups is divided into a plurality of stripes.
- Each of the plurality of stripes is composed of a plurality of strips which are a plurality of areas respectively corresponding to a plurality of physical storage devices.
- Each RAID configuration of one or more RAID groups is a RAID configuration that requires parity for each stripe.
- Address conversion information which is information referred to by the controller for address conversion between a logical address in the logical address space and a physical address in the physical address space for one or more RAID groups, is stored in the storage unit and the physical storage device.
- Present in at least one of the Data is additionally written in the physical address space by the controller.
- a fragment free area is generated by invalidating old data, which is data to be updated, compressed, or excluded, by the controller.
- the controller executes a copy GC (garbage collection) process using the address conversion information.
- the copy GC process selects (A) one or more non-writeable physical areas each including a fragment free area as one or more copy source physical areas from a plurality of physical areas which are a plurality of areas constituting the physical address space. (B) selecting a recordable physical area as a copy destination physical area from a plurality of physical areas, and (C) selecting one or more survival data in one or more selected copy source physical areas.
- the GC performance in the storage system can be improved.
- FIG. 1 shows a configuration example of a storage system according to a first embodiment.
- An example of data storage in a RAID group by a log structuring method is shown.
- a configuration example of each of the logical-physical table and the physical-logic table is shown.
- 2 shows an example of a GC processing flow according to the first embodiment.
- 10 shows a configuration example of a storage system according to Embodiment 2.
- An example of a processing flow for selecting a copy source is supported.
- An example of GC efficiency distribution is shown.
- Another example of the GC efficiency distribution is shown.
- 6 is an explanatory diagram of a threading GC according to Embodiment 3.
- FIG. FIG. 10 is an explanatory diagram of Copying GC according to the third embodiment.
- 18 shows an example of the relationship between the size of a page in a physical address space according to the fourth embodiment and the block size of the SMR drive according to the fourth embodiment.
- each table may be divided into two or more tables, or all or part of the two or more tables may be a single table. Good.
- the “storage unit” may be one or more storage devices including a memory.
- the storage unit may be at least a main storage device of a main storage device (typically a volatile memory) and an auxiliary storage device (typically a nonvolatile storage device).
- PDEV indicates a physical storage device, and may typically be a nonvolatile storage device (for example, an auxiliary storage device).
- the PDEV may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).
- the PDEV is called a “drive device”.
- RAID is an abbreviation of Redundant Array Of Independent (or Inexpensive) Disks.
- the RAID group is composed of a plurality of drive devices, and stores data according to the RAID level associated with the RAID group.
- a RAID group may be referred to as a parity group.
- the parity group may be, for example, a RAID group that stores parity.
- the program distribution server may include a processor (for example, a CPU) and a storage unit, and the storage unit may further store a distribution program and a program to be distributed. Then, the processor of the program distribution server executes the distribution program, so that the processor of the program distribution server may distribute the distribution target program to other computers.
- a processor for example, a CPU
- the storage unit may further store a distribution program and a program to be distributed. Then, the processor of the program distribution server executes the distribution program, so that the processor of the program distribution server may distribute the distribution target program to other computers.
- two or more programs may be realized as one program, or one program may be realized as two or more programs.
- VOL is an abbreviation for logical volume and may be a logical storage device.
- the VOL may be a substantial VOL (RVOL) or a virtual VOL (VVOL).
- the VOL may be an online VOL provided to a host system connected to a storage system that provides the VOL, and an offline VOL that is not provided to the host system (not recognized by the host system).
- RVOL may be a VOL based on a drive device (for example, one or more RAID groups) included in the storage system having the RVOL.
- VVOL may be at least one of an external connection VOL (EVOL), a capacity expansion VOL (TPVOL), and a snapshot VOL.
- the EVOL is based on the storage space (for example, VOL) of the external storage system and may be a VOL according to the storage virtualization technology.
- the TPVOL is composed of a plurality of virtual areas (virtual storage areas), and may be a VOL that complies with capacity virtualization technology (typically, Thin Provisioning).
- the snapshot VOL may be a VOL provided as a snapshot of the original VOL.
- the TPVOL is typically an online VOL.
- the snapshot VOL may be an RVOL.
- a “pool” is a logical storage area (for example, a set of a plurality of pool VOLs), and may be prepared for each use.
- the pool may include at least one of a TP pool and a snapshot pool.
- the TP pool may be a storage area composed of a plurality of real areas (substantial storage areas).
- a real area may be allocated from the TP pool to the TPVOL virtual area (for example, if a real area is not allocated to the write destination virtual area of the TPVOL, an unallocated real area is allocated from the TP pool to the write destination virtual area. May be).
- the snapshot pool may be a storage area in which data saved from the original VOL is stored.
- One pool may be used as both a TP pool and a snapshot pool.
- the “pool VOL” may be a VOL that is a component of the pool.
- the pool VOL may be an RVOL or an EVOL.
- the pool VOL is typically an offline VOL.
- the client (an example of a host system) 1 receives a data service provided by the storage system 2 via the network 3.
- a plurality of clients 1 may be provided.
- clients 1a and 1b are shown as clients 1.
- the network 3 may be an intra network, but may be the Internet or LAN (Local Area Network) connecting the client 1 and the storage system 2 via a network switch, or may be a SAN (Storage Area Network).
- the storage system 2 stores data transmitted from the client 1 (stores data to be written accompanying the write request received from the client 1), and also provides stored data to the client 1 (received from the client 1) The data to be read according to the read request is provided to the client 1).
- the storage system 2 may be composed of a plurality of storage devices, but in FIG. 1, the storage system 2 is composed of one storage device.
- the storage system 2 can provide various data services. Data services include, for example, data copy within a storage device, data remote copy between storage devices, and snapshot acquisition.
- the drive 111 stores at least a part of the data from the client 1.
- the plurality of drives 111 constitute a plurality (or one) of RAID groups 104.
- RAID groups 104a and 104b are shown in FIG.
- the RAID group 104a includes four drives 111a to 111d, but at least one of the RAID groups 104 other than the RAID group 104a may have a configuration different from that of the RAID group 104a.
- the controller includes a processor 101, a network interface 102, and a memory (an example of a storage unit) 103.
- the processor 101 executes the calculation of the program expanded in the memory 103.
- the network interface 102 is an interface for connecting to the network 3. Examples of programs developed in the memory 103 include an OS (Operating System) 201, a scheduler 202, a VOL controller 203, an arithmetic compression 204, a deduplication 205, and a GC (garbage collection) 206.
- At least one part of the processing performed by the processor 101 for example, at least one of arithmetic compression and deduplication is performed in place of the processor 101 by a hardware circuit (for example, Application Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA) ).
- the memory 103 is also used as a cache memory in which data written to the RAID group 104 and data read from the RAID group 104 are temporarily written.
- OS201 provides a memory load function, which is a basic function for other programs to operate.
- the scheduler 202 controls the operation timing of the VOL controller 203, arithmetic compression 204, deduplication 205, and GC206.
- the deduplication 205 checks for duplication (identity) between data stored in the drive 111, leaves at least one of the duplicate data, and deletes the remainder of the duplicate data.
- the deduplication 205 may perform deduplication by either inline or post process. In the case of inline, the deduplication 205 executes deduplication during the writing process according to the write request from the client 1. In the case of the post process, the deduplication 205 once writes the write target data from the client 1 to the drive 111, and then asynchronously reads the target data from the drive 111 to the memory 103 to execute deduplication.
- GC includes Threading GC and Copying GC.
- GC206 determines whether the total of the data arranged continuously in F5 is equal to or larger than the stripe data size (F6).
- the GC 206 selects an appendable page as the copy destination page.
- the additionally recordable page can be specified from the physical theory table T2.
- the GC 206 writes stripe data to the first empty stripe of the copy destination page. As a result, stripe data is written to the copy destination page in the order of physical addresses.
- the GC 206 determines whether or not GC has been performed for all target pages (F7). When the determination result of F7 is false, F2 to F5 are performed. In F2 to F5, if unwritten data remains in the memory 103 in the copy destination page, the survival data is arranged from the next of the data.
- the size of unwritten data is equal to or larger than the strip size but is not an integral multiple of the strip size. In this case, neither strip unit writing nor stripe unit writing can be performed without padding.
- the size of unwritten data is equal to or larger than the strip size and an integral multiple of the strip size, but writing in stripe units is required. In this case, writing in stripe units cannot be performed without padding.
- the table update in F11 and F13 is performed as follows, for example.
- the GC 206 updates the copy source physical addresses in the physical table T2 and the logical table T1 to copy destination physical addresses for all the survival data copied (written) on the copy destination page.
- the padding data write destination area is treated as an unused area.
- the physical theory table T2 holds not only the logical address of the data that was not excluded by deduplication but also all the linked logical addresses for one physical address
- the physical theory table The physical address indicated by the logical address in the logical / physical table T1 can be rewritten by referring to the logical address of the corresponding data from T2.
- padding is performed only when the size of the last unwritten data is less than the size of the stripe data or the size of the strip data. For this reason, the write load can be further reduced.
- the last unwritten data can be written in strip units. In the case of writing in strip units, it is not necessary to read data from the writing destination. For this reason, the write load can be reduced.
- the padding size may be limited to less than the strip size. Thereby, the load about padding can be reduced. Even if the padding size is less than the strip size, at least strip unit writing is possible between writing in stripe units and writing in strip units.
- unwritten data can be converted to stripe data with a slight padding. (10-3)
- padding for generating one strip data is performed. This is because writing can be executed with less padding than padding for generating stripe data.
- Example 1 was an example that focused on reducing the load of writing in GC (Copying GC).
- the present embodiment is an example paying attention to selection of a copy source page.
- FIG. 5 shows a configuration example of the storage system according to the second embodiment.
- FIG. 7 and FIG. 8 show the distribution efficiency of the logical / physical table T1 and the physical theory table T2, the variation in access frequency with respect to age, and the distribution of GC efficiency with different page conditions selected as a GC target. .
- These data are the results of creating a page having a random free space amount based on the actually measured values in the storage system 502 and simulating the GC efficiency by the Monte Carlo method. 7 and 8 are as follows. That is, the left diagram shows the GC efficiency distribution in a case where pages are selected as copy sources (GC targets) in chronological order (in order from the top of the physical address).
- the middle figure shows the GC efficiency distribution in the case where the logical / physical table T1 and the physical theory table T2 are in one or more drives 111 and a page with high GC efficiency obtained by Equation 1 is selected as a copy source.
- the right figure shows the GC efficiency distribution in the case where the logical-physical table T1 and the physical-logical table T2 are in the memory 103 and a page with high gain obtained by Equation 1 is selected as a copy source.
- “Segment” is read as “Page”.
- FIG. 7 shows the GC efficiency distribution when the age access frequency decrease rate is low
- FIG. 8 shows the GC efficiency distribution when the age access frequency decrease rate is high.
- the GC efficiency (benefit / cost) obtained as a result of FIGS. 7 and 8 was derived by Equation 1.
- Fig. 6 shows the processing flow for selecting the copy source page (page targeted for GC) according to the above results. This processing flow is executed in the GC processing flow (for example, executed between F1 and F2).
- the GC 506 selects the “page forward search” mode (F14). In this mode, GC target candidate pages are searched in chronological order (in the order of physical addresses), and the found pages are selected as copy sources.
- “Recordable drive” refers to SMR (Singled Magnetic Recording) drive, SSD (Solid State Drive) including NAND flash memory, etc. Each block in these drives has a drive recordable area (data in the order of address). Are areas to be written).
- the “block” here is an example of a unit area (for example, in the NAND flash memory, a block is a data erasing unit). The available free space cannot be secured without releasing the entire block. Therefore, the “high GC efficiency page search” mode for searching for a page with high GC efficiency that exists at random positions for a RAID group composed of a plurality of write-once drives is not suitable.
- the GC 506 specifies the rate of change in the access frequency decrease with respect to the physical address space by calling the data age statistics 208, and determines whether the rate of change in the access frequency decrease with age is slow ( For example, it is determined whether or not it is less than a predetermined threshold (F15). If the determination result of F15 is true, the GC 506 selects the “high GC efficiency page search” mode (F16). If the determination result of F15 is false, the GC 506 selects the “page forward search” mode. Note that, when the access-to-age access frequency decrease rate is fast, the “page forward search” mode is selected even when “high GC efficiency page search” and “page forward search” have the same GC efficiency. This is because it is guaranteed that free areas are grouped in order of physical addresses, and that the “page forward search” mode has a smaller TAT of the GC.
- the order of determination does not necessarily have to be F11, F13, and F15. Further, all determinations of F11, F13, and F15 may not be performed. For example, when the determination result of F11 is false, the page forward search mode may be selected without F13 and F15. Further, when the determination of F13 is performed without the determination of F11 and the determination result of F13 is true, the page forward search mode may be selected. Further, when the determination result of F13 is false, the high GC efficiency page search mode may be selected without performing any of the determination of F11 and the determination of F15.
- Example 3 will be described. At that time, the differences from the first and second embodiments will be mainly described, and the description of the common points with the first and second embodiments will be omitted or simplified.
- Threading GC there is only one type of GC used in a storage system that performs GC.
- Threading GC and Copying GC are employed.
- Threading GC is that at least one of the logical-physical table and the physical-logic table can be so large that it does not fit in memory for managing free space.
- the GC is performed in the same area unit as the area in which copying is performed by Copying GC.
- Coping GC for example, GC that starts in a predetermined GC cycle
- the controller copies the survival data of the pages 305A, 305B, 305C, and 305E that cannot be added to the page 305D that can be added.
- each of the pages 305A, 305B, 305C, and 305E becomes a recordable page.
- Threading GC is performed in addition to Copying GC, and Threading GC is performed in the same area unit as the area in which copying is performed in Copying GC. Therefore, Threading GC can be performed in addition to Copying GC without increasing the size of at least one of the logical / physical table T1 and the physical theory table T2.
- the controller determines the unit area size for the physical address space based on the block size of the write-once drive.
- each unit area in the physical address space is referred to as a “physical additional area”.
- a plurality of different block sets respectively correspond to the plurality of physical write-once areas.
- Each block set is a plurality of blocks respectively included in a plurality of different write-once drives in the write-once RAID group corresponding to the physical address space.
- the controller manages whether it is an area that cannot be additionally written or an area that can be additionally written in units of physical write-on areas instead of or in addition to pages.
- Copying GC the controller selects the copy destination of the survival data not in units of pages but in units of physical appending areas.
- SMR drive in FIG. 11
- one or more survival data is copied from one or more copy source blocks to a copy destination block.
- the one or more survival data are sequentially written in the copy destination block in the order of addresses.
- One or more copy source blocks correspond to one or more copy source physical write-once areas (non-writeable areas including fragment free areas).
- the copy destination block corresponds to the copy destination physical additional recording area.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Detection And Correction Of Errors (AREA)
Abstract
Description
(数1)benefit / cost = free space * age / live data space = (1-u) * a / u
(8-1)未書込みデータのサイズが、ストリップサイズ未満である。この場合、パディング無しに、ストリップ単位の書込み及びストライプ単位の書込みのいずれもできない。
(8-2)未書込みデータのサイズが、ストリップサイズ以上であるが、ストリップサイズの整数倍ではない。この場合、パディング無しに、ストリップ単位の書込み及びストライプ単位の書込みのいずれもできない。
(8-3)未書込みデータのサイズが、ストリップサイズ以上且つストリップサイズの整数倍であるが、ストライプ単位の書込みが必要とされる。この場合、パディング無しに、ストライプ単位の書込みをできない。
(10-1)未書込みデータのサイズが、ストライプサイズの1/2(50%)より大きければ(パディングサイズが、ストライプサイズの1/2未満であれば)、ストライプ単位の書込みのためのパディングが行われる。比較的パディングサイズが小さくて済むためである。
(10-2)未書込みデータのサイズをストリップサイズで割ったときの商Qが、(M-1)である場合、ストライプ単位の書込みのためのパディングが行われる。Mは、ストライプデータを構成するストリップデータの数である。従って、Q=M-1であるということは、パディングサイズがストリップサイズ未満である。この例によれば、わずかなパディングで未書込みデータをストライプデータにできる。
(10-3)未書込みデータのサイズが、ストリップサイズ未満の場合、1つのストリップデータの生成のためのパディングが行われる。ストライプデータを生成するためのパディングよりも少ないパディングで書込みを実行できるためである。
Claims (15)
前記複数の物理記憶デバイスに接続されプロセッサ及び記憶部を含んだコントローラと
を有し、
前記1以上のRAIDグループの各々の領域は、複数のストライプに分割されており、前記複数のストライプの各々は、複数の物理記憶デバイスにそれぞれ対応した複数の領域である複数のストリップで構成されており、
前記1以上のRAIDグループの各々のRAID構成は、ストライプ毎にパリティが必要なRAID構成であり、
論理アドレス空間の論理アドレスと、前記1以上のRAIDグループについての物理アドレス空間の物理アドレスとの間のアドレス変換のために前記コントローラにより参照される情報であるアドレス変換情報が、前記記憶部と物理記憶デバイスとのうちの少なくとも1つに存在し、
前記物理アドレス空間には、前記コントローラによりデータが追記的に書き込まれるようになっており、
前記物理アドレス空間には、更新、圧縮及び排除のうちのいずれかの対象のデータである旧データが前記コントローラにより無効化されることにより断片空き領域が生じるようになっており、
前記コントローラが、前記アドレス変換情報を用いてコピーGC(ガベージコレクション)処理を実行し、
前記コピーGC処理が、
(A)前記物理アドレス空間を構成する複数の領域である複数の物理領域から、それぞれ断片空き領域を含んだ1以上の追記不可物理領域を1以上のコピー元物理領域として選択すること、
(B)前記複数の物理領域から、追記可能物理領域をコピー先物理領域として選択すること、
(C)前記選択された1以上のコピー元物理領域内の1以上の生存データを、前記選択されたコピー先物理領域の空き領域の先頭からシーケンシャルに、ストリップ単位又はストライプ単位に書き込むこと
を含み、
(C)において、前記コントローラが、書込み対象データのサイズがストリップ単位又はストライプ単位の書込みに必要なサイズに満たない場合、その書込み対象データに対してパディングを行うことで、その書込み対象データをストリップ単位又はストライプ単位で書き込む、
ストレージシステム。
前記1以上の生存データのうちの最後の書込み対象データ以外の書込み対象データを、パディング無しにストライプ単位で書込み、
前記1以上の生存データのうちの最後の書込み対象データのサイズがストリップ単位又はストライプ単位の書込みに必要なサイズに満たない場合にのみ、ストリップ単位又はストライプ単位の書き込みためのパディングを行う、
請求項1記載のストレージシステム。
請求項2記載のストレージシステム。
(A)は、選択されたモードに従いコピー元物理領域を選択することである、
請求項1記載のストレージシステム。
請求項4記載のストレージシステム。
請求項5記載のストレージシステム。
請求項6記載のストレージシステム。
請求項4記載のストレージシステム。
請求項8記載のストレージシステム。
請求項9記載のストレージシステム。
請求項4記載のストレージシステム。
請求項4記載のストレージシステム。
請求項1記載のストレージシステム。
前記複数の追記型デバイスの各々は、複数のドライブ追記領域を有しており、
前記複数のドライブ追記領域の各々は、データがアドレス順に書き込まれるべき記憶領域であり、
前記物理アドレス空間内の前記複数の物理領域に、それぞれ、異なる複数のドライブ追記領域セットが対応付けられており、
前記複数のドライブ追記領域セットの各々は、前記複数の追記型デバイスがそれぞれ有する複数のドライブ追記領域である、
請求項1記載のストレージシステム。
前記ストレージシステムが、
1以上のRAIDグループを含む複数の物理記憶デバイスと、
前記複数の物理記憶デバイスに接続されプロセッサ及び記憶部を含んだコントローラと
を有し、
前記1以上のRAIDグループの各々の領域は、複数のストライプに分割されており、前記複数のストライプの各々は、複数の物理記憶デバイスにそれぞれ対応した複数の領域である複数のストリップで構成されており、
前記1以上のRAIDグループの各々のRAID構成は、ストライプ毎にパリティが必要なRAID構成であり、
論理アドレス空間の論理アドレスと、前記1以上のRAIDグループについての物理アドレス空間の物理アドレスとの間のアドレス変換のために前記コントローラにより参照される情報であるアドレス変換情報が、前記記憶部と物理記憶デバイスとのうちの少なくとも1つに存在し、
前記物理アドレス空間には、前記コントローラによりデータが追記的に書き込まれるようになっており、
前記物理アドレス空間には、更新、圧縮及び排除のうちのいずれかの対象のデータである旧データが前記コントローラにより無効化されることにより断片空き領域が生じるようになっており、
前記記憶制御方法が、
(A)前記アドレス変換情報を参照して、前記物理アドレス空間を構成する複数の領域である複数の物理領域から、それぞれ断片空き領域を含んだ1以上の追記不可物理領域を1以上のコピー元物理領域として選択し、
(B)前記アドレス変換情報を参照して、前記複数の物理領域から、追記可能物理領域をコピー先物理領域として選択し、
(C)前記選択された1以上のコピー元物理領域内の1以上の生存データを、前記選択されたコピー先物理領域の空き領域の先頭からシーケンシャルに、ストリップ単位又はストライプ単位に書き込み、書込み対象データのサイズがストリップ単位又はストライプ単位の書込みに必要なサイズに満たない場合、その書込み対象データに対してパディングを行うことで、その書込み対象データをストリップ単位又はストライプ単位で書き込む、
記憶制御方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/549,283 US10621083B2 (en) | 2015-05-12 | 2015-05-12 | Storage system and storage control method |
PCT/JP2015/063563 WO2016181481A1 (ja) | 2015-05-12 | 2015-05-12 | ストレージシステムおよび記憶制御方法 |
JP2017517503A JP6385570B2 (ja) | 2015-05-12 | 2015-05-12 | ストレージシステムおよび記憶制御方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2015/063563 WO2016181481A1 (ja) | 2015-05-12 | 2015-05-12 | ストレージシステムおよび記憶制御方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016181481A1 true WO2016181481A1 (ja) | 2016-11-17 |
Family
ID=57249076
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/063563 WO2016181481A1 (ja) | 2015-05-12 | 2015-05-12 | ストレージシステムおよび記憶制御方法 |
Country Status (3)
Country | Link |
---|---|
US (1) | US10621083B2 (ja) |
JP (1) | JP6385570B2 (ja) |
WO (1) | WO2016181481A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018173915A (ja) * | 2017-03-31 | 2018-11-08 | 日本電気株式会社 | バックアップシステム、ストレージ装置、データ転送方法及びプログラム |
CN110209340A (zh) * | 2018-02-28 | 2019-09-06 | 杭州宏杉科技股份有限公司 | 一种全闪存存储***的访问方法及装置 |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6721491B2 (ja) * | 2016-11-18 | 2020-07-15 | キオクシア株式会社 | 記憶システム |
JP2018181207A (ja) * | 2017-04-20 | 2018-11-15 | 富士通株式会社 | ストレージ制御装置、ストレージ制御方法及びストレージ制御プログラム |
JP2019057172A (ja) | 2017-09-21 | 2019-04-11 | 東芝メモリ株式会社 | メモリシステムおよび制御方法 |
US10884650B1 (en) * | 2017-10-25 | 2021-01-05 | EMC IP Holding Company LLC | Opportunistic compression of replicated data in a content addressable storage system |
JP6608468B2 (ja) * | 2018-01-26 | 2019-11-20 | 株式会社日立製作所 | ストレージ装置及びその制御方法 |
US10956078B2 (en) | 2018-03-27 | 2021-03-23 | EMC IP Holding Company LLC | Storage system with loopback replication process providing object-dependent slice assignment |
US10866969B2 (en) | 2018-03-28 | 2020-12-15 | EMC IP Holding Company LLC | Storage system with loopback replication process providing unique identifiers for collision-free object pairing |
US10983962B2 (en) | 2018-05-29 | 2021-04-20 | EMC IP Holding Company LLC | Processing device utilizing polynomial-based signature subspace for efficient generation of deduplication estimate |
US10977216B2 (en) | 2018-05-29 | 2021-04-13 | EMC IP Holding Company LLC | Processing device utilizing content-based signature prefix for efficient generation of deduplication estimate |
US11593313B2 (en) | 2018-05-29 | 2023-02-28 | EMC IP Holding Company LLC | Processing device configured for efficient generation of data reduction estimates for combinations of datasets |
US11609883B2 (en) | 2018-05-29 | 2023-03-21 | EMC IP Holding Company LLC | Processing device configured for efficient generation of compression estimates for datasets |
US10826990B2 (en) | 2018-07-23 | 2020-11-03 | EMC IP Holding Company LLC | Clustered storage system configured for bandwidth efficient processing of writes at sizes below a native page size |
US10684915B2 (en) | 2018-07-25 | 2020-06-16 | EMC IP Holding Company LLC | Efficient packing of compressed data in storage system implementing data striping |
US10635533B2 (en) | 2018-07-30 | 2020-04-28 | EMC IP Holding Company LLC | Efficient computation of parity data in storage system implementing data striping |
US10754559B1 (en) | 2019-03-08 | 2020-08-25 | EMC IP Holding Company LLC | Active-active storage clustering with clock synchronization |
US11137929B2 (en) | 2019-06-21 | 2021-10-05 | EMC IP Holding Company LLC | Storage system configured to support cascade replication |
US11079961B1 (en) | 2020-02-03 | 2021-08-03 | EMC IP Holding Company LLC | Storage system with write-via-hash functionality for synchronous replication of logical storage volumes |
US11360712B2 (en) | 2020-02-03 | 2022-06-14 | EMC IP Holding Company LLC | Storage system with continuous data verification for synchronous replication of logical storage volumes |
US11055028B1 (en) | 2020-02-03 | 2021-07-06 | EMC IP Holding Company LLC | Storage system with reduced read latency |
US11249654B2 (en) | 2020-02-18 | 2022-02-15 | EMC IP Holding Company LLC | Storage system with efficient data and parity distribution across mixed-capacity storage devices |
US11144232B2 (en) | 2020-02-21 | 2021-10-12 | EMC IP Holding Company LLC | Storage system with efficient snapshot pair creation during synchronous replication of logical storage volumes |
US11079969B1 (en) | 2020-02-25 | 2021-08-03 | EMC IP Holding Company LLC | Disk array enclosure configured for metadata and data storage processing |
US11281386B2 (en) | 2020-02-25 | 2022-03-22 | EMC IP Holding Company LLC | Disk array enclosure with metadata journal |
US11061618B1 (en) | 2020-02-25 | 2021-07-13 | EMC IP Holding Company LLC | Disk array enclosure configured to determine metadata page location based on metadata identifier |
US11144461B2 (en) | 2020-03-09 | 2021-10-12 | EMC IP Holding Company LLC | Bandwidth efficient access to persistent storage in a distributed storage system |
US11010251B1 (en) | 2020-03-10 | 2021-05-18 | EMC IP Holding Company LLC | Metadata update journal destaging with preload phase for efficient metadata recovery in a distributed storage system |
US11126361B1 (en) | 2020-03-16 | 2021-09-21 | EMC IP Holding Company LLC | Multi-level bucket aggregation for journal destaging in a distributed storage system |
US11194664B2 (en) | 2020-04-20 | 2021-12-07 | EMC IP Holding Company LLC | Storage system configured to guarantee sufficient capacity for a distributed raid rebuild process |
US11169880B1 (en) * | 2020-04-20 | 2021-11-09 | EMC IP Holding Company LLC | Storage system configured to guarantee sufficient capacity for a distributed raid rebuild process |
US11494301B2 (en) | 2020-05-12 | 2022-11-08 | EMC IP Holding Company LLC | Storage system journal ownership mechanism |
US11392295B2 (en) | 2020-05-27 | 2022-07-19 | EMC IP Holding Company LLC | Front-end offload of storage system processing |
US11093161B1 (en) | 2020-06-01 | 2021-08-17 | EMC IP Holding Company LLC | Storage system with module affinity link selection for synchronous replication of logical storage volumes |
US11513882B2 (en) | 2020-06-08 | 2022-11-29 | EMC IP Holding Company LLC | Dynamic modification of IO shaping mechanisms of multiple storage nodes in a distributed storage system |
US11886911B2 (en) | 2020-06-29 | 2024-01-30 | EMC IP Holding Company LLC | End-to-end quality of service mechanism for storage system using prioritized thread queues |
US11327812B1 (en) | 2020-10-19 | 2022-05-10 | EMC IP Holding Company LLC | Distributed storage system with per-core rebalancing of thread queues |
US11531470B2 (en) | 2020-10-21 | 2022-12-20 | EMC IP Holding Company LLC | Offload of storage system data recovery to storage devices |
US11853568B2 (en) | 2020-10-21 | 2023-12-26 | EMC IP Holding Company LLC | Front-end offload of storage system hash and compression processing |
US11436138B2 (en) | 2020-10-21 | 2022-09-06 | EMC IP Holding Company LLC | Adaptive endurance tuning of solid-state storage system |
US11616722B2 (en) | 2020-10-22 | 2023-03-28 | EMC IP Holding Company LLC | Storage system with adaptive flow control using multiple feedback loops |
US11314416B1 (en) | 2020-10-23 | 2022-04-26 | EMC IP Holding Company LLC | Defragmentation of striped volume in data storage system |
US11435921B2 (en) | 2020-11-19 | 2022-09-06 | EMC IP Holding Company LLC | Selective deduplication in a distributed storage system |
US11687245B2 (en) | 2020-11-19 | 2023-06-27 | EMC IP Holding Company LLC | Dynamic slice assignment in a distributed storage system |
US11494405B2 (en) | 2020-12-21 | 2022-11-08 | EMC IP Holding Company LLC | Lock contention resolution for active-active replication performed in conjunction with journal recovery |
US11481291B2 (en) | 2021-01-12 | 2022-10-25 | EMC IP Holding Company LLC | Alternative storage node communication channel using storage devices group in a distributed storage system |
US11875198B2 (en) | 2021-03-22 | 2024-01-16 | EMC IP Holding Company LLC | Synchronization object issue detection using object type queues and associated monitor threads in a storage system |
US11520527B1 (en) | 2021-06-11 | 2022-12-06 | EMC IP Holding Company LLC | Persistent metadata storage in a storage system |
US11775202B2 (en) | 2021-07-12 | 2023-10-03 | EMC IP Holding Company LLC | Read stream identification in a distributed storage system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000010738A (ja) * | 1998-06-17 | 2000-01-14 | Toshiba Corp | ディスクアレイシステム、同システムに適用される記憶容量拡張方法および記録媒体 |
US8527544B1 (en) * | 2011-08-11 | 2013-09-03 | Pure Storage Inc. | Garbage collection in a storage system |
WO2014184941A1 (ja) * | 2013-05-17 | 2014-11-20 | 株式会社日立製作所 | ストレージ装置 |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6785771B2 (en) * | 2001-12-04 | 2004-08-31 | International Business Machines Corporation | Method, system, and program for destaging data in cache |
US20100037091A1 (en) * | 2008-08-06 | 2010-02-11 | Anant Baderdinni | Logical drive bad block management of redundant array of independent disks |
US9268640B1 (en) * | 2013-12-20 | 2016-02-23 | Emc Corporation | Limiting data loss on parity RAID groups |
-
2015
- 2015-05-12 WO PCT/JP2015/063563 patent/WO2016181481A1/ja active Application Filing
- 2015-05-12 US US15/549,283 patent/US10621083B2/en active Active
- 2015-05-12 JP JP2017517503A patent/JP6385570B2/ja not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000010738A (ja) * | 1998-06-17 | 2000-01-14 | Toshiba Corp | ディスクアレイシステム、同システムに適用される記憶容量拡張方法および記録媒体 |
US8527544B1 (en) * | 2011-08-11 | 2013-09-03 | Pure Storage Inc. | Garbage collection in a storage system |
WO2014184941A1 (ja) * | 2013-05-17 | 2014-11-20 | 株式会社日立製作所 | ストレージ装置 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018173915A (ja) * | 2017-03-31 | 2018-11-08 | 日本電気株式会社 | バックアップシステム、ストレージ装置、データ転送方法及びプログラム |
CN110209340A (zh) * | 2018-02-28 | 2019-09-06 | 杭州宏杉科技股份有限公司 | 一种全闪存存储***的访问方法及装置 |
CN110209340B (zh) * | 2018-02-28 | 2022-08-02 | 杭州宏杉科技股份有限公司 | 一种全闪存存储***的访问方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
US20180095873A1 (en) | 2018-04-05 |
US10621083B2 (en) | 2020-04-14 |
JP6385570B2 (ja) | 2018-09-05 |
JPWO2016181481A1 (ja) | 2017-11-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6385570B2 (ja) | ストレージシステムおよび記憶制御方法 | |
JP6304406B2 (ja) | ストレージ装置、プログラム、情報処理方法 | |
US10152381B1 (en) | Using storage defragmentation function to facilitate system checkpoint | |
US8639669B1 (en) | Method and apparatus for determining optimal chunk sizes of a deduplicated storage system | |
JP6124902B2 (ja) | ストレージシステムにおける可変長符号化 | |
JP6240071B2 (ja) | ストレージシステムにおけるマッピングテーブルを効果的に管理するコンピューターシステムおよびその方法 | |
US8712963B1 (en) | Method and apparatus for content-aware resizing of data chunks for replication | |
US9229870B1 (en) | Managing cache systems of storage systems | |
JP5221756B2 (ja) | データ要素を圧縮して格納するストレージシステム及び記憶制御方法 | |
US10691354B1 (en) | Method and system of disk access pattern selection for content based storage RAID system | |
US20180267856A1 (en) | Distributed storage system, data storage method, and software program | |
US20190129971A1 (en) | Storage system and method of controlling storage system | |
US20120226672A1 (en) | Method and Apparatus to Align and Deduplicate Objects | |
US9507673B1 (en) | Method and system for performing an incremental restore from block-based backup | |
WO2014136183A1 (ja) | ストレージ装置及びデータ管理方法 | |
US10664397B2 (en) | Cache recovery method in a distributed storage system | |
US10713221B2 (en) | Dual layer deduplication for a file system running over a deduplicated block storage | |
JP6840113B2 (ja) | ストレージコントローラ及びストレージ制御方法 | |
US10929066B1 (en) | User stream aware file systems with user stream detection | |
KR101525453B1 (ko) | 데이터 접근빈도를 이용한 raid 시스템의 데이터 복제 방법 | |
US20200341848A1 (en) | Time Addressable Storage In a Content Addressable Storage System | |
WO2016006023A1 (ja) | ストレージ装置、計算機システム及び方法 | |
US10956366B2 (en) | Dynamic physical capacity allocation in an unbalanced CAS system | |
JP6419662B2 (ja) | ストレージシステム及びデータ重複検出方法 | |
WO2018061161A1 (ja) | ストレージ装置およびその制御方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15891806 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15549283 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2017517503 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15891806 Country of ref document: EP Kind code of ref document: A1 |